Batch Request API (v3): The New Standard Deployment of Great Expectations

We are promoting the v3 or Batch Request API to be the default API for interacting with Datasources in Great Expectations
Written By  Spencer Hardwick and James CampbellDecember 14, 2021

Join the Community Meetup Tuesday, December 21, 12:00pm EST for more details and Q&A

We are Promoting the Batch Request API!

An exciting day for the Great Expectations team is coming: we’re going to promote the “v3” or “Batch Request” API to be the default API for interacting with Datasources in Great Expectations. That’s a big deal for us--it unlocks really powerful ways of reasoning about data in multiple batches--but it’s probably not a big deal for you. In fact, if you’ve been using Great Expectations at all in the past year (and who hasn’t!?), you’ve already been using parts of this new API under the hood.

Making the Batch Request API the default will mean that the great expectations CLI tool will no longer require the --v3-api flag to use the new features; since that means the change will break scripted CLI operations, we’re going to release this change with the 0.14 version number.

We’ll share more about the features and capabilities of the Batch Request API over the coming months, but you don’t need to wait to upgrade! We have published a migration guide to assist with updating your installation. As always, if you have any questions, please let us know. Thanks to the teams at Lifespark and Provectus for working closely with us to refine and improve the migration guide and tools that back it up!

Ok, so what’s actually different?

There are two key things that changed in this new API: modular expectations and data connectors.

With our new modular design, all of the Expectations in the library are completely pluggable at runtime. That makes it really easy to extend Great Expectations with domain-specific Expectations, change how Expectations look in data docs, or contribute a new Expectation yourself.

Modular Expectations are also the part of the new API that you’ve already been using: all builds of data docs have been using our new modular Expectations framework for over a year. (And while you’ll probably never need to notice this part unless you’re actually adding new Expectations, we’d love to brag a bit: modular Expectations also make it easier for us to add additional metrics and even run validation jobs more efficiently by reusing some computations).

DataConnectors let you tell Great Expectations how you’ll be accessing your data. The migration guide covers how to switch your current Datasources to use DataConnectors, and it’s a really straightforward process. We’re really excited about how the change will let us build new features in the future--like profilers that look across multiple batches of data--but until you’re ready to use those capabilities, you won’t even notice the change.

Making the Batch Request (“v3”) API default means we can focus all our development efforts on features that use this API. That’s a big deal for our team; while you’ll see less energy on bug fixes or feature extensions for the old API, we’re working hard to make Great Expectations easier to adopt and use as we help you on your data quality journey. We hope you’ll love it!

If you have additional feedback or discover something unusual during your migration, please reach out to our Developer Relations team on our Slack

Join the Community Meetup Tuesday, December 21, 12:00pm EST for more details and Q&A

Like our blogs?

Join our newsletter and get more blogs and news

Great Expectations Newsletter and Updates Sign-up

Hello friend of Great Expectations!

Our newsletter content will feature product updates from the open-source platform and our upcoming Cloud product, new blogs and community celebrations.

Please let us know what matters to you in regards to your use (or potential use) of Great Expectations below. We want to make sure that we keep you informed and notified only about what matters to you.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

We are hiring! Please check out our job board here:

Great Expectations

Developed By

Netlify Logo
Brought to you by the Superconductive TeamCopyright ©2020 Great Expectations