Join the Community Meetup Tuesday, December 21, 12:00pm EST for more details and Q&A
We are Promoting the Batch Request API!
An exciting day for the Great Expectations team is coming: we’re going to promote the “v3” or “Batch Request” API to be the default API for interacting with Datasources in Great Expectations. That’s a big deal for us--it unlocks really powerful ways of reasoning about data in multiple batches--but it’s probably not a big deal for you. In fact, if you’ve been using Great Expectations at all in the past year (and who hasn’t!?), you’ve already been using parts of this new API under the hood.
Making the Batch Request API the default will mean that the great expectations CLI tool will no longer require the
We’ll share more about the features and capabilities of the Batch Request API over the coming months, but you don’t need to wait to upgrade! We have published a migration guide to assist with updating your installation. As always, if you have any questions, please let us know. Thanks to the teams at Lifespark and Provectus for working closely with us to refine and improve the migration guide and tools that back it up!
Ok, so what’s actually different?
There are two key things that changed in this new API: modular expectations and data connectors.
With our new modular design, all of the Expectations in the library are completely pluggable at runtime. That makes it really easy to extend Great Expectations with domain-specific Expectations, change how Expectations look in data docs, or contribute a new Expectation yourself.
Modular Expectations are also the part of the new API that you’ve already been using: all builds of data docs have been using our new modular Expectations framework for over a year. (And while you’ll probably never need to notice this part unless you’re actually adding new Expectations, we’d love to brag a bit: modular Expectations also make it easier for us to add additional metrics and even run validation jobs more efficiently by reusing some computations).
DataConnectors let you tell Great Expectations how you’ll be accessing your data. The migration guide covers how to switch your current Datasources to use DataConnectors, and it’s a really straightforward process. We’re really excited about how the change will let us build new features in the future--like profilers that look across multiple batches of data--but until you’re ready to use those capabilities, you won’t even notice the change.
Making the Batch Request (“v3”) API default means we can focus all our development efforts on features that use this API. That’s a big deal for our team; while you’ll see less energy on bug fixes or feature extensions for the old API, we’re working hard to make Great Expectations easier to adopt and use as we help you on your data quality journey. We hope you’ll love it!
If you have additional feedback or discover something unusual during your migration, please reach out to our Developer Relations team on our Slack
Join the Community Meetup Tuesday, December 21, 12:00pm EST for more details and Q&A
