Community Roundup: September 2022

September’s community meetup included a demo of Data Assistants—our new feature for getting started quickly with GX data quality—and more!

Erin Kapp
September 22, 2022
Erin Kapp
September 22, 2022
Great Expectations demo of Data Assistants cover card

We get the community together on the third Tuesday of every month. At each meetup, we discuss the Great Expectations Roadmap, watch ecosystem integration demos, and explore different ways data leaders have implemented Great Expectations. Sign up here to join the next one! 

Now, let's dive into the roundup.

Feature Demo

For September’s feature demo, James Campbell, Great Expectations co-founder and CTO, shared our new GX feature, Data Assistants! James answered questions including:

Why did we make Data Assistants? 

Data Assistants help you solve the “pale blue dot problem”—when you’re staring down all the ways you can approach data quality, where do you even start? Data Assistants get you moving with curated, relevant Expectations.

What is a Data Assistant?

You can think of a Data Assistant as a partner that can help you understand the shape of your data within minutes, transforming tacit knowledge into something that can be explicitly shared across the business—not a black box that tries to guess your intent. 

Data Assistants make observations about your initial data and propose Expectations based on the data’s characteristics. This creates a comprehensive and semantically-grounded picture of your data that you can use to guide your next steps.

How do Data Assistants work?

Data Assistants wrap functionality from rule-based profilers into rapid, one-shot profiling. Their pre-packaged rules have simple parameters but have been carefully selected to guide users about what to explore in their data, what is (or isn’t) ready for use, and other insights.

How do I use Data Assistants today?

A Data Assistant can be run in two different modes, to support different parts of your process.

For your processed/normalized data, Data Assistants can help you launch your data quality project quickly. The Data Assistant will build Expectations based on the data you know and are confident about, which you can immediately start using on data that you expect to be similar.

(But don’t forget to bring in your SMEs—they’re key to identifying which Expectations are most useful and whether the defaults need adjustment!)

For your new/unprocessed data, Data Assistants are great for bootstrapping your initial data exploration. The Data Assistant builds Expectations that the initial data already fails, identifying areas with significant variances or other reasons for further investigation.

(Why is this better than anomaly detection? Because it’s more transparent and you understand how it works, so you have more control.)

Download the slides here: Announcing Data Assistants

And access the documentation here: 

Questions? Feedback? Jump into our Slack channel or this GitHub Discussion 

Roadmap Update

We covered:

  • Data Assistants launch today!

    • What’s in a launch? Gathering user feedback, documentation updates, and updates in Slack. 

  • Next up: Improvements to DataContext, the heart of Great Expectations, plus updates to Datasources that will increase user-friendliness and move toward better batch metrics.

  • @tal in Slack if you’d like to be part of the developer/user experience journey.

  • And more!

Join the Conversation

Whether you’re a fledgling data practitioner or a seasoned data expert, our community welcomes you! Here are some conversations happening now: 

Additional Updates

Have you done something cool with Great Expectations that you'd like to share? If you want to demo or have some data quality content you'd like us to feature, DM @kyle on Slack.

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management

©2023 Great Expectations. All Rights Reserved.