Community roundup: October 2022

October’s community meetup included a sneak preview of our new style of data sources—and more!

Erin Kapp
October 27, 2022
Erin Kapp
October 27, 2022
Great Expectations community roundup October 2022 New Data Sources cover card

We get the community together on the third Tuesday of every month—usually! This time, we met on the fourth Tuesday, October 25.

At meetups, we discuss the Great Expectations Roadmap, watch ecosystem integration demos, and explore different ways data leaders have implemented Great Expectations. Sign up here to join the next one! 

Now, let's dive into the roundup.

We covered:

  • Our new approach to data sources

  • Community update and celebrations

  • Cloud early adoption surveys

  • And more!

Feature demo

For October’s feature demo, Tal Gluck previewed our new style of data sources.

Watch the complete presentation here:

Or read on for a summary!

Data sources now

Right now, configuring a Datasource in GX involves use of a YAML file. This generic approach (YAML) to a specific job (connecting to a data source) lets us support a lot of scenarios!

But it has a few drawbacks, including that if your configuration falls off the rails it can be tricky to get it back on. Plus, if you personally aren’t working with a lot of data sources on a regular basis, the general approach isn’t that useful for you.

Goals of the new approach

  • Be more usable (including more Pythonic!) than the current way

  • Make it easier to work with multiples (including multiple Batches)

  • Make it easier to build and contribute new data sources

  • Be fully compatible with the old data sources–no breaking changes!

Some of the new features

  • Suggest-as-you-go autocomplete

  • Regex parser for automatically adding components of the filename to metadata

  • String sorter for assets

  • Batch name template

How it looks

Community meetup October 2022 screenshot

Coming next

The initial data source types that will be supported in the new way are Pandas and PostgreSQL. Once we have these two, we’ll continue to build out support for more sources.

We want community feedback, so if you’d like to be involved in testing the new-style data sources, ping @Tal Gluck in our Slack.

Cloud note

We’ve been sending out surveys about GX Cloud to the community! If you’re interested in being an early adopter, please respond to the survey or reach out on Slack to @Alina Weinstein or @Matthew Lundgren.

Join the conversation

Whether you’re a fledgling data practitioner or a seasoned data expert, our community welcomes you! Here are some conversations happening now: 

Additional updates

Have you done something cool with Great Expectations that you'd like to share? If you want to demo or have some data quality content you'd like us to feature, DM @Kyle Eaton on Slack.

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management

©2023 Great Expectations. All Rights Reserved.