Community roundup: December 2023

Featuring Lakehouse Engine plus GX for data quality, plans and feedback for GX 1.0, and more

Erin Kapp
December 19, 2023
December 2023 event cover with title 'Meet Lakehouse Engine: an enabler for GX in your Lakehouse platform by Jose Correia and Pedro Rocha' and two cartoon people looking engaged/thoughtful

At this month’s meetup, we: 

  • Shared the latest accomplishments of the community

  • Got a product roadmap update

  • Learned more about & gave feedback for GX OSS 1.0

  • Heard about using Lakehouse Engine with GX for better data

  • And more

Check out the recording if you missed it:

To join us in January, sign up here.

Thanks and kudos

We appreciate everything that our contributors do to make GX better: thank you all! 

This month, we highlighted these contributions:

  • There’s a new Expectation, ExpectMulticolumnDatetimeDifferenceToBeLessThanTwoMonths, thanks to kcs-rohankolappa.

  • A bug in MySQL deployments that was caused by reuse of temporary tables was fixed by FilipeJesus.

December 2023 featured GitHub contributors

Our Slack supporters are the members of the GX community who go above and beyond in engaging with others. For December, we’d like to recognize:

December 2023 top Slack supporters

Roadmap update

Hernan Alvarez, GX’s VP of product, shared the recent developments in GX and what’s coming up soon!

Recently, we’ve:

  • Continued building up our CRUD functionality, most recently with Expectation Suites and Checkpoints

  • Cleaned up the default object creation for GX Cloud power users who are working with the API

  • Added the ability to create Expectations from column descriptive metrics

Next up:

  • Time-based splitting for SQL tables in the GX Cloud UI

  • UI-based editing for Data Assets

  • Expectation change history in the GX Cloud UI


Mollie Pettit, GX's senior community manager, shared some announcements:

  • We’re temporarily pausing PRs until GX 1.0 is released! The work for GX 1.0 will be making major changes to the codebase, and we don’t want any contributors to put in time and effort only for their code to immediately get broken by a change. Please hold on to your ideas—we’re excited to begin accepting PRs again after the GX 1.0 release.

  • Our developer advocates will be taking a well-deserved break from December 25-29: during that time, you’ll see fewer posts from the GX team than normal in Slack and Discourse. We’ll be back in the office on January 2! 

Great Expectations 1.0

James Campbell, GX’s co-founder and CTO, took an in-depth dive into our plans for GX 1.0 and covered topics like:

  • The future of the Validator

  • Changes to Expectation instantiation

  • Dividing your data into subsets for validation

And more!

Get the details, view the code snippets, and listen as James talks with some community members about their workflows:

You can also see the code James presented in this PR.

If you have any thoughts or feedback based on James’ presentation or the example code, you can share it by tagging or DMing @Mollie Pettit in the GX Slack—she’ll make sure your input gets passed along to James and the rest of the team.

Meet Lakehouse Engine: an enabler for GX on your Lakehouse platform

José Correia (senior data engineer and team leader) and Pedro Rocha (data engineer) from Adidas gave an excellent and informative presentation about their open source framework Lakehouse Engine, which integrates with GX to enable data quality on your Lakehouse!

Watch their presentation here:

Additional updates

  • Next month’s community event is Tuesday, January 16, 2024 at noon ET! Click here to add it to your calendar.

  • Did you catch GX Champion Cesar Garcia Saez’s talk about using Python for data quality at the PyData Global Conference 2023? You can find the materials and code he used in his presentation in this GitHub repo.

  • We’re hiring! Check out our open positions for a product marketing manager and a senior software engineer: 

Have you done something cool with Great Expectations that you'd like to share? If you're interested in demoing or have a piece of data quality content that you'd like us to feature, DM @Mollie Pettit on our Slack.

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management

©2024 Great Expectations. All Rights Reserved.