At this month’s meetup, we:
Learned about the experimental Missingness Data Assistant
Explored Fluent Datasources through flowcharts
Heard from community member Bogdan Volodarskiy about how a team at Provectus uses GX to implement a data quality gate
We also learned about work by this month’s contributors, the latest on the product roadmap, and more.
Catch up with the recording:
Sign up here to join the next one!
Thanks and kudos
Kudos to all of our Slack supporters for August: you’re a vital part of the GX community!
We want to especially recognize the work that José Correia and calvingdu did this month:
GX users can now use the
The Expectation expect_column_values_to_be _a_valid_currency_code is now available in Spark as well as Pandas with calvingdu’s contribution.
Thank you to everyone who contributed to GitHub in August:
ICYMI: the GX Discourse
If you have a question about GX, check out our new Discourse forum!
Discourse has two big advantages over Slack as a knowledgebase. First, Discourse’s messages are publicly searchable when Slack’s aren’t, making it easier for more users to find answers to their questions. And second, Discourse doesn’t have a message retention limit, whereas Slack only keeps messages for 90 days on the free plan.
To take advantage of those factors, we’ve revamped the GX Discourse forum to be a more welcoming and effective place for Q&A! The GX Slack will be a place for GX community members to connect in a more social setting.
We know not everyone currently in Slack will be willing or able to move to Discourse right away. So we’ve established an automated cross-posting pipeline to keep Q&As up-to-date across platforms. Haebichan reviewed how the integration works in the meetup:
Check out the Discourse at https://discourse.greatexpectations.io!
Roadmap update
We’ve added a new experimental Data Assistant: the Missingness DA, which we also introduced in a demo later in the meetup. Your feedback is appreciated!
Product manager Erik Hencier also previewed some of the upcoming work on GX:
If you’d like to check out Cloud yourself, sign up for the Beta here!
Fluent Datasource flowcharts
Developer advocate and stand-in meetup host Haebichan Jung showed some flowcharts that dive into the inner workings of a Fluent Datasource. He covers both a basic implementation and a more advanced use case:
If you have questions, @Haebichan Jung is happy to talk! Reach out to him in the Feedback section of the Discourse or the #gx-feedback channel in Slack.
Missingness Data Assistant
Software engineer Thu Pham presented the new experimental Missingness Data Assistant! Check out her demo of the Missingness DA:
Up next for iteration on the Missingness DA are the visualizations and the ability to trigger the Missingness DA from Cloud.
Give it a try and let us know what you think!
Community demo: DQG: ‘one-click’ Data Quality Gate for your project (Bogdan Volodarskiy)
Community member Bogdan Volodarskiy of Provectus gave a use case demo of the Data Quality Gate, a Terraform module that lets you deploy Provectus’ data quality solution (including GX) in one click.
Watch the full presentation—including a checklist to help you assess when Data Quality Gate is going to be most helpful to you—here:
You can visit the Data Quality Gate on GitHub, or see the slides from Bogdan’s presentation here.
Join the conversation
Matt Palmer shared an invite to the first-ever Bay Data Club low-key happy hour.
James Campbell reshared an article from the WSJ about the toll that data quality for AI takes on human workers.
Ed Zahurak is looking for an easy way to return a dataframe of the data in a particular datasource or asset.
Lmandruzzato is wondering about ways to optimize Batch requests to fully leverage pruning in BigQuery.
Additional updates
Next month we’re meeting on Tuesday, September 19. Get the invite here: https://greatexpectations.io/meetup.
We have a new guide to GX’s community spaces, technical support, and more.
If you’re thinking of creating a Custom Expectation, check out our introduction to Metrics in GX.
We’re hiring a senior product manager! Please share with your networks and apply if you’re interested.
Have you done something cool with Great Expectations that you'd like to share? If you're interested in demoing or have a piece of data quality content that you'd like us to feature, DM @Mollie Pettit on our Slack.
