backgroundImage

March 2024: community roundup

Featuring GX 1.0 usability updates, a demo of Postgres with GX Cloud, and more

Erin Kapp
March 21, 2024
Great Expectations community roundup March 2024 cover card

At this month’s meetup, we:

  • Learned what’s coming next for GX in a product roadmap update

  • Heard details about the latest usability updates in GX 1.0

  • Saw a demo of GX Cloud with a Postgres data source

  • And more

If you missed it, you can catch up on YouTube:

 

Follow our calendar to join future meetups!

Community celebration

GX’s vibrant community is one of the things that sets it apart. Thank you to all the community members who responded to questions or otherwise gave a helping hand to other users last month:

March 2024 top Slack supporters

We especially want to recognize GX Champion Toivo Mattila for going above and beyond in the past couple of months! 

Community talk

If you haven’t seen it yet, be sure to take a look at this talk that César García, another of our GX Champions, delivered at PyData Global! It’s about using GX for data quality on open data:

In this session we will demonstrate how to measure and improve the quality of open data using the open source Python library Great Expectations. Attendees will learn quality testing techniques and methodologies to prepare high-quality longitudinal datasets using Open Data from cities and regional portals.

 

Thanks for sharing this, César!

Roadmap

Anna Barr, one of GX’s senior product managers, shared what’s been new in GX lately and what’s coming up next.

Two items of special note:

  • We’re working on improving the getting-started experience for GX Cloud! If you’ve tried it out and have any feedback about your initial experiences, we’d love to hear them. You can reach out to @Anna Barr on Slack or Discourse, or email our product team at product_feedback@greatexpectations.io.

  • We introduced Fluent Data Sources just under a year ago and deprecated the old block style of Data Source configuration. In the very near future, we’ll be taking the final step of removing the code that supports block style from GX OSS. If you have Data Sources that aren’t yet using fluent configuration, be sure to migrate in the next couple of weeks!

GX 1.0 ease of use improvements

James Campbell, GX's CTO and co-founder, reviewed some of the major changes in GX 1.0. 

⚠️ Important note: These improvements are not live and GX 1.0 is still in active development! These names and implementations could still change.

In 1.0, Expectations use class names instead of method names. For example,

expect_column_values_to_be_in_set
will become
ExpectColumnValuesToBeInSet
.

The Validator has also been removed: GX 1.0 will have specific, separate Batch Definitions and Validation Definitions. Batch Definitions are attached to Data Assets, while Validation Definitions associate Batch Definitions with Expectation Suites. 

Validation Definitions can be created independently of a Checkpoint, which is also a major departure from pre-1.0 GX.

Speaking of Checkpoints, the Checkpoint API has been radically simplified:

  • It’s declared using only two parameters: the Validation Definition(s) you want to run, and any Actions that you want to take based on the results.

  • When you run a Checkpoint, you define any dynamic Batch or Expectation Suite parameters that are needed.

As part of the streamlining process, some things have been removed from the GX 1.0 API entirely:

  • The

    add_or_update
    methods have been removed in favor of explicit add, save, and delete options.

  • Convenience methods on Data Contexts (such as

    context.suites.get
    ) have also been removed because they were confusing.

  • The Dataset and ValidationOperator APIs are being removed: they’ve been effectively deprecated for years, and their functionality is replicated elsewhere.

  • The Profiler and Data Assistant APIs are being removed so we can reintroduce profiling from scratch. The new profiling functionality will reflect the lessons we learned from the Profiler and Data Assistant implementations and in creating the new API.

GX 1.0 will still have some experimental features (like

row_condition
), but GX 1.0 will be clearer about their experimental status.

GX Cloud + PostgreSQL

Austin Robinson, one of GX's developer advocates, showed off a demo of connecting to Postgres in the GX Cloud UI:

You can try it yourself by signing up for GX Cloud, or sign up for our upcoming GX Cloud + Postgres workshop.

Public preview recap

Mollie Pettit, GX’s senior community product manager, recapped our February 21 public preview event. You can watch it anytime on YouTube:

 


Click here to learn more about public preview.

Additional updates

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management


©2024 Great Expectations. All Rights Reserved.