What's new in GX: March 2023

New Expectations, fluent Datasouce connection work, documentation updates, a SQLAlchemy 2.0 hotfix, and more

Erin Kapp
March 29, 2023
Erin Kapp
March 29, 2023
Two monk seal pups exit an enclosure on the beach and head toward the ocean
Catch up on what’s new out there with the GX release notes. Also released into the wild: these monk seals. (📸: Pacific Islands Fisheries Science Center, 2019)

Going great: new and updated Expectations

March had three community-contributed Expectations!

Looking good:

  • [CONTRIB] expect_multicolumn_values_not_to_be_all_null (#6912) (thanks @yussaaa)

  • [MAINTENANCE] Time series expectations [#7182]

  • [CONTRIB] expect_multicolumn_sum_values_to_be_equal_to_single_column (#7224) (thanks @swittchawa)

  • [CONTRIB] Feature/expect column values to match thai (#7238) (thanks @swittchawa)

  • [MAINTENANCE] Enable snowflake V3 Expectations Tests (#7370)

  • [CONTRIB] Limit results for two expectations (#7403) (thanks @itaise)

Smooth moves: fluent Datasource connection work

Check out the March community roundup for a demo of fluent-configuration Datasources!

Sliding into a release near you:

  • [FEATURE] Fluent CRUD operation stubs (#7347)

  • [FEATURE] Implement DataBricks (DBFS) DataConnector for Fluent Datasources needs (#7355)

  • [FEATURE] Fluent SQLDatasource accepts arbitrary kwargs (#7394)

  • [FEATURE] Fluent SQLDatasource create_temp_table (#7407)

  • [FEATURE] Fluent SparkDataframeDatasource with DataframeDataAsset (#7425)

New chapters: documentation updates

Notable items:

  • [DOCS] Versioning instructions and fix references for versioning [#7221]

  • [DOCS] Scan yaml examples for class names that should be part of the public API [#7267]

  • [DOCS] Fix unsupported chars in API docs [#7310]

  • [DOCS] Add instructions to cli for creating data assets (#7326)

  • [DOCS] DOC-473 Adds guide "How to set up GX to work with data on GCS" (#7408)

  • [DOCS] DOC-473 Adds guide "How to set up GX to work with SQL databases" (#7409)

  • [DOCS] Put code examples under test (#7417)

Coming in hot: hotfix for SQLAlchemy 2.0

We recommend using SQLAlchemy 1.4 until 2.0 is fully supported. However, if you are using 2.0, hotfix 0.16.3 fixes this show-stopping issue:

  • [BUGFIX] Fix LegacyRow import. (#7446)

Sprucing things up: other new features

  • [FEATURE] Improve Trino types support (#6588) (thanks @ms32035)

  • [CONTRIB] -add athena credential support (#7186) (thanks @tmilitino)

  • [FEATURE] EphemeralDataContext.convert_to_file_context() [#7175]

  • [FEATURE] PandasDatasource in-memory DataFrameAsset [#7280]

  • [FEATURE] Enable creating Checkpoint and SimpleCheckpoint with Validator [#7275]

  • [CONTRIB] Adding support for date for the row condition parser (#7359) (thanks @maayaniti)

Pest control: other bug fixes

  • [BUGFIX] Adding exception logging to store-related failures (#7202) (thanks @ciguaran)

  • [BUGFIX] Remove Query ID from exception for query.template_values metric (#7373) (thanks @itaise)

  • [BUGFIX] Fix bug with case sensitive execution env (#7393)

Upkeep: additional maintenance

  • [MAINTENANCE] Parallelize integration tests - move test definitions (#7345)

  • [MAINTENANCE] Clean up miscellaneous deprecated code (#7358)

  • [MAINTENANCE] Validate ExpectationConfiguration before adding to suite (#7366)

  • [MAINTENANCE] remove imprecise wording in the new datasource notebook (#7369)

  • [MAINTENANCE] Make sure sqlalchemy 2.0 warnings are emitted when running pipelines (#7379)

  • [MAINTENANCE] Change for connection.execute() for SQLAlchemy 2 compatibility (#7384)

  • [CONTRIB][MAINTENANCE] Custom query expectation and editing query.template_values metric (#7390) (thanks @mantasmy)

  • [MAINTENANCE] Bump SQLAlchemy lower bound to 1.4.0 (#7413)

  • [MAINTENANCE] only provide py.typed files for fully typed sub-packages (#7438)

Get the full GX changelog here.

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management

©2023 Great Expectations. All Rights Reserved.