backgroundImage

What's new in GX: February 2023

New Expectations, DataContext usability improvements, new AWS integration and API documentation, SQL features and fixes, and more

Erin Kapp
February 28, 2023
Catch up on the GX release notes. Also released into the wild: this black-footed ferret. (📸: Ryan Moehring, US FWS, 2016)
Catch up on the GX release notes. Also released into the wild: this black-footed ferret. (📸: Ryan Moehring, US FWS, 2016)

Growing greatness: new and updated Expectations

February featured five community-contributed Expectations!

They’re more than good:

  • [CONTRIB] added new Expectations  - India_zip_code Expectation and not_to_be_future_date Expectation (#6086) (thanks @prachijain136)

  • [CONTRIB] expect_multicolumn_values_not_to_be_all_null (#6912) (thanks @yussaaa)

  • [CONTRIB] Add experimental expectation to check column values after split (#7120) (thanks @ace-racer)

  • [FEATURE] Add expectation_column_values_to_be_continuous (#5861) (thanks @jmoskovc)

  • [FEATURE] ID/PK ColumnPairExpectations and MultiColumnMapExpectations - Pandas (#6941)

  • [FEATURE] ID/PK ColumnPairExpectations and MultiColumnMapExpectations - Spark (#7001)

  • [FEATURE] ID/PK ColumnPairExpectations and MultiColumnMapExpectations - SQL (#7046)

Making more sense in (Data)Context

Usability updates to DataContext standardize the behavior of CRUD methods for all DataContext objects. You can get more details about these changes in the February community meetup!

Clarifications:

  • [DOCS] Add new DataContext CRUD to public API (#7058)

  • [MAINTENANCE] Deprecate old DataContext CRUD methods (#7031)

  • [MAINTENANCE] Update update_ methods in DataContext to return persisted object (#7034)

Adding some polish: documentation improvements

Four new AWS integration guides are the crown jewels of this month's documentation improvements, along with more additions to the API docs.

Other gems: 

  • [DOCS] Add displayHTML method to view Data Docs (#7125) (thanks @swittchawa)

  • [DOCS] Fix capitalization of Slack (#7136) (thanks @JoelGritter)

  • [DOCS] Use named snippets for improved docs (#7214)

Better than the original: SQL and SQLAlchemy updates

Check out the series of SQL- and SQLAlchemy-related features and fixes that GX got this month.

Installments:

  • [CONTRIB] Row condition parser sqlalchemy: adding support for != operator & adding support all operators for string (#7053) (thanks @maayaniti)

  • [FEATURE] F/great 1393/add initial non datetime sql splitters (#7183)

  • [FEATURE] Add SQL query data asset for new experimental datasources (#6999)

  • [BUGFIX] Snowflake/Oracle/DB2 <--> SQLAlchemy table and column names case insensitivity representation (#6951)

  • [MAINTENANCE] Fix incorrect label on "How to configure a SQL Datasource" docs page (#7106)

  • [MAINTENANCE] Refactor sql splitter to take selectable instead of str. (#7133)

Sprucing things up: other new features

  • [FEATURE] Add API action (#6902) (thanks @itaise)

  • [FEATURE] Enable

    gx.get_context()
    to work without any inputs (#7074)

  • [FEATURE] Utilize DataConnector in service of new DataAsset implementations (#7094)

  • [FEATURE] Enable customization of candidate Regular Expression patterns when running OnboardingDataAssistant (#7104)

Pest control: other bug fixes

  • [BUGFIX] Fix jsonschema - altair conflict (#6984)

  • [BUGFIX] Parse pandas version correctly for development builds (#7147)

  • [BUGFIX] ID/PK Spark and Sql fall back when

    unexpected_index_column_names
    have not been defined (#7150)

  • [BUGFIX] pydantic>=1.10.4 - ImportError: cannot import name dataclass_transform (#7163)

  • [BUGFIX] : Allow CLI to work with

    RuntimeDataConnector
    (#7187) (thanks @luke321321)

Upkeep: additional maintenance

  • [MAINTENANCE] Replace

    regex
    with
    batching_regex
    for fluent filesystem-like datasources (#7207)

  • [MAINTENANCE] Make Pandas installation with Python 3.10 less restrictive (#7013)



Get the full GX changelog here.

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management


©2024 Great Expectations. All Rights Reserved.