What’s new in GX: July 2023

Going to greater lengths: new Expectations

The Expectation Gallery gained three new Expectations this month and added a new backend for another, with two contributions by the community!

Achievements: 

• [CONTRIB] Contributed expect_column_to_have_no_months_missing (#8307) (thanks @HadasManor)

• [CONTRIB] Create custom expectation for Chi Square Test (#8314)

• [CONTRIB] Add KS test custom expectation (#8344)

• [CONTRIB] Pyspark Implementation for expect_column_values_to_be_valid_currency_code (#8418) (thanks @calvingdu)

Education expansion: new conceptual guides

These new explainers have a ton of useful information for new and veteran GX users alike.

Subjects:

• [DOCS] Adds a consolidated overview of using GX as a conceptual guide (#8045)

• [DOCS] Add MetricProviders Conceptual Content (#8175)

• [DOCS] Docs TOC Reorg (#8421)

Looking it up: reference environments

Reference environments will improve our testing and debugging capabilities, and make it easier to get started with the corresponding backend! This month, we released new reference environments for AWS RDS and Airflow; we also enabled Data Doc hosting within some existing environments.

Footnotes: 

• [FEATURE] Host data docs for the Postgres reference environment (#8221)

• [FEATURE] Reference Environment for AWS RDS (#8222)

• [FEATURE] View data docs inside Snowflake and BigQuery reference environments (#8231)

• [FEATURE] Airflow reference environment (#8257)

Sprucing things up: other new features

• [FEATURE] DatabricksSQL FDS (#8184)

• [FEATURE] Missingness DataAssistant initial implementation (#8268)

• [FEATURE] single batch missingness data assistant (#8278)

• [FEATURE] Register MissingnessDataAssistant (#8337)

• [FEATURE] Allow a rule to fail within DataAssistant and still run other rules (#8393)

New chapters: documentation updates

• [DOCS] Improvements on how_to_edit_expectationsuite guide (#8096) (thanks @Ismar11)

• [DOCS] Replaces outdated integration guides with redirects to versioned documentation (#8173)

• [DOCS] update result_format configuration documentation (#8209)

• [DOCS] Update docs on expectation docstrings (#8405)

• [DOCS] Fix markdown link in install_gx.md (#8416) (thanks @jmorakuebler)

• [DOCS] corrects default value for result_format in documentation (#8419)

• [DOCS] Fix snippet reference in how_to_use_great_expectations_with_sql.md (#8430) (thanks @jmorakuebler)

Pest control: other bug fixes

• [BUGFIX] Ensure DataAssistantResult.plot_expectations_and_metrics does not raise exceptions when no Metrics or Expectations are available to plot (#8238)

• [BUGFIX] Ensure that GX supports database column names in lowercase and mixcase for Oracle, DB2, and Snowflake (#8345)

• [BUGFIX] fixing minor bug in sqlalchemy_execution_engine.py (#8374) (thanks @SaeedFarahani)

Upkeep: additional maintenance

• [MAINTENANCE] Improvement to contributor documentation (#8043) (thanks @christian-bromann)

• [MAINTENANCE] Support individual connection args for Snowflake FDS (#8183)

• [MAINTENANCE] Replace dynamic datasource deletion with single

  delete
  method (#8189)

• [MAINTENANCE] mypy 1.4.1 (#8226)

• [MAINTENANCE] Refactor

  Table.head()
  for
  sqlalchemy
  (#8234)

• [MAINTENANCE] Add marker cli to all cli tests. (#8286)

• [MAINTENANCE] Fix requirements and CONTRIBUTING_CODE doc (#8312)

• [MAINTENANCE] Experimental column descriptive metrics repository (#8335)

• [MAINTENANCE] conditional

  snowflake-connector-python
  version bump (#8412)

Get the full GX changelog here.