Greater heights: new and updated Expectations
Thanks to long-time contributor @HadasManor and first-time contributor @eden-o for their work on Expectations this month!
Peaks:
[CONTRIB] expect_queried_column_pair_values_to_be_both_filled_or_null (#7949) (thanks @eden-o)
[FEATURE] ExpectDaySumToBeCloseToEquivalentWeekDayMean: User can give weeks_back as argument (#8139) (thanks @HadasManor)
[BUGFIX] expect_day_count_to_be_close_to_equivalent_week_day_mean (#7782) (thanks @HadasManor)
Click click boom: Clickhouse integration
Clickhouse integration has been the subject of many requests, and it’s finally here thanks to Plozano94!
[FEATURE] Clickhouse Integration (#7719) (thanks @plozano94)
Ready, set, go: upgraded support and compatibilities
GX is now compatible with Pandas 2.0.0 and SQLAlchemy 2.0.0, and now has initial support for Python 3.11! The Prefect tutorial has also been updated for Prefect 2.
Starters:
[FEATURE] Pandas 2.0.0 and Sqlalchemy 2.0.0 compatibility (#7633)
[FEATURE] Python 3.11 support (#8174)
[DOCS] Update prefect gx tutorial (#8009) (thanks @discdiver)
[MAINTENANCE] Pandas and SqlAlchemy 2.0 follow-up (#8112)
Living standards: new reference environments
Reference environments will make it easier for you to get started with the associated backend, plus help with testing and debugging.
Habitats:
[FEATURE] Reference Environment - Snowflake (#8148)
[FEATURE] AWS S3 reference environment (#8166)
[FEATURE] Reference Environment - BigQuery (#8176)
[FEATURE] Reference Environment - Google Cloud Storage (#8197)
[FEATURE] Reference Environment: Azure Blob Storage (#8212)
Sprucing things up: other new features
[FEATURE] Add scaffolding for gx-agent (#7837)
[FEATURE] Add Onboarding Data Assistant agent action (#7882)
[FEATURE] Connect to Cloud for agent config (#8006)
[FEATURE] Fluent Datasource ABS, GCS, and S3 recursive file discovery (#8118) (thanks @toivomattila)
[FEATURE] Snowflake FDS (#8157)
New chapters: documentation updates
[FEATURE] Checker for correct Python code snippets in documentation. (#8000)
[DOCS] Create New Landing Pages (#7993)
[DOCS] Create New Expectation Classes Conceptual Guide (#8004)
[DOCS] Adds redirects for removed UserConfigurableProfiler documentation (#8026)
[DOCS] Update Quickstart guide to be Cloud-compatible (#8036)
[DOCS] Update How to create Expectations Interactively in Python (#8052)
[DOCS] Update Links in How to use auto-initializing Expectations (#8054)
[DOCS] Add guidance for "DataFrameAsset.build_batch_request()". (#8069)
[DOCS] Getting Started with GX (#8143)
Pest control: other bug fixes
[BUGFIX] fix issue-7954 (#7963) (thanks @James Kingdon)
[BUGFIX] Ensure That DataFrame for Fluent Datasource Pandas/Spark DataFrame Data Assets is specified only in one API method (#8032)
[BUGFIX] respect result format bool only for validators and checkpoints (#8111)
[BUGFIX] Fix config substitution for substrings (#8145)
[BUGFIX] Fix 'great_expectations[cloud]' extra install (#8151)
[BUGFIX] Allow for `create_temp_table` to be `False` when creating a `SqlAlchemyBatchData` object (#8160)
[BUGFIX] Persist SQLDatasource splitters on creation (#8164)
[BUGFIX] Raise more informative error on failure of getting an expectation suite by name (#8170)
Upkeep: additional maintenance
[MAINTENANCE] Fluent Datasource Update: Credentials Doc update (#7968)
[MAINTENANCE] Deleting References to UserConfigurableProfiler Documentation (#7983)
[MAINTENANCE] Add support for Validator in Checkpoint CRUD (#7999)
[MAINTENANCE] Update default action list in Checkpoint based on user environment (#8074)
[MAINTENANCE] Delete LegacyCheckpoint (#8082)
[MAINTENANCE] Typing Data Assistant Result (#8110)
[MAINTENANCE] Add default table_name to TableAsset if omitted (#8152)
[MAINTENANCE] Python 3.11 SQLAlchemy import time fix (#8180)
[MAINTENANCE] Rich comments and docstrings for DataAssistant related modules. (#8185)
Get the full GX changelog here.
