Increasing greatness: new and updated Expectations
Contributors @mkopec87 and @HadasManor have been busy this January, with 9 merged PRs between them—all growing and improving the available Expectations!
Pieces of great:
[CONTRIB] Improve contrib schwifty expectations (#6812) (thanks @mkopec87)
[CONTRIB] Added condition to ExpectQueriedColumnListToBeUnique (#6702) (thanks @maayaniti)
[CONTRIB] Implement Spark backend for several expectations (#6683) (thanks @mkopec87)
[CONTRIB] Improve Spark backend support for contrib query based expectations (#6733) (thanks @mkopec87)
[CONTRIB] Refactor ExpectColumnValuesToBeHexadecimal expectation to be RegexBased (#6734) (thanks @mkopec87)
[CONTRIB] Fix regex based expectations for spark (#6725) (thanks @mkopec87)
[FEATURE] Added new expectation: ExpectYesterdayCountComparedToAvgEquivalentDaysOfWeek… (#6622) (thanks @HadasManor)
[FEATURE] ExpectDaySumToBeCloseToEquivalentWeekDayMean (#6664) (thanks @HadasManor)
[BUGFIX] : expect_day_count_to_be_close_to_equivalent_week_day_mean (#6811) (thanks @HadasManor)
[MAINTENANCE] Enable more backends for some contrib expectations (#6775)
[MAINTENANCE] More backends for expect_yesterday_count_compared_to_avg_equivalent_days_of_week (#6790)
[MAINTENANCE] expect day count to be close to equivalent week day mean (#6680) (thanks @HadasManor)
Time for a readathon: documentation expansion
GX has had an abundance of documentation updates in January. The biggest is the overhaul to our public API documentation… and we’re not done yet. Expect to see the API docs continue to improve. (Check out the blog about the big update push.)
Citations:
[FEATURE] API docs (#6766)
[MAINTENANCE] Use docusaurus style code block in api docs (#6906)
[BUGFIX] Fix issues rendering code blocks in API docs (#6917)
[DOCS] Add anonymous_usage_statistics configuration in documentation (#6626) (thanks @tmilitino)
[DOCS] fixing wrong line reference on docs (#6599) (thanks @wagneralbjr)
[DOCS] edit term(data_conext, checkpoints)-link in with airflow (#6646) (thanks @jx2lee)
[DOCS] Change prefix reference for tutorial folder/directory (#6751) (thanks @medeirosthiago)
[DOCS] Fix line-links in 4th step's 5th and 6th block (#6752) (thanks @OnkarMadli)
[DOCS] Fixed code reference in documentation (#6732) (thanks @tmilitino)
[DOCS] Change prefix reference for tutorial folder/directory (#6751) (thanks @medeirosthiago)
[DOCS] Fix line-links in 4th step's 5th and 6th block (#6752) (thanks @OnkarMadli)
[DOCS] Fixed code reference in documentation (#6732) (thanks @tmilitino)
[DOCS] Link to gh issue #4152 for ruamel.yaml ([#6799] (thanks @jamesmyatt)
[DOCS] ExpectationSuite and remove_expectation api docs (#6785)
[DOCS] Adding docstring for ExpectationSuite.add_expectation (#6829)
[DOCS] DOC-394: Fix broken redirect links (#6835)
See something, do something: ID/PK
Identify and retrieve the exact records that failed your Expectations with the new ID/PK feature.
Index:
[FEATURE] ID/PK Demo Files (#6833)
[BUGFIX] Pandas ID/PK query was causing DataDocs error (#6832)
[MAINTENANCE] ID/PK Tests at Expectations-level with Warnings caught (#6713)
[MAINTENANCE] ID/PK - Adding semi-colon to SQL Query output (#6743)
[MAINTENANCE] ID/PK result_format documentation update (#6716)
Laundry time: faster linting
Our linting is lightning-fast now that we’ve changed to using Ruff.
Care instructions:
[FEATURE] Docstring linter for public api (#6638)
[MAINTENANCE] Switch to ruff linter (#6888)
[MAINTENANCE] Use ruff linter for docstring linting (#6913)
Sprucing things up: other new features
[FEATURE] Accept a pathlib.Path context_root_dir (#6613)
[FEATURE] Add pandas datasource sorter by refactoring into DataAsset ([#6787]
[FEATURE] Experimental datasources batch.head() (#6765)
[FEATURE] Add Validation Result URL to Checkpoint Result (#6908)
Pest control: other bug fixes
[BUGFIX] Support non-string datetime evaluation parameters (#6571)
[BUGFIX] Stop overwriting query with static string in RuntimeBatchRequests for SQL (#6614)
[BUGFIX] Add connect args to execution engine schema (#6663) (thanks @itaise)
[BUGFIX] Implementing deep copy of runtime_configuration variable (#6682) (thanks @tmilitino)
[BUGFIX] Fix observed value (#6759) (thanks @itaise)
[BUGFIX] Fix missing not operator ~ (#6808)
[BUGFIX] Implemented lowercase function to check what type of file endswith (#6810) (thanks @tmilitino)
[BUGFIX] Disable RendererConfiguration constraint to support legacy renderer fallback behavior
[BUGFIX] Fix list_keys method for TupleS3StoreBackend (#6901) (thanks @enagovitsyn)
[BUGFIX] Render bar graph with boolean values (#6910) (thanks @tmilitino)
[BUGFIX] Capital one contrib/micdavis/import hotfix (#6922) (thanks @micdavis)
Upkeep: additional maintenance
[MAINTENANCE] Exclude scripts directory from package (#6744) (thanks @cburroughs)
[MAINTENANCE] Convert the validation results to JSON serializable (#6776) (thanks @lu-lz)
[MAINTENANCE] Fix the error message for invalid batch request options (#6848)
[MAINTENANCE] Standardize Store CRUD (#6826)
[MAINTENANCE] Fix argument name typo (#6850) (thanks @KirillUlich)
[MAINTENANCE] More clearly specifies range of supported python versions (#6870)
[MAINTENANCE] Standardize project config CRUD (#6837)
[MAINTENANCE] Add exception message to RenderedAtomicContent failure renderer (#6795)
Get the full GX changelog here.
