Expect a statistic's value for a given column of a DataProfiler difference report to be less than or equal to the specified threshold.
expect_profile_numeric_columns_diff_less_than_or_equal_to_threshold
This expectation level is EXPERIMENTAL
Contributors:
Tags:
Metrics:
Description
Expect a statistic's value for a given column of a DataProfiler difference report to be less than or equal to the specified threshold.
This expectation takes the difference report between the data it is called on and a DataProfiler profile of the same schema loaded from a provided path.
This function builds upon the custom ProfileNumericColumnsDiff Expectation of Capital One's DataProfiler Expectations.
Each numerical column will be checked against a user provided dictionary of columns paired with dictionaries of statistics containing a threshold value.
It is expected that a statistics value for a given column is less than or equal to the specified threshold.
Args:
- profile_path (str): A path to a saved DataProfiler profile object on the local filesystem.
- limit_check_report_keys (dict): A dict, containing column names as keys and dicts as values that contain statistics as keys and ints or floats as values
denoting the threshold that the statistic delta is expectated to be below. - mostly (float - optional): a value indicating the lower bound percentage of successful values that must be present to evaluate to success=True.
validator.expect_profile_numerical_columns_diff_less_than_threshold( - profile_path = "C:/path_to/my_profile.pkl",
limit_check_report_keys = { - "column_one": {
- "min": 0
}, - "*": {
- "*": 1
},
}
) - Note: In limit_check_report_keys, "" in place of a column denotes a general operator in which the value it stores will be applied to every column in the data that has no explicit key.
"" in place of a statistic denotes a general operator in which the bounds it stores will be applied to every statistic for the given column that has no explicit key.
Want to make your own Expectation or an improvement to this one?
We've put together some great how to guides (including videos) on how to create your own expectations in a flash!
You can see those resources here: Contributor Resources