The fluent way to connect to data sources in GX

It’s so much easier to connect to data & create a Datasource in Great Expectations now that we could have fit the whole update in the headline—so here’s a short post for a big announcement.

To illustrate, here’s a little now & then for configuring the same Datasource.

Now: fluent configuration in GX 0.16+

1datasource = context.sources.add_or_update_pandas(name="my_pandas_dataframe")

Then: block configuration in GX before 0.16

1datasource_config = {
2	"name": "my_pandas_datasource",
3	"class_name": "Datasource",
4	"module_name": "great_expectations.datasource",
5	"asset_name": "my_pandas_asset",
6	"execution_engine": {
7    	"class_name": "PandasExecutionEngine",
8     	"module_name": "great_expectations.execution_engine",
9	},
10	"data_connectors": {
11    	"my_pandas_connector": {
12        	"module_name": "great_expectations.datasource.data_connector",
13        	"class_name": "RuntimeDataConnector",
14    	}
15	},
16}
17datasource =  context.add_datasource(**ds_config, save_changes=True)

If you’re not a visual learner, here are some stats. To establish the same Datasource, the examples above take:

• 77 characters using fluent configuration

• 534 characters using block configuration

We can’t promise that every fluent configuration will be 86% shorter (or, to be precise, 85.58% shorter) than its block equivalent. But regardless of the exact percentage, the fluent configuration style is a huge upgrade.

Block configuration is still supported, but we encourage everyone to use the fluent configuration for new Datasources and to convert their existing ones when feasible.

For more details about fluent configuration, watch a video demo here or see our docs on connecting to data here. 

That’s all—happy connecting!