Improving the developer experience has been one of our top priorities over the past year. Last month, we made major progress on that by introducing Fluent Datasources.
Fluent Datasources are so easy to use that the CLI is superfluous or even a hindrance. So we’ve decided that we won’t be updating the CLI to support them.
Why we made this decision
Fluent Datasources are the culmination of a ton of great feedback from our users on how we could make Great Expectations more usable. Simply put, they’re a much, much easier way to connect to data, compared to the previous block-configuration style.
With Fluent Datasources, you can:
Define a Datasource in one line of code
Configure Datasources and Data Assets incrementally, with immediate feedback as things work (or don’t work)
Set up the configuration using discoverable Python APIs with autocomplete, instead of needing large blocks of YAML configuration
Here’s how easy it is to connect to data with a Fluent Datasource:
1datasource = context.sources.add_sql(name="my_sql_datasource",2connection_string=my_connection_string)3
4data_asset = datasource.add_table_asset(5 name="my_monthly_taxi_data", table_name="taxi_data"6)In the past, the CLI-to-Jupyter-Notebook workflow was a critical helper for users configuring YAML-based Datasources. The series of prompts the CLI provided helped users scaffold their config by pre-populating values, and the test_yaml_config method let them confirm that the configuration worked as expected.
With all of the improvements we’ve made with Fluent Datasources, this scaffolding support from the CLI is no longer needed.
In fact, the CLI now complicates the workflow. Configuring Fluent Datasources with the CLI would require you to go back and forth between the command line and the notebook, when you could just stay in the notebook and use the clear type-hinting and autocomplete to discover the code instead.
You can see this clearly when comparing the CLI process for adding a Datasource…
…with the recommended Fluent approach.
Given this, we’ve decided that we’re not going to update the CLI to work with Fluent Datasources.
What happens now
Since we encourage all newly-created Datasources to be Fluent Datasources, we’ll be updating our documentation to remove references to the CLI over the next month. As a first step, pages referencing the CLI will have banners marking them for update; these will be removed when the page gets updated.
If you’re already using the CLI and want to keep doing that, it’s still supported for Block-config Datasources and you don’t need to change anything. Do note that if you need to look up anything related to the CLI in our docs, you’ll have to reference a version of the documentation before v0.16.
Please know that you can always provide feedback in our #gx-feedback Slack channel, and post questions in #gx-community-support.
We thrive off of this feedback—discussions with you are what got us to the point of making these usability improvements, and we continue to be grateful to the community for all of the feedback and contributions we receive every day.
