A fond farewell to the CLI

With the usability improvements introduced in Fluent Datasources, GX is outgrowing its CLI

Don Heppner
May 01, 2023
Don Heppner
May 01, 2023
A sailboat sailing against a sunset

Improving the developer experience has been one of our top priorities over the past year. Last month, we made major progress on that by introducing Fluent Datasources.

Fluent Datasources are so easy to use that the CLI is superfluous or even a hindrance. So we’ve decided that we won’t be updating the CLI to support them.

Why we made this decision

Fluent Datasources are the culmination of a ton of great feedback from our users on how we could make Great Expectations more usable. Simply put, they’re a much, much easier way to connect to data, compared to the previous block-configuration style. 

With Fluent Datasources, you can: 

  • Define a Datasource in one line of code

  • Configure Datasources and Data Assets incrementally, with immediate feedback as things work (or don’t work)

  • Set up the configuration using discoverable Python APIs with autocomplete, instead of needing large blocks of YAML configuration 

Here’s how easy it is to connect to data with a Fluent Datasource:

1datasource = context.sources.add_sql(name="my_sql_datasource",
4data_asset = datasource.add_table_asset(
5   name="my_monthly_taxi_data", table_name="taxi_data"

In the past, the CLI-to-Jupyter-Notebook workflow was a critical helper for users configuring YAML-based Datasources. The series of prompts the CLI provided helped users scaffold their config by pre-populating values, and the test_yaml_config method let them confirm that the configuration worked as expected. 

With all of the improvements we’ve made with Fluent Datasources, this scaffolding support from the CLI is no longer needed.

In fact, the CLI now complicates the workflow. Configuring Fluent Datasources with the CLI would require you to go back and forth between the command line and the notebook, when you could just stay in the notebook and use the clear type-hinting and autocomplete to discover the code instead.

You can see this clearly when comparing the CLI process for adding a Datasource…

Connect Block-config Datasource with CLI

…with the recommended Fluent approach.

Connect to Fluent Datasource

Given this, we’ve decided that we’re not going to update the CLI to work with Fluent Datasources.

What happens now

Since we encourage all newly-created Datasources to be Fluent Datasources, we’ll be updating our documentation to remove references to the CLI over the next month. As a first step, pages referencing the CLI  will have banners marking them for update; these will be removed when the page gets updated.

If you’re already using the CLI and want to keep doing that, it’s still supported for Block-config Datasources and you don’t need to change anything. Do note that if you need to look up anything related to the CLI in our docs, you’ll have to reference a version of the documentation before v0.16.

Please know that you can always provide feedback in our #gx-feedback Slack channel, and post questions in #gx-community-support. 

We thrive off of this feedback—discussions with you are what got us to the point of making these usability improvements, and we continue to be grateful to the community for all of the feedback and contributions we receive every day.

Like our blogs?

Sign up for emails and get more blogs and news

Great Expectations email sign-up

Hello friend of Great Expectations!

Our email content features product updates from the open source platform and our upcoming Cloud product, new blogs and community celebrations.

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Error message placeholder

Banner Image

Search our blog for the latest on data management

©2023 Great Expectations. All Rights Reserved.