Skip to content

Commit 5fad697

Browse files
authored
Fix generated pipeline README to use --refresh instead of --select (#5626)
## Changes The README generated for pipeline projects (`databricks bundle init default-python`, `databricks pipelines init`) documented `databricks bundle run <pipeline> --select <table>` to run a single transformation. `bundle run` has no `--select` flag, so the command fails with `Error: unknown flag: --select`. This switches the template to the supported `--refresh <table>` flag, which maps to `refresh_selection` (update only the named tables), and regenerates the affected acceptance golden outputs. Fixes #5252 ## Why Reported in #5252: the scaffolded README ships a command the CLI rejects. `--refresh` is the correct flag for running a single transformation (see `bundle/run/pipeline_options.go`). ## Tests Docs/template-only change. Regenerated golden outputs via the acceptance `-update` runs for `pipelines/init`, `pipelines/e2e`, and `bundle/templates`. Verified no `--select` remains in any generated README.
1 parent 7a8119d commit 5fad697

9 files changed

Lines changed: 9 additions & 8 deletions

File tree

  • acceptance
    • bundle/templates
      • default-python
        • classic/output/my_default_python/src/my_default_python_etl
        • serverless/output/my_default_python/src/my_default_python_etl
      • lakeflow-pipelines
        • python/output/my_lakeflow_pipelines/src/my_lakeflow_pipelines_etl
        • sql/output/my_lakeflow_pipelines/src/my_lakeflow_pipelines_etl
    • pipelines
      • e2e/output/lakeflow_project/src/lakeflow_project_etl
      • init
        • python/output/my_python_project/src/my_python_project_etl
        • sql/output/my_sql_project/src/my_sql_project_etl
  • libs/template/templates/default/template/{{.project_name}}/src/{{.project_name}}_etl

NEXT_CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
* Ignore unity catalog managed schema property defaults to avoid unnecessary drift ([#5195](https://github.com/databricks/cli/pull/5195)).
2424
* Add Postgres role as a bundle resource ([#5467](https://github.com/databricks/cli/pull/5467)).
2525
* direct: Stop spurious recreate/rename on redeploy when the backend normalizes a resource's name-based ID (e.g. Unity Catalog lowercasing a schema or volume name) ([#5599](https://github.com/databricks/cli/pull/5599)).
26+
* Fix the generated pipeline README to suggest `databricks bundle run <pipeline> --refresh <table>` for running a single transformation; the previously documented `--select` flag is not supported by `bundle run` ([#5252](https://github.com/databricks/cli/issues/5252)).
2627

2728
### Dependency updates
2829

acceptance/bundle/templates/default-python/classic/output/my_default_python/src/my_default_python_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_my_default_python.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run my_default_python_etl --select sample_trips_my_default_python` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run my_default_python_etl --refresh sample_trips_my_default_python` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

acceptance/bundle/templates/default-python/serverless/output/my_default_python/src/my_default_python_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_my_default_python.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run my_default_python_etl --select sample_trips_my_default_python` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run my_default_python_etl --refresh sample_trips_my_default_python` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

acceptance/bundle/templates/lakeflow-pipelines/python/output/my_lakeflow_pipelines/src/my_lakeflow_pipelines_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_my_lakeflow_pipelines.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run my_lakeflow_pipelines_etl --select sample_trips_my_lakeflow_pipelines` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run my_lakeflow_pipelines_etl --refresh sample_trips_my_lakeflow_pipelines` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

acceptance/bundle/templates/lakeflow-pipelines/sql/output/my_lakeflow_pipelines/src/my_lakeflow_pipelines_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_my_lakeflow_pipelines.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run my_lakeflow_pipelines_etl --select sample_trips_my_lakeflow_pipelines` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run my_lakeflow_pipelines_etl --refresh sample_trips_my_lakeflow_pipelines` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

acceptance/pipelines/e2e/output/lakeflow_project/src/lakeflow_project_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_lakeflow_project.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run lakeflow_project_etl --select sample_trips_lakeflow_project` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run lakeflow_project_etl --refresh sample_trips_lakeflow_project` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

acceptance/pipelines/init/python/output/my_python_project/src/my_python_project_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_my_python_project.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run my_python_project_etl --select sample_trips_my_python_project` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run my_python_project_etl --refresh sample_trips_my_python_project` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

acceptance/pipelines/init/sql/output/my_sql_project/src/my_sql_project_etl/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_my_sql_project.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run my_sql_project_etl --select sample_trips_my_sql_project` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run my_sql_project_etl --refresh sample_trips_my_sql_project` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

libs/template/templates/default/template/{{.project_name}}/src/{{.project_name}}_etl/README.md.tmpl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,6 @@ To get started, go to the `transformations` folder -- most of the relevant sourc
1515
* Take a look at the sample called "sample_trips_{{.project_name}}.py" to get familiar with the syntax.
1616
Read more about the syntax at https://docs.databricks.com/dlt/python-ref.html.
1717
* If you're using the workspace UI, use `Run file` to run and preview a single transformation.
18-
* If you're using the CLI, use `databricks bundle run {{.project_name}}_etl --select sample_trips_{{.project_name}}` to run a single transformation.
18+
* If you're using the CLI, use `databricks bundle run {{.project_name}}_etl --refresh sample_trips_{{.project_name}}` to run a single transformation.
1919

2020
For more tutorials and reference material, see https://docs.databricks.com/dlt.

0 commit comments

Comments
 (0)