Skip to content

Commit 790596c

Browse files
authored
docs(databricks): merge databricks_tags additively across config hierarchy levels (#9379)
2 parents 2b1a47d + f561658 commit 790596c

1 file changed

Lines changed: 40 additions & 2 deletions

File tree

website/docs/reference/resource-configs/databricks-configs.md

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ dbt-databricks v1.9 adds support for the `table_format: iceberg` config. Try it
3030

3131
† When `table_format` is `iceberg`, `file_format` must be `delta`.
3232

33-
`databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook.
33+
`databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook. Starting in `dbt-databricks` v1.12, `databricks_tags` set at multiple config hierarchy levels [merge additively](#databricks_tags) instead of the lower (more specific) level fully replacing the higher one.
3434

3535
<sup>^</sup> When `liquid_clustered_by` is enabled, dbt-databricks issues an `OPTIMIZE` (Liquid Clustering) operation after each run. To disable this behavior, set the variable `DATABRICKS_SKIP_OPTIMIZE=true`, which can be passed into the dbt run command (`dbt run --vars "{'databricks_skip_optimize': true}"`) or set as an environment variable. See [issue #802](https://github.com/databricks/dbt-databricks/issues/802).
3636

@@ -151,7 +151,7 @@ When materializing models of various types, you may include several optional col
151151
\* `using_columns` supports all parameter types listed in [Databricks column mask parameters](https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-column-mask#parameters).
152152

153153

154-
† `databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook.
154+
† `databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook. Starting in `dbt-databricks` v1.12, `databricks_tags` set at multiple config hierarchy levels [merge additively](#databricks_tags) instead of the lower (more specific) level fully replacing the higher one.
155155

156156
This example uses the column-level configurations in the previous table:
157157

@@ -1329,6 +1329,44 @@ _Available in versions 1.11 or higher_
13291329

13301330
Tags are applied via `ALTER` statements after the materialization is created. Once applied, tags cannot be removed through dbt-databricks configuration changes. To remove tags, you must use Databricks directly or a post-hook.
13311331

1332+
:::caution Behavior change in v1.12
1333+
Starting in `dbt-databricks` v1.12.0, `databricks_tags` configurations are merged additively across config hierarchy levels (for example, project-level and model-level), rather than having lower-level configs completely replace higher-level ones.
1334+
1335+
When the same tag key is defined at multiple levels, the lower-level value takes precedence. Tag keys defined only at higher levels are retained.
1336+
1337+
This behavior applies anywhere `databricks_tags` can be configured, including tables, columns, materialized views, and streaming tables.
1338+
:::
1339+
1340+
For example, with the following project-level and model-level configs:
1341+
1342+
<File name='dbt_project.yml'>
1343+
1344+
```yaml
1345+
models:
1346+
my_project:
1347+
+databricks_tags:
1348+
a: "b"
1349+
c: "project_value"
1350+
```
1351+
1352+
</File>
1353+
1354+
<File name='models/my_model.sql'>
1355+
1356+
```sql
1357+
{{ config(
1358+
databricks_tags={'c': 'model_value', 'k': 'v'}
1359+
) }}
1360+
```
1361+
1362+
</File>
1363+
1364+
The resulting tags are:
1365+
1366+
- `a: b` — retained from the project level
1367+
- `c: model_value` — the model-level value overrides the project-level `c`
1368+
- `k: v` — added at the model level
1369+
13321370
#### description
13331371
As with views and tables, adding a `description` to your configuration will lead to a table-level comment getting added to your materialization.
13341372

0 commit comments

Comments
 (0)