You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
`databricks_tags` now merges additively across dbt config hierarchy levels: a model-level `databricks_tags` value no longer clobbers tags defined at the project level — instead the dictionaries are merged (child keys override matching parent keys, other parent keys are retained). This is a breaking change for projects that relied on lower-level configs fully replacing higher-level ones.
Copy file name to clipboardExpand all lines: website/docs/reference/resource-configs/databricks-configs.md
+32-2Lines changed: 32 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,7 +29,7 @@ dbt-databricks v1.9 adds support for the `table_format: iceberg` config. Try it
29
29
30
30
† When `table_format` is `iceberg`, `file_format` must be `delta`.
31
31
32
-
‡ `databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook.
32
+
‡ `databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook. As of dbt-databricks v1.12.0, `databricks_tags` set at multiple config hierarchy levels [merge additively](#databricks_tags) rather than the lower level fully replacing the higher one.
33
33
34
34
<sup>^</sup> When `liquid_clustered_by` is enabled, dbt-databricks issues an `OPTIMIZE` (Liquid Clustering) operation after each run. To disable this behavior, set the variable `DATABRICKS_SKIP_OPTIMIZE=true`, which can be passed into the dbt run command (`dbt run --vars "{'databricks_skip_optimize': true}"`) or set as an environment variable. See [issue #802](https://github.com/databricks/dbt-databricks/issues/802).
35
35
@@ -150,7 +150,7 @@ When materializing models of various types, you may include several optional col
150
150
\* `using_columns` supports all parameter types listed in [Databricks column mask parameters](https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-column-mask#parameters).
151
151
152
152
153
-
† `databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook.
153
+
† `databricks_tags` are applied via `ALTER` statements. Tags cannot be removed via dbt-databricks once applied. To remove tags, use Databricks directly or a post-hook. As of dbt-databricks v1.12.0, `databricks_tags` set at multiple config hierarchy levels [merge additively](#databricks_tags) rather than the lower level fully replacing the higher one.
154
154
155
155
This example uses the column-level configurations in the previous table:
156
156
@@ -1208,6 +1208,36 @@ _Available in versions 1.11 or higher_
1208
1208
1209
1209
Tags are applied via `ALTER` statements after the materialization is created. Once applied, tags cannot be removed through dbt-databricks configuration changes. To remove tags, you must use Databricks directly or a post-hook.
1210
1210
1211
+
:::caution Breaking change in v1.12.0
1212
+
As of dbt-databricks v1.12.0, `databricks_tags` set at different config hierarchy levels (for example, project-level and model-level) merge additively instead of the lower-level config completely replacing the higher-level one. Keys defined at the lower level override matching keys from the higher level, and any other keys from the higher level are retained. This applies wherever `databricks_tags` is set, including tables, columns, materialized views, and streaming tables.
1213
+
:::
1214
+
1215
+
For example, with the following project-level and model-level configs:
1216
+
1217
+
<Filename='dbt_project.yml'>
1218
+
1219
+
```yaml
1220
+
models:
1221
+
my_project:
1222
+
+databricks_tags:
1223
+
a: "b"
1224
+
c: "project_value"
1225
+
```
1226
+
1227
+
</File>
1228
+
1229
+
<File name='models/my_model.sql'>
1230
+
1231
+
```sql
1232
+
{{ config(
1233
+
databricks_tags={'c': 'model_value', 'k': 'v'}
1234
+
) }}
1235
+
```
1236
+
1237
+
</File>
1238
+
1239
+
The model is tagged with `a: b` (kept from the project level), `c: model_value` (the model-level value overrides the project-level `c`), and `k: v` (added at the model level).
1240
+
1211
1241
#### description
1212
1242
As with views and tables, adding a `description` to your configuration will lead to a table-level comment getting added to your materialization.
0 commit comments