Skip to content

Commit 45f2a3c

Browse files
committed
perf: honor incremental_apply_config_changes flag in V1 incremental path
The V1 incremental merge path called adapter.get_relation_config() unconditionally, forcing metadata fetches (tags, column_tags, constraints, column_masks, tblproperties, describe_extended) on every incremental run even when the user opted out via incremental_apply_config_changes=false. The V2 path already honors the flag via the process_config_changes() macro; V1 ignored it. Wrap the V1 get_relation_config call in a flag check matching the V2 semantics. When the flag is false, _configuration_changes stays None and the existing apply-block guard (`if _configuration_changes is not none`) short-circuits the apply, so no metadata is fetched or applied. Default behavior (flag=true) is unchanged. Co-authored-by: Isaac
1 parent 86d26c9 commit 45f2a3c

4 files changed

Lines changed: 58 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
## dbt-databricks 1.13.0 (TBD)
2+
3+
### Under the Hood
4+
5+
- Honor `incremental_apply_config_changes` in the V1 incremental merge path, allowing users to skip metadata diff queries (tags, column_tags, constraints, column_masks, tblproperties, describe_extended) when set to `false`. Matches the existing V2 behavior. ([#1402](https://github.com/databricks/dbt-databricks/issues/1402))
6+
17
## dbt-databricks 1.12.1 (June 10, 2026)
28

39
### Features

dbt/include/databricks/macros/materializations/incremental/incremental.sql

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -133,9 +133,12 @@
133133
{{ set_overwrite_mode('DYNAMIC') }}
134134
{%- endif -%}
135135
{#-- Relation must be merged --#}
136-
{%- set model_config = adapter.get_config_from_model(config.model) -%}
137-
{%- set _existing_config = adapter.get_relation_config(existing_relation, model_config) -%}
138-
{%- set _configuration_changes = model_config.get_changeset(_existing_config) -%}
136+
{%- set _configuration_changes = none -%}
137+
{%- if config.get('incremental_apply_config_changes', True) | as_bool -%}
138+
{%- set model_config = adapter.get_config_from_model(config.model) -%}
139+
{%- set _existing_config = adapter.get_relation_config(existing_relation, model_config) -%}
140+
{%- set _configuration_changes = model_config.get_changeset(_existing_config) -%}
141+
{%- endif -%}
139142
{%- call statement('create_temp_relation', language=language) -%}
140143
{{ create_table_as(True, temp_relation, compiled_code, language) }}
141144
{%- endcall -%}

tests/functional/adapter/incremental/fixtures.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,16 @@
8585
select cast(1 as bigint) as id
8686
"""
8787

88+
metadata_fetch_incremental_skip_config_changes_sql = """
89+
{{ config(
90+
materialized = 'incremental',
91+
unique_key = 'id',
92+
incremental_apply_config_changes = false,
93+
) }}
94+
95+
select cast(1 as bigint) as id
96+
"""
97+
8898
metadata_fetch_no_tags_schema = """
8999
version: 2
90100
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
import pytest
2+
from dbt.tests import util
3+
4+
from tests.functional.adapter.fixtures import (
5+
MaterializationV1Mixin,
6+
fail_if_tag_and_column_tag_fetch_called_macros,
7+
)
8+
from tests.functional.adapter.incremental import fixtures
9+
10+
11+
@pytest.mark.skip_profile("databricks_cluster")
12+
class TestV1IncrementalApplyConfigChangesFalseSkipsTagFetch(MaterializationV1Mixin):
13+
"""When `incremental_apply_config_changes` is false, the V1 incremental merge path
14+
must skip the get_relation_config metadata fetches even if the model declares tags.
15+
"""
16+
17+
@pytest.fixture(scope="class")
18+
def models(self):
19+
return {
20+
"metadata_fetch_incremental.sql": (
21+
fixtures.metadata_fetch_incremental_skip_config_changes_sql
22+
),
23+
"schema.yml": fixtures.metadata_fetch_table_tags_schema,
24+
}
25+
26+
@pytest.fixture(scope="class")
27+
def macros(self):
28+
return {"fail_if_tag_fetch_called.sql": fail_if_tag_and_column_tag_fetch_called_macros}
29+
30+
def test_v1_incremental_skips_metadata_fetch_when_flag_false(self, project):
31+
# First run creates the table; second run exercises the existing-relation merge path
32+
# where get_relation_config would normally fire metadata queries.
33+
# Tags are declared on the model, so without the flag the run would call fetch_tags
34+
# and fail. The flag must bypass the entire get_relation_config call.
35+
util.run_dbt(["run"])
36+
util.run_dbt(["run"])

0 commit comments

Comments
 (0)