Skip to content

Commit ac48f74

Browse files
authored
test: make incremental constraint functional tests rerun-safe (#1503)
## Problem The constraint tests in `tests/functional/adapter/incremental/test_incremental_constraints.py` rewrite `schema.yml` in place mid-test (via `util.write_file`) and build relations, but they aren't safe under `pytest --reruns`. pytest-rerunfailures retries a failed test **without** tearing down the class-scoped `project` fixture, so the mutated `schema.yml` and the relations created by the failed attempt leak into the retry and fail it deterministically — meaning a single intermittent error burns all retries. Observed on `TestIncrementalRemoveForeignKeyConstraint::test_remove_foreign_key_constraint` in a nightly: an intermittent server `INTERNAL_ERROR` failed the initial attempt, and **both** `--reruns 2` retries then failed because they ran against the un-reset, half-built state (only the corrupted FK-parent kept failing; its siblings kept succeeding). ## Change Apply the existing `RerunSafeMixin` (added in #1499) to the `schema.yml`-mutating constraint classes. Before each attempt its autouse fixture restores the initial model files and drops the relations the test builds (named via `relations_to_reset`). FK-holding relations are listed before their parents so the drop order respects the constraint. The two `...DescribeJsonOn` subclasses inherit the mixin from their parents. Test-only; no adapter/runtime changes. ## Verification - Full `test_incremental_constraints.py` on `databricks_uc_cluster`: **15 passed, 0 failed**. - Deterministic rerun-recovery proof (throwaway): a class that mutates `schema.yml` then fails its first attempt **recovers on rerun with the mixin (`RERUN → PASSED`)** and **fails without it (`RERUN → FAILED`)** — the mixin restores `schema.yml` and drops the relations before the retry. Mirrors #1499's forced-rerun repro. - ruff / ruff-format / mypy pass.
1 parent ee495c2 commit ac48f74

2 files changed

Lines changed: 48 additions & 9 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818

1919
### Under the Hood
2020

21+
- Make the incremental constraint functional tests rerun-safe so a `pytest --reruns` retry no longer inherits mutated state (rewritten `schema.yml`, half-built relations) from the failed attempt (test-only, no runtime impact). ([#1503](https://github.com/databricks/dbt-databricks/pull/1503))
2122
- Make the column-tag functional tests rerun-safe so a `pytest --reruns` retry no longer inherits mutated state (updated `schema.yml`, leftover column tags, a running streaming-table query) from the failed attempt (test-only, no runtime impact). ([#1499](https://github.com/databricks/dbt-databricks/pull/1499))
2223
- Raise the `dbt-tests-adapter` test-dependency floor to `>=1.20.0` to pick up its `persist_docs` fixture typo fix (test-only, no runtime impact) ([#1490](https://github.com/databricks/dbt-databricks/pull/1490))
2324
- Defer SDK `Config` construction to connection-open time so offline paths (`dbt parse`/`list`/`compile`) don't trigger the host-metadata probe introduced in `databricks-sdk>=0.103`; as a side effect, auth errors now surface at first connection rather than during profile parsing. ([#1474](https://github.com/databricks/dbt-databricks/pull/1474))

tests/functional/adapter/incremental/test_incremental_constraints.py

Lines changed: 47 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,19 @@
22
from dbt.contracts.results import RunStatus
33
from dbt.tests import util
44

5-
from tests.functional.adapter.fixtures import RequiresDescribeAsJsonCapabilityMixin
5+
from tests.functional.adapter.fixtures import (
6+
RequiresDescribeAsJsonCapabilityMixin,
7+
RerunSafeMixin,
8+
)
69
from tests.functional.adapter.incremental import fixtures
710

811

912
@pytest.mark.skip_profile("databricks_cluster")
10-
class TestIncrementalSetNonNullConstraint:
13+
class TestIncrementalSetNonNullConstraint(RerunSafeMixin):
14+
@pytest.fixture(scope="class")
15+
def relations_to_reset(self):
16+
return ("non_null_constraint_sql",)
17+
1118
@pytest.fixture(scope="class")
1219
def project_config_update(self):
1320
return {
@@ -46,7 +53,11 @@ def project_config_update(self):
4653

4754

4855
@pytest.mark.skip_profile("databricks_cluster")
49-
class TestIncrementalUnsetNonNullConstraint:
56+
class TestIncrementalUnsetNonNullConstraint(RerunSafeMixin):
57+
@pytest.fixture(scope="class")
58+
def relations_to_reset(self):
59+
return ("non_null_constraint_sql",)
60+
5061
@pytest.fixture(scope="class")
5162
def project_config_update(self):
5263
return {
@@ -82,7 +93,11 @@ def test_remove_non_null_constraint(self, project):
8293

8394

8495
@pytest.mark.skip_profile("databricks_cluster")
85-
class TestIncrementalSetCheckConstraint:
96+
class TestIncrementalSetCheckConstraint(RerunSafeMixin):
97+
@pytest.fixture(scope="class")
98+
def relations_to_reset(self):
99+
return ("check_constraint_sql",)
100+
86101
@pytest.fixture(scope="class")
87102
def project_config_update(self):
88103
return {
@@ -107,7 +122,11 @@ def test_add_check_constraint(self, project):
107122

108123

109124
@pytest.mark.skip_profile("databricks_cluster")
110-
class TestIncrementalRemoveCheckConstraint:
125+
class TestIncrementalRemoveCheckConstraint(RerunSafeMixin):
126+
@pytest.fixture(scope="class")
127+
def relations_to_reset(self):
128+
return ("check_constraint_sql",)
129+
111130
@pytest.fixture(scope="class")
112131
def project_config_update(self):
113132
return {
@@ -146,14 +165,18 @@ def test_remove_check_constraint(self, project):
146165

147166

148167
@pytest.mark.skip_profile("databricks_cluster")
149-
class TestIncrementalUpdatePrimaryKeyConstraint:
168+
class TestIncrementalUpdatePrimaryKeyConstraint(RerunSafeMixin):
150169
primary_key_constraint_sql = """
151170
SELECT constraint_name, column_name
152171
FROM {database}.information_schema.key_column_usage
153172
WHERE constraint_schema = '{schema}'
154173
ORDER BY ordinal_position
155174
"""
156175

176+
@pytest.fixture(scope="class")
177+
def relations_to_reset(self):
178+
return ("primary_key_constraint_sql",)
179+
157180
@pytest.fixture(scope="class")
158181
def project_config_update(self):
159182
return {
@@ -205,7 +228,12 @@ def project_config_update(self):
205228

206229

207230
@pytest.mark.skip_profile("databricks_cluster")
208-
class TestCascadingConstraintDrop:
231+
class TestCascadingConstraintDrop(RerunSafeMixin):
232+
@pytest.fixture(scope="class")
233+
def relations_to_reset(self):
234+
# ref_table holds the FK to primary_key_constraint_sql, so drop it first.
235+
return ("ref_table", "primary_key_constraint_sql")
236+
209237
@pytest.fixture(scope="class")
210238
def project_config_update(self):
211239
return {
@@ -258,7 +286,12 @@ def test_cascading_constraint_drop(self, project):
258286

259287

260288
@pytest.mark.skip_profile("databricks_cluster")
261-
class TestIncrementalSetForeignKeyConstraint:
289+
class TestIncrementalSetForeignKeyConstraint(RerunSafeMixin):
290+
@pytest.fixture(scope="class")
291+
def relations_to_reset(self):
292+
# fk_referenced_from_table holds the FKs, so drop it before its parents.
293+
return ("fk_referenced_from_table", "fk_referenced_to_table", "fk_referenced_to_table_2")
294+
262295
@pytest.fixture(scope="class")
263296
def project_config_update(self):
264297
return {
@@ -323,7 +356,12 @@ def test_warn_unenforced_false(self, project):
323356

324357

325358
@pytest.mark.skip_profile("databricks_cluster")
326-
class TestIncrementalRemoveForeignKeyConstraint:
359+
class TestIncrementalRemoveForeignKeyConstraint(RerunSafeMixin):
360+
@pytest.fixture(scope="class")
361+
def relations_to_reset(self):
362+
# fk_referenced_from_table holds the FKs, so drop it before its parents.
363+
return ("fk_referenced_from_table", "fk_referenced_to_table", "fk_referenced_to_table_2")
364+
327365
@pytest.fixture(scope="class")
328366
def project_config_update(self):
329367
return {

0 commit comments

Comments
 (0)