Skip to content

feat: add full ClickHouse support - remove all skip_targets markers (CORE-397) #934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
haritamar merged 23 commits into from
Mar 1, 2026
Merged

feat: add full ClickHouse support - remove all skip_targets markers (CORE-397) #934

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
dd75799
chore: remove all skip_targets(["clickhouse"]) markers from test files
devin-ai-integration[bot] Feb 25, 2026
0a2dd80
ci: temporarily limit CI matrix to clickhouse-only for iteration
devin-ai-integration[bot] Feb 25, 2026
41701e9
fix: use NOT IN instead of LEFT JOIN IS NULL for ClickHouse compatibi…
devin-ai-integration[bot] Feb 25, 2026
1d2fe6c
fix: ClickHouse Nullable(Float32) cast + HTTP API seed null fix
devin-ai-integration[bot] Feb 25, 2026
d43adc6
fix: address CodeRabbit review + revert NOT IN back to LEFT JOIN
devin-ai-integration[bot] Feb 25, 2026
e36e33e
fix: address CodeRabbit review round 2 - env vars, timeout, SQL injec…
devin-ai-integration[bot] Feb 25, 2026
ca0db37
fix: ClickHouse full_names adapter.dispatch, seasonality macros, even…
devin-ai-integration[bot] Feb 25, 2026
749d582
fix: ClickHouse event_freshness timediff NULL handling + list_concat …
devin-ai-integration[bot] Feb 25, 2026
4cd2e2a
fix: dynamically resolve ClickHouse schema from dbt profiles.yml inst…
devin-ai-integration[bot] Feb 25, 2026
956c061
ci: restore full CI matrix with all warehouse types
devin-ai-integration[bot] Feb 25, 2026
ad08596
refactor: extract ClickHouse seed repair utils + dispatch empty-strin…
devin-ai-integration[bot] Feb 25, 2026
92d0e89
refactor: remove unused clickhouse__ dispatch from replace_empty_stri…
devin-ai-integration[bot] Feb 25, 2026
48ea275
ci: retrigger CI to verify flaky test_seed_group_attribute failure
devin-ai-integration[bot] Feb 25, 2026
81219e3
Merge origin/master into core-397-clickhouse-support
devin-ai-integration[bot] Feb 27, 2026
8893238
Merge origin/master into core-397-clickhouse-support
devin-ai-integration[bot] Feb 28, 2026
1915c34
refactor: replace clickhouse_utils.py with ClickHouseDirectSeeder
devin-ai-integration[bot] Mar 1, 2026
5b38158
fix: add type inference to ClickHouseDirectSeeder
devin-ai-integration[bot] Mar 1, 2026
4d6e7c2
fix: treat booleans as strings in ClickHouseDirectSeeder
devin-ai-integration[bot] Mar 1, 2026
601db95
fix: use Nullable(Bool) for boolean columns in ClickHouseDirectSeeder
devin-ai-integration[bot] Mar 1, 2026
207d9bd
fix: write CSV for dbt node discovery in ClickHouseDirectSeeder
devin-ai-integration[bot] Mar 1, 2026
335fbf3
refactor: remove run_operation retry logic from run_query path
devin-ai-integration[bot] Mar 1, 2026
6f886e7
Merge origin/master into core-397-clickhouse-support
devin-ai-integration[bot] Mar 1, 2026
8823952
docs: add comment explaining why clickhouse__has_temp_table_support r…
devin-ai-integration[bot] Mar 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 1 addition & 25 deletions .github/workflows/test-all-warehouses.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,31 +87,7 @@ jobs:
dbt-version:
${{ inputs.dbt-version && fromJSON(format('["{0}"]', inputs.dbt-version)) ||
fromJSON('["latest_official"]') }}
warehouse-type:
[
postgres,
snowflake,
bigquery,
redshift,
databricks_catalog,
athena,
trino,
clickhouse,
dremio,
]
include:
- dbt-version: "${{ inputs.dbt-version || 'latest_pre' }}"
warehouse-type: postgres
- dbt-version: "${{ inputs.dbt-version || 'fusion' }}"
warehouse-type: snowflake
- dbt-version: "${{ inputs.dbt-version || 'fusion' }}"
warehouse-type: bigquery
# fusion/redshift temporarily disabled - Fusion sidecar can't resolve
# source schemas via SVV_COLUMNS on Redshift (see dbt-labs/dbt-fusion issues)
# - dbt-version: "${{ inputs.dbt-version || 'fusion' }}"
# warehouse-type: redshift
- dbt-version: "${{ inputs.dbt-version || 'fusion' }}"
warehouse-type: databricks_catalog
warehouse-type: [clickhouse]
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Restore full warehouse matrix before merge (current single-target matrix is merge-blocking).

Line 90 limits CI to ClickHouse only, which drops regression coverage for all other warehouses/fusion variants. Keep this for WIP iteration if needed, but restore the full matrix before this PR is merged.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/test-all-warehouses.yml at line 90, The CI workflow
currently restricts the test matrix by setting the "warehouse-type" matrix to
just [clickhouse]; restore the full warehouse matrix before merging by reverting
that change so "warehouse-type" includes all intended entries (e.g., clickhouse
plus the other warehouse and fusion variants used elsewhere in the matrix),
ensuring the job definitions that reference "warehouse-type" (the matrix key
"warehouse-type") run across the full set of targets rather than a single
[clickhouse] value.

uses: ./.github/workflows/test-warehouse.yml
with:
warehouse-type: ${{ matrix.warehouse-type }}
Expand Down
10 changes: 0 additions & 10 deletions integration_tests/tests/test_all_columns_anomalies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_all_columns_anomalies(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
data: List[Dict[str, Any]] = [
Expand All @@ -31,8 +29,6 @@ def test_anomalyless_all_columns_anomalies(test_id: str, dbt_project: DbtProject
assert all([res["status"] == "pass" for res in test_results])


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalous_all_columns_anomalies(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand All @@ -57,8 +53,6 @@ def test_anomalous_all_columns_anomalies(test_id: str, dbt_project: DbtProject):
assert col_to_status == {"superhero": "fail", TIMESTAMP_COLUMN: "pass"}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_all_columns_anomalies_with_where_parameter(
test_id: str, dbt_project: DbtProject
):
Expand Down Expand Up @@ -128,8 +122,6 @@ def test_all_columns_anomalies_with_where_parameter(
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_all_columns_anomalies_all_monitors_sanity(
test_id: str, dbt_project: DbtProject
):
Expand All @@ -155,8 +147,6 @@ def test_anomalyless_all_columns_anomalies_all_monitors_sanity(
assert all([res["status"] == "pass" for res in test_results])


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
@pytest.mark.parametrize(
"exclude_detection,expected_status",
[
Expand Down
15 changes: 0 additions & 15 deletions integration_tests/tests/test_anomalies_backfill_logic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from datetime import datetime, time, timedelta

import dateutil.parser
import pytest
from data_generator import DATE_FORMAT, generate_dates
from dbt_project import DbtProject

Expand Down Expand Up @@ -74,8 +73,6 @@ def get_latest_anomaly_test_metrics(dbt_project: DbtProject, test_id: str):
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_full_backfill_for_non_incremental_model(dbt_project: DbtProject, test_id: str):
utc_today = datetime.utcnow().date()
data_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -109,8 +106,6 @@ def test_full_backfill_for_non_incremental_model(dbt_project: DbtProject, test_i
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_partial_backfill_for_incremental_models(dbt_project: DbtProject, test_id: str):
utc_today = datetime.utcnow().date()
data_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -157,8 +152,6 @@ def test_partial_backfill_for_incremental_models(dbt_project: DbtProject, test_i
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_longer_backfill_in_case_of_a_gap(dbt_project: DbtProject, test_id: str):
date_gap_size = 5
utc_today = datetime.utcnow().date()
Expand Down Expand Up @@ -211,8 +204,6 @@ def test_longer_backfill_in_case_of_a_gap(dbt_project: DbtProject, test_id: str)
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_full_backfill_if_metric_not_updated_for_a_long_time(
dbt_project: DbtProject, test_id: str
):
Expand Down Expand Up @@ -272,8 +263,6 @@ def test_full_backfill_if_metric_not_updated_for_a_long_time(
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_backfill_when_metric_doesnt_exist_back_enough(
dbt_project: DbtProject, test_id: str
):
Expand Down Expand Up @@ -318,8 +307,6 @@ def test_backfill_when_metric_doesnt_exist_back_enough(
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_backfill_with_middle_buckets_gap(dbt_project: DbtProject, test_id: str):
utc_today = datetime.utcnow().date()
data_start = utc_today - timedelta(21)
Expand Down Expand Up @@ -388,8 +375,6 @@ def test_backfill_with_middle_buckets_gap(dbt_project: DbtProject, test_id: str)
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_bucket_size_not_aligned_with_days(dbt_project: DbtProject, test_id: str):
"""
In this test we choose a bucket size that is not aligned with one day - specifically 7 hours.
Expand Down
5 changes: 0 additions & 5 deletions integration_tests/tests/test_anomalies_ranges.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from datetime import datetime, timedelta
from typing import Any, Dict, List

import pytest
from data_generator import DATE_FORMAT, generate_dates
from dbt_project import DbtProject

Expand Down Expand Up @@ -34,8 +33,6 @@ def get_latest_anomaly_test_points(dbt_project: DbtProject, test_id: str):
return [json.loads(result["result_row"]) for result in results]


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomaly_ranges_are_valid(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -69,8 +66,6 @@ def test_anomaly_ranges_are_valid(test_id: str, dbt_project: DbtProject):
assert all([row["min_value"] == row["max_value"] for row in anomaly_test_points])


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomaly_ranges_are_valid_with_seasonality(
test_id: str, dbt_project: DbtProject
):
Expand Down
9 changes: 0 additions & 9 deletions integration_tests/tests/test_anomaly_exclude_metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from datetime import datetime, timedelta
from typing import Any, Dict, List

import pytest
from data_generator import DATE_FORMAT, generate_dates
from dbt_project import DbtProject
from parametrization import Parametrization
Expand All @@ -25,8 +24,6 @@
time_bucket={"period": "hour", "count": 6},
dates_step=timedelta(hours=6),
)
# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_exclude_specific_dates(
test_id: str, dbt_project: DbtProject, time_bucket: dict, dates_step: timedelta
):
Expand Down Expand Up @@ -71,8 +68,6 @@ def test_exclude_specific_dates(
assert test_result["status"] == "fail"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_exclude_specific_timestamps(test_id: str, dbt_project: DbtProject):
# To avoid races, set the "custom_started_at" to the beginning of the hour
test_started_at = datetime.utcnow().replace(minute=0, second=0)
Expand Down Expand Up @@ -128,8 +123,6 @@ def test_exclude_specific_timestamps(test_id: str, dbt_project: DbtProject):
assert test_result["status"] == "fail"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_exclude_date_range(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -165,8 +158,6 @@ def test_exclude_date_range(test_id: str, dbt_project: DbtProject):
assert test_result["status"] == "fail"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_exclude_by_metric_value(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down
1 change: 0 additions & 1 deletion integration_tests/tests/test_anomaly_test_configuration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,6 @@ def get_value(key: str):
test_config={key: value.test for key, value in PARAM_VALUES.items()},
expected_config=_get_expected_adapted_config("test"),
)
@pytest.mark.skip_targets(["clickhouse"])
@pytest.mark.skip_for_dbt_fusion
def test_anomaly_test_configuration(
dbt_project: DbtProject,
Expand Down
9 changes: 0 additions & 9 deletions integration_tests/tests/test_collect_metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from datetime import datetime, timedelta
from typing import Any, Dict, List

import pytest
from data_generator import DATE_FORMAT, generate_dates
from dbt_project import DbtProject

Expand Down Expand Up @@ -35,8 +34,6 @@
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_collect_metrics(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
data: List[Dict[str, Any]] = [
Expand Down Expand Up @@ -65,8 +62,6 @@ def test_collect_metrics(test_id: str, dbt_project: DbtProject):
assert col_to_metric_names == EXPECTED_COL_TO_METRIC_NAMES


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_collect_no_timestamp_metrics(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
data: List[Dict[str, Any]] = [
Expand Down Expand Up @@ -96,8 +91,6 @@ def test_collect_no_timestamp_metrics(test_id: str, dbt_project: DbtProject):
assert col_to_metric_names == EXPECTED_COL_TO_METRIC_NAMES


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_collect_group_by_metrics(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
data: List[Dict[str, Any]] = [
Expand Down Expand Up @@ -148,8 +141,6 @@ def test_collect_group_by_metrics(test_id: str, dbt_project: DbtProject):
assert dim_to_col_to_metric_names == expected_dim_to_col_to_metric_names


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_collect_metrics_unique_metric_name(test_id: str, dbt_project: DbtProject):
args = DBT_TEST_ARGS.copy()
args["metrics"].append(args["metrics"][0])
Expand Down
27 changes: 0 additions & 27 deletions integration_tests/tests/test_column_anomalies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from datetime import datetime, timedelta
from typing import Any, Dict, List

import pytest
from data_generator import DATE_FORMAT, generate_dates
from dbt_project import DbtProject
from parametrization import Parametrization
Expand All @@ -14,8 +13,6 @@
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_column_anomalies(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
data: List[Dict[str, Any]] = [
Expand All @@ -32,8 +29,6 @@ def test_anomalyless_column_anomalies(test_id: str, dbt_project: DbtProject):
assert test_result["status"] == "pass"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_no_timestamp_column_anomalies(
test_id: str, dbt_project: DbtProject
):
Expand All @@ -54,8 +49,6 @@ def test_anomalyless_no_timestamp_column_anomalies(
assert test_result["status"] == "pass"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalous_column_anomalies(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand All @@ -79,8 +72,6 @@ def test_anomalous_column_anomalies(test_id: str, dbt_project: DbtProject):
assert test_result["status"] == "fail"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_column_anomalies_with_where_parameter(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -138,8 +129,6 @@ def test_column_anomalies_with_where_parameter(test_id: str, dbt_project: DbtPro
assert test_result["status"] == "fail"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_column_anomalies_with_timestamp_as_sql_expression(
test_id: str, dbt_project: DbtProject
):
Expand Down Expand Up @@ -182,8 +171,6 @@ def test_column_anomalies_with_timestamp_as_sql_expression(
drop_failure_percent_threshold=5,
metric_value=1,
)
# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_volume_anomaly_static_data_drop(
test_id: str,
dbt_project: DbtProject,
Expand Down Expand Up @@ -228,8 +215,6 @@ def test_volume_anomaly_static_data_drop(
assert test_result["status"] == expected_result


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_column_anomalies_group(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
data: List[Dict[str, Any]] = [
Expand All @@ -246,8 +231,6 @@ def test_anomalyless_column_anomalies_group(test_id: str, dbt_project: DbtProjec
assert test_result["status"] == "pass"


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_column_anomalies_group_by(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -301,8 +284,6 @@ def test_column_anomalies_group_by(test_id: str, dbt_project: DbtProject):
assert test_result["failures"] == 2


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_column_anomalies_group_by_none_dimension(
test_id: str, dbt_project: DbtProject
):
Expand Down Expand Up @@ -347,8 +328,6 @@ def test_anomalyless_column_anomalies_group_by_none_dimension(
assert test_result["failures"] == 2


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_column_anomalies_group_by_multi(
test_id: str, dbt_project: DbtProject
):
Expand Down Expand Up @@ -400,8 +379,6 @@ def test_anomalyless_column_anomalies_group_by_multi(
assert test_result["failures"] == 3


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalyless_column_anomalies_group_by_description(
test_id: str, dbt_project: DbtProject
):
Expand Down Expand Up @@ -436,8 +413,6 @@ def test_anomalyless_column_anomalies_group_by_description(
assert "not enough data" not in test_result["test_results_description"].lower()


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_anomalous_boolean_column_anomalies(test_id: str, dbt_project: DbtProject):
utc_today = datetime.utcnow().date()
test_date, *training_dates = generate_dates(base_date=utc_today - timedelta(1))
Expand Down Expand Up @@ -478,8 +453,6 @@ def test_anomalous_boolean_column_anomalies(test_id: str, dbt_project: DbtProjec
}


# Anomalies currently not supported on ClickHouse
@pytest.mark.skip_targets(["clickhouse"])
def test_col_anom_excl_detect_train(test_id: str, dbt_project: DbtProject):
"""
Test the exclude_detection_period_from_training flag functionality for column anomalies.
Expand Down
Loading
Loading