Skip to content

Commit ab21363

Browse files
fix: set explicit incremental_strategy='append' for test_result_rows.sql (#918)
* fix: remove incremental_strategy from test_result_rows.sql The model no longer has a unique_key (removed in f5c062d), so using merge incremental strategy causes Redshift to fail with: 'MERGE statement with a single WHEN clause is not enabled.' Removing incremental_strategy lets dbt use the default append strategy which is appropriate for this append-only table. Fixes CORE-279 Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> * ci: update Python version to 3.10 for tests The elementary-data package now requires Python >=3.10, so update the test workflow to use Python 3.10 instead of 3.9. Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> * fix: explicitly set incremental_strategy='append' for test_result_rows.sql Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> * fix: add get_append_only_incremental_strategy macro for append-only tables - Creates new macro that returns 'append' for Athena and None for other adapters - Uses this macro in test_result_rows.sql instead of hardcoded 'append' - Fixes Athena's default insert_overwrite strategy issue while allowing other adapters to use their defaults Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> * fix: update get_append_only_incremental_strategy to return append for all except BigQuery - Default now returns 'append' for all adapters - BigQuery-specific override returns 'merge' (BigQuery doesn't support append) - Removed Athena-specific override (now covered by default) Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Itamar Hartstein <haritamar@gmail.com>
1 parent b8ebc80 commit ab21363

3 files changed

Lines changed: 14 additions & 2 deletions

File tree

.github/workflows/test-warehouse.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ jobs:
112112
- name: Setup Python
113113
uses: actions/setup-python@v6
114114
with:
115-
python-version: "3.9"
115+
python-version: "3.10"
116116
cache: "pip"
117117

118118
- name: Install Spark requirements

macros/utils/cross_db_utils/incremental_strategy.sql

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,15 @@
1717
{% macro default__get_default_incremental_strategy() %}
1818
{% do return(none) %}
1919
{% endmacro %}
20+
21+
{% macro get_append_only_incremental_strategy() %}
22+
{% do return(adapter.dispatch("get_append_only_incremental_strategy", "elementary")()) %}
23+
{% endmacro %}
24+
25+
{%- macro bigquery__get_append_only_incremental_strategy() %}
26+
{% do return("merge") %}
27+
{% endmacro %}
28+
29+
{% macro default__get_append_only_incremental_strategy() %}
30+
{% do return("append") %}
31+
{% endmacro %}

models/edr/run_results/test_result_rows.sql

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
"prev_timestamp_column": "detected_at",
1111
},
1212
table_type=elementary.get_default_table_type(),
13-
incremental_strategy=elementary.get_default_incremental_strategy()
13+
incremental_strategy=elementary.get_append_only_incremental_strategy()
1414
)
1515
}}
1616

0 commit comments

Comments
 (0)