Skip to content

Commit be30aac

Browse files
jonhopper-dataengineersCortex Code
andauthored
Fix clean macro dropping UDFs with DEFAULT parameters (#27)
* Fix clean macro dropping UDFs with DEFAULT parameters The types-only fallback in has_matching_nodes was extracting types from the dbt side only, while comparing against the original Snowflake signature which still contained parameter names (e.g. TARGET_TIMEZONE VARCHAR). This caused functions with DEFAULT parameters to never match and get dropped as orphans. The fallback now extracts types from both sides before comparing. Bump version to 1.0.2. .... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-Authored-By: Cortex Code <noreply@snowflake.com> * Fix extract_param_type for type-only params with precision When a type-only parameter like NUMBER(15, 2) was passed through collapse_whitespace, the internal space caused it to split into multiple tokens. The first token (e.g. number(15,) was incorrectly treated as a param name. Now checks if the first token contains '(' to detect precision types and reassembles all tokens as the type. .... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-Authored-By: Cortex Code <noreply@snowflake.com> --------- Co-authored-by: Cortex Code <noreply@snowflake.com>
1 parent 44e21d3 commit be30aac

4 files changed

Lines changed: 56 additions & 16 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,14 @@
11
# Data Engineers Snowflake DataOps Utils Project Changelog
22
This file contains the changelog for the Data Engineers Snowflake DataOps Utils project, detailing updates, fixes, and enhancements made to the project over time.
33

4+
## v1.0.2 - 2026-05-12 - Clean Functions Signature Fallback Fix
5+
6+
### Fixed
7+
- Fixed `has_matching_nodes` macro: the types-only fallback comparison was only extracting types from the dbt side while comparing against the original Snowflake signature which still contained parameter names. This caused UDFs with DEFAULT parameters (e.g. `target_timezone STRING DEFAULT 'Pacific/Auckland'`) to be incorrectly dropped, because the Snowflake signature `(TARGET_TIMEZONE VARCHAR)` includes parameter names that didn't match the dbt types-only signature `(varchar)`. The fallback now extracts types from both sides before comparing.
8+
9+
### Changed
10+
- Bumped version from 1.0.1 to 1.0.2
11+
412
## v1.0.1 - 2026-05-10 - Function Signature Matching Fix
513

614
### Fixed

dbt_project.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
name: 'dbt_dataengineers_utils'
2-
version: '1.0.1'
2+
version: '1.0.2'
33
config-version: 2
44

55
require-dbt-version: [">=1.9.4", "<3.0.0"]

macros/clean/_helpers.sql

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -65,15 +65,29 @@
6565
{# Type-only (no param name) #}
6666
{{ return(parts[0]) }}
6767
{% else %}
68-
{# First token is the param name; collect type tokens until we hit "default" #}
69-
{% set type_parts = [] %}
70-
{% for token in parts[1:] %}
71-
{% if token | lower == 'default' %}
72-
{{ return(type_parts | join(' ')) }}
73-
{% else %}
74-
{% do type_parts.append(token) %}
75-
{% endif %}
76-
{% endfor %}
77-
{{ return(type_parts | join(' ')) }}
68+
{# If the first token contains '(' it is a type with precision (e.g. "number(15, 2)")
69+
not a param name, so reassemble all tokens as the type. #}
70+
{% if '(' in parts[0] %}
71+
{% set type_parts = [] %}
72+
{% for token in parts %}
73+
{% if token | lower == 'default' %}
74+
{{ return(type_parts | join(' ')) }}
75+
{% else %}
76+
{% do type_parts.append(token) %}
77+
{% endif %}
78+
{% endfor %}
79+
{{ return(type_parts | join(' ')) }}
80+
{% else %}
81+
{# First token is the param name; collect type tokens until we hit "default" #}
82+
{% set type_parts = [] %}
83+
{% for token in parts[1:] %}
84+
{% if token | lower == 'default' %}
85+
{{ return(type_parts | join(' ')) }}
86+
{% else %}
87+
{% do type_parts.append(token) %}
88+
{% endif %}
89+
{% endfor %}
90+
{{ return(type_parts | join(' ')) }}
91+
{% endif %}
7892
{% endif %}
7993
{% endmacro %}

macros/clean/has_matching_nodes.sql

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,11 @@
2929
{% endif %}
3030

3131
{# ── Fallback: compare types-only signatures ──
32-
Snowflake information_schema returns type-only signatures e.g. (VARCHAR)
33-
while dbt parameters may include names and DEFAULT clauses e.g.
34-
"target_timezone VARCHAR DEFAULT 'Pacific/Auckland'".
32+
Snowflake information_schema may return signatures with param names
33+
e.g. (TARGET_TIMEZONE VARCHAR) while dbt parameters include names and
34+
DEFAULT clauses e.g. "target_timezone VARCHAR DEFAULT 'Pacific/Auckland'".
3535
Parameters may also contain parenthesised precision e.g. NUMBER(3, 0).
36-
Extract just the types from the dbt side and compare again. #}
36+
Extract just the types from BOTH sides and compare again. #}
3737
{% set dbt_types = [] %}
3838
{% set clean_dbt_args = dbt_arguments | trim %}
3939
{% if clean_dbt_args | length > 0 %}
@@ -45,13 +45,31 @@
4545
{% endfor %}
4646
{% endif %}
4747

48+
{# Also extract types-only from the Snowflake side (which may include param names) #}
49+
{% set sql_types = [] %}
50+
{% set clean_sql_args = sql_arguments | replace("(", "", 1) %}
51+
{% if clean_sql_args.endswith(")") %}
52+
{% set clean_sql_args = clean_sql_args[:-1] %}
53+
{% endif %}
54+
{% set clean_sql_args = clean_sql_args | trim %}
55+
{% if clean_sql_args | length > 0 %}
56+
{% for param in dbt_dataengineers_utils.split_params(clean_sql_args) %}
57+
{% set param_type = dbt_dataengineers_utils.extract_param_type(param) %}
58+
{% if param_type | length > 0 %}
59+
{% do sql_types.append(param_type) %}
60+
{% endif %}
61+
{% endfor %}
62+
{% endif %}
63+
4864
{% if name_property == "config.override_name" %}
4965
{% set dbt_types_signature = (node.schema ~ "." ~ node_name ~ "(" ~ dbt_types | join(", ") ~ ")") | lower %}
66+
{% set sql_types_signature = (sql_object_schema ~ "." ~ sql_object_name ~ "(" ~ sql_types | join(", ") ~ ")") | lower %}
5067
{% else %}
5168
{% set dbt_types_signature = (node.schema ~ "." ~ node.name ~ "(" ~ dbt_types | join(", ") ~ ")") | lower %}
69+
{% set sql_types_signature = (sql_object_schema ~ "." ~ sql_object_name ~ "(" ~ sql_types | join(", ") ~ ")") | lower %}
5270
{% endif %}
5371

54-
{% if sql_signature == dbt_types_signature %}
72+
{% if sql_types_signature == dbt_types_signature %}
5573
{{ return(true) }}
5674
{% endif %}
5775
{% endif %}

0 commit comments

Comments
 (0)