fix(dbt): fix 3 buggy starrocks__ cross-db macro variants#2401
Open
blarghmatey wants to merge 1 commit into
Open
fix(dbt): fix 3 buggy starrocks__ cross-db macro variants#2401blarghmatey wants to merge 1 commit into
blarghmatey wants to merge 1 commit into
Conversation
Doc-verified against docs.starrocks.io: - starrocks__json_extract_value used json_extract, which does not exist in StarRocks; switched to json_query(parse_json(x), path). - starrocks__from_iso8601_timestamp did a bare cast(x as datetime), which returns NULL for strings with a trailing 'Z' or '+HH:MM'/'-HH:MM' offset since StarRocks DATETIME is zone-less; strip the offset suffix and 'T' separator before casting. - starrocks__array_join silently dropped the null_replacement argument, even though StarRocks' array_join supports the same 3-arg null_replace_str signature as Trino; mirrored the default__ branch structure. Closes #2397 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the StarRocks-dispatched variants of existing cross-database dbt macros in src/ol_dbt/macros/cross_db_functions.sql to align with StarRocks function availability and parsing behavior, improving semantic correctness for a future StarRocks target.
Changes:
- Fix
starrocks__json_extract_valueto usejson_query(parse_json(...), path)instead of a non-existentjson_extract. - Fix
starrocks__from_iso8601_timestampto strip timezone suffixes and normalize theTseparator before casting to StarRocksDATETIME. - Fix
starrocks__array_jointo correctly pass the optionalnull_replacement(3rd argument) when provided.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+76
to
+81
| {# | ||
| StarRocks: DATETIME is zone-less, and a bare cast() returns NULL for strings with | ||
| a trailing 'Z' or '+HH:MM'/'-HH:MM' offset. Strip any such suffix and the 'T' | ||
| separator before casting. Non-UTC offsets are dropped rather than applied, which | ||
| is acceptable since this project's source data is UTC/naive. | ||
| #} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What are the relevant tickets?
Closes #2397
Description (What does it do?)
Fixes 3 correctness bugs in the
starrocks__dispatch variants ofmacros/cross_db_functions.sql, doc-verified against docs.starrocks.io as part of the dbt warehouse semantic-accuracy audit's StarRocks cross-database macro epic:starrocks__json_extract_valueusedjson_extract(), which does not exist in StarRocks. Switched tojson_query(parse_json(x), path).starrocks__from_iso8601_timestampdid a barecast(x as datetime). StarRocksDATETIMEis zone-less, so any string with a trailingZor±HH:MMoffset made the cast silently returnNULL. Now strips a trailing zone suffix and normalizes theTseparator to a space before casting. Non-UTC offsets are dropped rather than applied — acceptable since this project's data is UTC/naive.starrocks__array_joinsilently dropped thenull_replacementargument even though StarRocks'array_joinsupports the same 3-argnull_replace_strsignature as Trino (per StarRocks docs:array_join([1,3,5,null],'_','NULL')). Mirrored thedefault__(Trino) branch structure.How can this be tested?
dbt parseagainst thedev_localDuckDB target succeeds (verified) — these macros aren't dispatched today since no model targets StarRocks yet, so this is Jinja-syntax + doc-verification, not a live StarRocks run.select {{ json_extract_value("'{\"a\":1}'", "'$.a'") }}select {{ from_iso8601_timestamp("'2024-01-01T00:00:00Z'") }}select {{ array_join('array(1,3,5,null)', '_', 'NULL') }}against a live StarRocks warehouse and confirm non-NULL, correctly-formatted results.
Additional Context
Part of
tk-epic-starrocks-support-for-cross-database-dbt-ma-968247in the dbt warehouse audit — a sibling task (tk-implement-18-missing-starrocks-macro-variants-do-57059b) tracks implementing the 18 currently-missingstarrocks__macro variants; not addressed here.🤖 Generated with Claude Code