Conversation
…_name_split - Add trino__full_name_split: Trino arrays are 1-based, not 0-based like the default - Add trino__edr_get_create_table_as_sql: bypass dbt-trino's create_table_as which accesses model.config (fails outside model materialization context) - Add spark__edr_get_create_table_as_sql: use temporary view for temp tables Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
👋 @devin-ai-integration[bot] |
📝 WalkthroughWalkthroughThis pull request adds database adapter support for Trino and Spark by introducing three new public macros. A Trino-specific name-splitting macro handles database/schema/table name extraction, while Trino and Spark-specific table creation macros provide simplified SQL generation paths that bypass existing model context dependencies. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Trino requires TRIM(BOTH '"' FROM value) instead of TRIM(value, '"') Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>
There was a problem hiding this comment.
🧹 Nitpick comments (1)
macros/utils/table_operations/create_table_as.sql (1)
94-99: Consider documenting behavior whentemporary=trueis requested.The
temporaryparameter is silently ignored, always creating a permanent table. Other adapters with similar constraints (like Redshift with dbt-fusion) include a comment explaining cleanup behavior. Consider adding similar documentation to clarify:
- That temporary table requests create permanent tables
- How/when these tables get cleaned up
📝 Suggested documentation improvement
{% macro trino__edr_get_create_table_as_sql(temporary, relation, sql_query) %} {# dbt-trino's create_table_as accesses model.config which fails when called - outside a model context (e.g. from edr_create_table_as). Use simplified SQL. #} + outside a model context (e.g. from edr_create_table_as). Use simplified SQL. + Note: Trino temporary tables are not supported here - creates regular tables + that are cleaned up by Elementary's normal cleanup logic. #} create table {{ relation }} as {{ sql_query }} {% endmacro %}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@macros/utils/table_operations/create_table_as.sql` around lines 94 - 99, Update the trino__edr_get_create_table_as_sql macro to document that the temporary parameter is ignored: inside the macro (trino__edr_get_create_table_as_sql) add a brief comment stating that when temporary=true a permanent table is created, explain why (dbt-trino model.context limitation), and describe the expected cleanup behavior or how/when callers should remove these tables (e.g., manual cleanup or external job). Ensure the comment references the temporary parameter by name so future readers know it is intentionally not implemented.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@macros/utils/table_operations/create_table_as.sql`:
- Around line 94-99: Update the trino__edr_get_create_table_as_sql macro to
document that the temporary parameter is ignored: inside the macro
(trino__edr_get_create_table_as_sql) add a brief comment stating that when
temporary=true a permanent table is created, explain why (dbt-trino
model.context limitation), and describe the expected cleanup behavior or
how/when callers should remove these tables (e.g., manual cleanup or external
job). Ensure the comment references the temporary parameter by name so future
readers know it is intentionally not implemented.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
macros/edr/system/system_utils/full_names.sqlmacros/utils/table_operations/create_table_as.sql
Summary
Adds Trino- and Spark-specific macro dispatches to fix two categories of runtime errors when elementary runs against these adapters:
trino__full_name_split: The defaultfull_name_splitmacro uses 0-based array indexing on the result ofsplit(). Trino arrays are 1-based, causingTrinoUserError: SQL array indices start at 1in schema_changes_from_baseline tests. The new macro uses indices 1/2/3.trino__edr_get_create_table_as_sql: The default dispatch callsdbt.get_create_table_as_sql()→ dbt-trino'strino__create_table_as, which accessesmodel.config. When called fromedr_create_table_as(outside model materialization), there is no model — only a relation dict — causing'dict object' has no attribute 'config'. The fix uses a simplifiedCREATE TABLE ... ASstatement directly.spark__edr_get_create_table_as_sql: Adds Spark-specific table creation usingCREATE OR REPLACE TEMPORARY VIEWfor temp tables andCREATE TABLEfor regular tables.Review & Testing Checklist for Human
trim()syntax intrino__full_name_split: The macro usestrim(value, '"')(two-argument form). Trino may only supporttrim(BOTH '"' FROM value). Verify this works against an actual Trino instance — the ClickHouse implementation explicitly uses theBOTH ... FROMsyntax for this reason.temporaryflag:trino__edr_get_create_table_as_sqlalways creates permanent tables. Verify this is acceptable given thattrino__has_temp_table_support()returnsfalse.spark__edr_get_create_table_as_sqlusesCREATE TABLE ... AS SELECTwithout aUSINGclause. Verify Spark uses the correct default format (e.g., Hive metastore format or parquet).Test Plan
schema_changes_from_baselinetests pass (should fixSQL array indices start at 1errors)edr monitor reportpasses (should fix'dict object' has no attribute 'config'errors)Notes
Summary by CodeRabbit
Release Notes