fix: skip DESCRIBE TABLE EXTENDED AS JSON for foreign/federated tables#1472
Merged
sd-db merged 7 commits intoJun 2, 2026
Merged
Conversation
Foreign/federated tables do not support the AS JSON form, causing the adapter to attempt it (and fail) before falling back to plain DESCRIBE TABLE on every code path that calls get_columns_in_relation. Add relation.is_foreign_table to the use_legacy_logic guard, consistent with the existing check in is_describe_as_json_supported(). Signed-off-by: willweld <williamweld@outlook.com>
Signed-off-by: willweld <williamweld@outlook.com>
Add a UC functional test that verifies get_columns_in_relation skips the DESCRIBE AS JSON path for Foreign relations, plus a unit test assertion that the legacy macro is invoked exactly once.
sd-db
approved these changes
Jun 1, 2026
sd-db
left a comment
sd-db
left a comment
Collaborator
There was a problem hiding this comment.
Changes look good, I am looking to add a functional test + run all functional tests and will merge. Thanks for the PR !
Collaborator
|
/integration-test |
|
Integration tests dispatched for PR #1472 by @sd-db. Track progress in the Actions tab. |
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
|
Integration results for PR #1472 — UC cluster ✅ success · SQL warehouse ✅ success · All-purpose cluster ✅ success · Shard coverage ✅ success |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #
Description
For federated/foreign tables,
DESCRIBE TABLE EXTENDED ... AS JSONis not supported. The adapter was attempting it on everyget_columns_in_relationcall before catching the error and falling back to plainDESCRIBE TABLE, causing repeated failures and extra latency on affected code paths (e.g. incremental merge strategy, table create).The fix adds
relation.is_foreign_tableto theuse_legacy_logicguard inget_columns_in_relation, so the JSON path is never attempted for foreign tables. This is consistent with the existing check inis_describe_as_json_supported()which already excludes foreign tables — the column-fetch path was simply missing the same guard.No new infrastructure is needed:
DatabricksRelationType.Foreignandrelation.is_foreign_tablealready exist.Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.Note: I do not have access to a Databricks workspace with federated tables to run functional tests against. Per CONTRIBUTING.md, I'm flagging this for a maintainer
/integration-testrun.