Feat(lineage): add support for UNPIVOT#7729
Merged
Merged
Conversation
geooo109
approved these changes
Jun 10, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Adds lineage tracing support for UNPIVOT so that lineage() resolves UNPIVOT-generated value/name columns back to the source columns listed in the IN (...) clause (fixing the phantom-column behavior described in #7727).
Changes:
- Extend lineage pivot-handling logic to cover
UNPIVOTby mapping UNPIVOT output columns to theirIN (...)source columns. - Refactor existing PIVOT column mapping logic into a shared helper (
_pivot_column_mapping). - Add lineage tests for
UNPIVOT, including coverage for UNPIVOT over a CTE.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/test_lineage.py |
Adds regression tests asserting that score/metric_name trace to the IN (...) source columns, including through a CTE. |
sqlglot/lineage.py |
Implements shared (UN)PIVOT output-to-source column mapping and uses it during lineage expansion (including a CTE tracing workaround). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+469
to
+488
| def _pivot_column_mapping(pivot: exp.Pivot) -> dict[str, list[exp.Column]]: | ||
| """Map each (UN)PIVOT output column name to the source columns it's derived from.""" | ||
| mapping: dict[str, list[exp.Column]] = {} | ||
|
|
||
| if pivot.unpivot: | ||
| # UNPIVOT(val FOR name IN (a, b)): both the value column(s) and the name column | ||
| # are derived from the IN-list source columns | ||
| unpivot_columns = [ | ||
| col | ||
| for field in pivot.fields | ||
| for e in field.expressions | ||
| for col in e.find_all(exp.Column) | ||
| ] | ||
| for value_column in pivot.expressions: | ||
| for identifier in value_column.find_all(exp.Identifier): | ||
| mapping[identifier.name] = unpivot_columns | ||
| for field in pivot.fields: | ||
| if isinstance(field, exp.In): | ||
| mapping[field.this.name] = unpivot_columns | ||
|
|
Contributor
SQLGlot Integration Test Results✅ All tests passedComparing: Overallmain: 192441 total, 153536 passed (pass rate: 79.8%) sqlglot:jo/unpivot_lineage: 180247 total, 142391 passed (pass rate: 79.0%) Transitions: Dialect pair changes: 0 previous results not found, 3 current results not found ✅ All tests passed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7727