Skip to content

SNOW-3489991: Block flattening of dropped NEW columns in connect mode#4259

Draft
sfc-gh-joshi wants to merge 2 commits into
mainfrom
joshi-SNOW-3489991-withcolumn-flatten
Draft

SNOW-3489991: Block flattening of dropped NEW columns in connect mode#4259
sfc-gh-joshi wants to merge 2 commits into
mainfrom
joshi-SNOW-3489991-withcolumn-flatten

Conversation

@sfc-gh-joshi

Copy link
Copy Markdown
Contributor

In can_select_statement_be_flattened, the existing guard only refused to flatten when a dropped NEW column was also is_referenced_by_same_level_column. This missed the case where a future filter/sort (not yet added at select() time) references the dropped column, generating invalid SQL.

In connect mode, extend the guard to block flattening unconditionally when a NEW column is dropped. This preserves the defining subquery and keeps the column visible to subsequent WHERE/ORDER BY clauses. Non-connect behavior is unchanged.

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-3489991

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  3. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

@codecov-commenter

codecov-commenter commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.49%. Comparing base (9975596) to head (b1a9a74).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4259   +/-   ##
=======================================
  Coverage   95.49%   95.49%           
=======================================
  Files         171      171           
  Lines       44243    44244    +1     
  Branches     7548     7548           
=======================================
+ Hits        42252    42253    +1     
  Misses       1226     1226           
  Partials      765      765           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sfc-gh-joshi and others added 2 commits June 11, 2026 21:45
In can_select_statement_be_flattened, the existing guard only refused to
flatten when a dropped NEW column was also is_referenced_by_same_level_column.
This missed the case where a future filter/sort (not yet added at select()
time) references the dropped column, generating invalid SQL.

In connect mode, extend the guard to block flattening unconditionally when
a NEW column is dropped. This preserves the defining subquery and keeps the
column visible to subsequent WHERE/ORDER BY clauses. Non-connect behavior
is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tement flag

Instead of checking `context._is_snowpark_connect_compatible_mode` broadly in
`can_select_statement_be_flattened` (which blocked flattening for *all* dropped-NEW
column cases in connect mode), introduce a `protect_dropped_new_columns` bool on
`SelectStatement`.  SCOS sets this flag specifically on the select statement produced
by the withColumn path; `can_select_statement_be_flattened` honours it only for that
statement.  Non-withColumn paths (e.g. `select().order_by().select()`) are unaffected
and continue to flatten normally, keeping generated SQL concise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sfc-gh-joshi sfc-gh-joshi force-pushed the joshi-SNOW-3489991-withcolumn-flatten branch from 25dd36a to b1a9a74 Compare June 11, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants