Skip to content

SNOW-3489633: remove redundant LIMIT 1 on each COUNT slot subquery#4251

Closed
sfc-gh-yuwang wants to merge 4 commits into
mainfrom
SNOW-3489633
Closed

SNOW-3489633: remove redundant LIMIT 1 on each COUNT slot subquery#4251
sfc-gh-yuwang wants to merge 4 commits into
mainfrom
SNOW-3489633

Conversation

@sfc-gh-yuwang

@sfc-gh-yuwang sfc-gh-yuwang commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator
  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-NNNNNNN

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  3. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

@codecov-commenter

codecov-commenter commented Jun 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.49%. Comparing base (495acce) to head (125b43d).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4251      +/-   ##
==========================================
+ Coverage   95.38%   95.49%   +0.10%     
==========================================
  Files         171      171              
  Lines       44243    44187      -56     
  Branches     7557     7535      -22     
==========================================
- Hits        42203    42198       -5     
+ Misses       1252     1226      -26     
+ Partials      788      763      -25     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sfc-gh-yuwang sfc-gh-yuwang added the NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md label Jun 8, 2026
Comment on lines 691 to 692
# add limit 1 because aggregate may be on non-aggregate function in a scalar aggregation
# for example, df.agg(lit(1))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this assumption legit in the context of SCOS? what cases will be broken if we always remove limit_expression(1) for SCOS?

given the complexity of the implementation (extra tree walk, code maintainability, etc.) and the minimal perf improvement it can help, I'm wondering if it's worth the efforts.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

locally verified that the scenario mentioned in the comment would not break SCOS with current change in snowpark

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any metrics on the improvements from this change? I agree with Adam that this change seems risky for little benefit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about why the no-aggregation function in SCOS avoids the issue after we remove limit 1

However, if the underlying logic is overly complex or confusing, let's skip it. Without a clear understanding, we won't have high confidence against regressions.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a few re-test, it turns out that this change is also not safe for scos, so we will have to add a lot of logic to make it safe for scos like the first version of this change.
At this point, I think it does not worth it to have this change just to remove limit 1.

@sfc-gh-yuwang

Copy link
Copy Markdown
Collaborator Author

@github-actions github-actions Bot locked and limited conversation to collaborators Jun 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants