Skip to content

SNOW-2356405: dbapi pyodbc and udtf fixes#3805

Merged
sfc-gh-aling merged 10 commits into
mainfrom
SNOW-2356405-dbapi-stored-proc-local-ingestion-perf-vs-jdbc-perf
Sep 26, 2025
Merged

SNOW-2356405: dbapi pyodbc and udtf fixes#3805
sfc-gh-aling merged 10 commits into
mainfrom
SNOW-2356405-dbapi-stored-proc-local-ingestion-perf-vs-jdbc-perf

Conversation

@sfc-gh-aling

Copy link
Copy Markdown
Collaborator
  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-NNNNNNN

  • Fixed the following bugs in DataFrameReader.dbapi(PuPr):
    • udtf ingestion does not work for pyodbc driver.
    • query input does not work for SQL Server.
    • nullability of output schema of udft ingestion is not preserved.
  • Improved DataFrameReader.dbapi (PuPr) reading performance by setting the default fetch_size parameter value to 10000.
  1. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  2. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

@sfc-gh-aling sfc-gh-aling force-pushed the SNOW-2356405-dbapi-stored-proc-local-ingestion-perf-vs-jdbc-perf branch from 7fa1665 to c017657 Compare September 25, 2025 17:03
@sfc-gh-yuwang

Copy link
Copy Markdown
Collaborator

should we also trigger a snowfort test to verify this change is safe for server?

@sfc-gh-yuwang sfc-gh-yuwang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-approval assume changelog nint will be fixed

Comment thread CHANGELOG.md Outdated
@sfc-gh-aling

Copy link
Copy Markdown
Collaborator Author

@sfc-gh-yuwang we can and I just triggered. I will do extra manual test to ensure the features work as expected in the sproc env.

Comment thread CHANGELOG.md Outdated

class SqlServerDialect(BaseDialect):
pass
def generate_select_query(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious about how this is used, is it related to the bug fixes?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is related to the bug query input does not work for SQL Server.

default base driver add backtik to column name which is invalid in sql server.
the fix is to use double quote

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the changelog, hope the bug is easier to understand now

@sfc-gh-aling sfc-gh-aling merged commit 02e594e into main Sep 26, 2025
26 of 28 checks passed
@sfc-gh-aling sfc-gh-aling deleted the SNOW-2356405-dbapi-stored-proc-local-ingestion-perf-vs-jdbc-perf branch September 26, 2025 19:11
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 26, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants