SNOW-3375781: Preserve 1-pass read for SCOS XML user schema performance#4185
Merged
Conversation
…of inferSchema value
|
All contributors have signed the CLA ✍️ ✅ |
Collaborator
Author
|
I have read the CLA Document and I hereby sign the CLA |
sfc-gh-yuwang
approved these changes
Apr 20, 2026
sfc-gh-bkogan
approved these changes
Apr 20, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4185 +/- ##
=======================================
Coverage 95.42% 95.42%
=======================================
Files 171 171
Lines 43615 43684 +69
Branches 7459 7475 +16
=======================================
+ Hits 41620 41686 +66
- Misses 1220 1221 +1
- Partials 775 777 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-3375781
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
Please write a short description of how your code change solves the related issue.
Add internal
_XML_SKIP_INFERENCEoption to preserve 1-pass read in SCOS XML with user schema.Snowpark Python now supports XML
inferSchema=Falseto produce all StringType columns for leaf tags while preserving nested structures (to be released in v1.50.0). Since SCOS custom-schema turns off inferSchema flag, a 2-pass read would be triggered in Snowpark after release.This PR serves to preserve the 1-pass read performance in SCOS XML custom-schema scenarios to adhere to Spark behaviors -- no behavioral changes whatsoever, pure performance improvement. The private
_XML_SKIP_INFERENCEoption in SCOS is an internal contract to preserve performance.Relevant SCOS PR: https://github.com/snowflake-eng/sas/pull/3687