SNOW-3081273: fix Creating a stored procedure locally results in an unsupported version of cloudpicker#4234
Conversation
…g a Python stored procedure or UDF with `runtime_version='3.13'`. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4234 +/- ##
=======================================
Coverage 95.39% 95.39%
=======================================
Files 171 171
Lines 44241 44243 +2
Branches 7556 7557 +1
=======================================
+ Hits 42204 42206 +2
Misses 1250 1250
Partials 787 787 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
https://snowpark-python-001.jenkinsdev1.us-west-2.aws-dev.app.snowflake.com/job/PythonStoredProcBuildSnowfortTest/957/console |
|
|
||
|
|
||
| @pytest.mark.skipif(IS_IN_STORED_PROC, reason="not supported in stored proc") | ||
| def test_sproc_runtime_313_cloudpickle_ge_spec_compiles_and_executes(session): |
There was a problem hiding this comment.
Not blocking: since this test specifies python==3.13, wondering whether other test matrices with 3.11, 3.12 etc would execute this? Would this be potentially flaky for other python versions, though the merge gate seems to be passing here?
There was a problem hiding this comment.
I think this should be good even if 3.11/3.12 execute this test.
so this change is only making sure that cloudpickle can be correctly picked in stored proc. Say if 3.12 execute this with cloudpickle 2.2.1, it will try to pick cloudpickle >=2.2.1 for py313, as long as this cloudpickle pkg exist on server, the test would not fail. For the situation the test do fail, I think that is more of a packaging issue instead of flakiness
| res.append(m) | ||
| elif isinstance(m, ModuleType) and m.__name__ not in result_dict: | ||
| res.append(f"{m.__name__}=={m.__version__}") | ||
| res.append(f"{m.__name__}>={m.__version__}") |
There was a problem hiding this comment.
Seems like cloudpickle is the only ModuleType that goes through this check now, so we're good. but >= applies to any module a future caller might pass. Can we maybe add a one-line comment noting the intent so nobody fixes it back to ==, or limit the >= logic to cloudpickle only?
There was a problem hiding this comment.
great call out! I'll limit this to only cloudpickle
4 participants
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-3081273
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
This PR fixes a Python 3.13 local-registration issue for Snowpark UDFs/stored procedures by changing auto-injected cloudpickle dependency specs from strict pins (==) to compatible lower bounds (>=), so server-side resolution can choose a supported version instead of failing on an exact local version match.
This change is safe because cloudpickle is backward compatible, meaning higher version can deserialize encoding create by lower version