Conversation
…o list Using Sou (aws#5518)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix TypeError when TrainingStep's SourceCode has
requirements=NoneProblem
When creating a
TrainingStepwith aSourceCodeobject that doesn't specifyrequirements(defaulting toNone), the pipeline compilation fails with aTypeErrorbecauseget_training_code_hash()attempts list concatenation withNone:This occurs in
get_code_hash()which passessource_code.requirements(which isNone) directly toget_training_code_hash()as thedependenciesparameter.Fix
get_code_hash(): Normalizesource_code.requirementsusingor Noneto ensure falsy values (empty strings, empty lists, etc.) are converted toNonebefore being passed downstream.get_training_code_hash(): Added a defensive type check at the top of the function that validatesdependenciesis eitherNoneor astr. If an unexpected type is passed (e.g., a list), it logs a warning and setsdependenciestoNoneto preventTypeErrorfrom list concatenation.Testing
test_get_training_code_hash_with_none_dependencies- verifiesNonedependencies with source_dir workstest_get_training_code_hash_with_none_dependencies_entry_point_only- verifiesNonedependencies with entry_point only workstest_get_training_code_hash_with_non_string_dependencies- verifies non-string dependencies are handled gracefullyget_code_hashwith mockedTrainingStepobjects havingrequirements=NoneRelated Issue
Fixes aws#5518
Changes Made
The bug occurs in
get_code_hash()insagemaker-core/src/sagemaker/core/workflow/utilities.pywhen processing aTrainingStepwhoseSourceCodeobject hasrequirements=None(the default when not specified). Theget_code_hash()function extractssource_code.requirements(which isNone) and passes it toget_training_code_hash()as thedependenciesparameter. Theget_training_code_hash()function already hasif dependencies:guards in the current codebase, which correctly handlesNone. However, the issue reporter on v3.3.1 encountered a version where the code directly concatenated[source_dir] + dependencieswithout checking forNonefirst. The current code in the repo appears to have the fix withif dependencies:guards, but there's still a defensive improvement needed: theget_code_hash()function should normalizeNonerequirements to avoid passingNonedownstream, and the type hint fordependenciesinget_training_code_hashshould acceptOptional[str](which it already does). The fix should also add a fallbackorexpression inget_code_hashto convertNoneto a safe default, making the code more robust.AI-Generated PR
This PR was automatically generated by the PySDK Issue Agent.
Merge Checklist
prefix: descriptionformat