Skip to content

Commit f34118a

Browse files
nargokulnargokul
authored
Merge branch 'master' into dependency-upgrade
2 parents f3e03b5 + 0e4cdfd commit f34118a

418 files changed

Lines changed: 23078 additions & 64554 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
name: Bug report
3+
about: File a report to help us reproduce and fix the problem
4+
title: ''
5+
labels: 'bug'
6+
assignees: ''
7+
8+
---
9+
10+
**PySDK Version**
11+
- [ ] PySDK V2 (2.x)
12+
- [ ] PySDK V3 (3.x)
13+
14+
**Describe the bug**
15+
A clear and concise description of what the bug is.
16+
17+
**To reproduce**
18+
A clear, step-by-step set of instructions to reproduce the bug.
19+
The provided code need to be **complete** and **runnable**, if additional data is needed, please include them in the issue.
20+
21+
**Expected behavior**
22+
A clear and concise description of what you expected to happen.
23+
24+
**Screenshots or logs**
25+
If applicable, add screenshots or logs to help explain your problem.
26+
27+
**System information**
28+
A description of your system. Please provide:
29+
- **SageMaker Python SDK version**:
30+
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
31+
- **Framework version**:
32+
- **Python version**:
33+
- **CPU or GPU**:
34+
- **Custom Docker image (Y/N)**:
35+
36+
**Additional context**
37+
Add any other context about the problem here.

‎.github/ISSUE_TEMPLATE/config.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
blank_issues_enabled: false
2+
contact_links:
3+
- name: Ask a question
4+
url: https://github.com/aws/sagemaker-python-sdk/discussions
5+
about: Use GitHub Discussions to ask and answer questions

‎.github/ISSUE_TEMPLATE/documentation-request.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
name: Documentation request
3+
about: Request improved documentation
4+
title: ''
5+
labels: ''
6+
assignees: ''
7+
8+
---
9+
10+
**What did you find confusing? Please describe.**
11+
A clear and concise description of what you found confusing. Ex. I tried to [...] but I didn't understand how to [...]
12+
13+
**Describe how documentation can be improved**
14+
A clear and concise description of where documentation was lacking and how it can be improved.
15+
16+
**Additional context**
17+
Add any other context or screenshots about the documentation request here.

‎.github/ISSUE_TEMPLATE/feature_request.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
name: Feature request
3+
about: Suggest new functionality for this library
4+
title: ''
5+
labels: 'feature request'
6+
assignees: ''
7+
8+
---
9+
10+
**Describe the feature you'd like**
11+
A clear and concise description of the functionality you want.
12+
13+
**How would this feature be used? Please describe.**
14+
A clear and concise description of the use case for this feature. Please provide an example, if possible.
15+
16+
**Describe alternatives you've considered**
17+
A clear and concise description of any alternative solutions or features you've considered.
18+
19+
**Additional context**
20+
Add any other context or screenshots about the feature request here.

‎.github/workflows/pr-checks-master-v2.yml‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
name: Sagemaker PR Checks (Master-v2)
2+
on:
3+
pull_request_target:
4+
branches:
5+
- "master-v2"
6+
7+
concurrency:
8+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.head_ref }}
9+
cancel-in-progress: true
10+
11+
permissions:
12+
id-token: write
13+
14+
jobs:
15+
collab-check:
16+
runs-on: ubuntu-latest
17+
outputs:
18+
approval-env: ${{ steps.collab-check.outputs.result }}
19+
steps:
20+
- name: Collaborator Check
21+
uses: actions/github-script@v7
22+
id: collab-check
23+
with:
24+
github-token: ${{ secrets.COLLAB_CHECK_TOKEN }}
25+
result-encoding: string
26+
script: |
27+
try {
28+
const res = await github.rest.repos.checkCollaborator({
29+
owner: context.repo.owner,
30+
repo: context.repo.repo,
31+
username: "${{ github.event.pull_request.user.login }}",
32+
});
33+
console.log("Verifed ${{ github.event.pull_request.user.login }} is a repo collaborator. Auto Approving PR Checks.")
34+
return res.status == "204" ? "auto-approve" : "manual-approval"
35+
} catch (error) {
36+
console.log("${{ github.event.pull_request.user.login }} is not a collaborator. Requiring Manual Approval to run PR Checks.")
37+
return "manual-approval"
38+
}
39+
wait-for-approval:
40+
runs-on: ubuntu-latest
41+
needs: [collab-check]
42+
environment: ${{ needs.collab-check.outputs.approval-env }}
43+
steps:
44+
- run: echo "Workflow Approved! Starting PR Checks."
45+
codestyle-doc-tests:
46+
runs-on: ubuntu-latest
47+
needs: [wait-for-approval]
48+
steps:
49+
- name: Configure AWS Credentials
50+
uses: aws-actions/configure-aws-credentials@v4
51+
with:
52+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
53+
aws-region: us-west-2
54+
role-duration-seconds: 10800
55+
- name: Run Codestyle & Doc Tests
56+
uses: aws-actions/aws-codebuild-run-build@v1
57+
with:
58+
project-name: ${{ github.event.repository.name }}-ci-codestyle-doc-tests
59+
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
60+
unit-tests:
61+
runs-on: ubuntu-latest
62+
needs: [wait-for-approval]
63+
strategy:
64+
fail-fast: false
65+
matrix:
66+
python-version: ["py39","py310","py311","py312"]
67+
steps:
68+
- name: Configure AWS Credentials
69+
uses: aws-actions/configure-aws-credentials@v4
70+
with:
71+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
72+
aws-region: us-west-2
73+
role-duration-seconds: 10800
74+
- name: Run Unit Tests
75+
uses: aws-actions/aws-codebuild-run-build@v1
76+
with:
77+
project-name: ${{ github.event.repository.name }}-ci-unit-tests
78+
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
79+
env-vars-for-codebuild: |
80+
PY_VERSION
81+
env:
82+
PY_VERSION: ${{ matrix.python-version }}
83+
integ-tests:
84+
runs-on: ubuntu-latest
85+
needs: [wait-for-approval]
86+
steps:
87+
- name: Configure AWS Credentials
88+
uses: aws-actions/configure-aws-credentials@v4
89+
with:
90+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
91+
aws-region: us-west-2
92+
role-duration-seconds: 10800
93+
- name: Run Integ Tests
94+
uses: aws-actions/aws-codebuild-run-build@v1
95+
with:
96+
project-name: ${{ github.event.repository.name }}-ci-integ-tests
97+
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'

.github/workflows/submodule-codebuild-ci.yml renamed to .github/workflows/pr-checks-master.yml

Lines changed: 69 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
name: Sagemaker PR Checks
1+
name: Sagemaker PR Checks (Master)
22
on:
33
pull_request_target:
44
branches:
5-
- "master*"
5+
- "master"
66
paths:
77
- 'sagemaker-train/**'
88
- 'sagemaker-serve/**'
@@ -56,51 +56,97 @@ jobs:
5656
- uses: actions/checkout@v3
5757
with:
5858
fetch-depth: 0
59-
token: ${{ secrets.GH_PAT }} # or use appropriate token
60-
ref: ${{ github.event.pull_request.base.ref }} # Target branch (master-v3)
59+
token: ${{ secrets.GH_PAT }}
60+
ref: ${{ github.event.pull_request.base.ref }}
6161
- name: Detect Changes
6262
id: check-changes
6363
run: |
64-
set -e # Exit on error
64+
set -e
6565
66-
# Debug information
6766
echo "Target Branch: ${{ github.event.pull_request.base.ref }}"
6867
echo "Current Target SHA: $(git rev-parse HEAD)"
6968
echo "PR Number: ${{ github.event.pull_request.number }}"
7069
echo "PR Latest SHA: ${{ github.event.pull_request.head.sha }}"
71-
# Fetch PR without creating a branch
70+
7271
git fetch origin pull/${{ github.event.pull_request.number }}/head
7372
CHANGES=$(git diff --name-only HEAD FETCH_HEAD)
7473
7574
echo "Changed files:"
7675
echo "$CHANGES"
7776
78-
SUBMODULES=[]
77+
# Function to extract dependencies from pyproject.toml
78+
get_dependencies() {
79+
local module=$1
80+
grep "sagemaker-" "$module/pyproject.toml" | grep -o 'sagemaker-[a-z]*' | sort -u
81+
}
82+
83+
# Function to find all modules that depend on a given module (recursively)
84+
find_dependents() {
85+
local target=$1
86+
local all_modules=("sagemaker-core" "sagemaker-train" "sagemaker-serve" "sagemaker-mlops")
87+
local dependents=()
88+
89+
for module in "${all_modules[@]}"; do
90+
if [ "$module" != "$target" ]; then
91+
if get_dependencies "$module" | grep -q "^$target$"; then
92+
dependents+=("$module")
93+
fi
94+
fi
95+
done
96+
97+
echo "${dependents[@]}"
98+
}
99+
100+
# Initialize set of submodules to test (using associative array)
101+
declare -A SUBMODULES_SET
102+
103+
# Function to recursively add module and all its dependents
104+
add_module_and_dependents() {
105+
local module=$1
106+
107+
if [ -z "${SUBMODULES_SET[$module]}" ]; then
108+
SUBMODULES_SET["$module"]=1
109+
echo "Adding $module to test set"
110+
111+
# Find all modules that depend on this one and add them recursively
112+
local dependents=$(find_dependents "$module")
113+
for dependent in $dependents; do
114+
add_module_and_dependents "$dependent"
115+
done
116+
fi
117+
}
118+
119+
# Check which submodules changed and add them plus their dependents
120+
if echo "$CHANGES" | grep -q "^sagemaker-core/"; then
121+
echo "sagemaker-core changed - will add core and all dependents"
122+
add_module_and_dependents "sagemaker-core"
123+
fi
79124
80125
if echo "$CHANGES" | grep -q "^sagemaker-train/"; then
81-
SUBMODULES='["sagemaker-train"]'
126+
echo "sagemaker-train changed - will add train and all dependents"
127+
add_module_and_dependents "sagemaker-train"
82128
fi
129+
83130
if echo "$CHANGES" | grep -q "^sagemaker-serve/"; then
84-
if [ "$SUBMODULES" = '[]' ]; then
85-
SUBMODULES='["sagemaker-serve"]'
86-
else
87-
SUBMODULES=$(echo $SUBMODULES | sed 's/\]$/,"sagemaker-serve"\]/')
88-
fi
131+
echo "sagemaker-serve changed - will add serve and all dependents"
132+
add_module_and_dependents "sagemaker-serve"
89133
fi
134+
90135
if echo "$CHANGES" | grep -q "^sagemaker-mlops/"; then
91-
if [ "$SUBMODULES" = '[]' ]; then
92-
SUBMODULES='["sagemaker-mlops"]'
93-
else
94-
SUBMODULES=$(echo $SUBMODULES | sed 's/\]$/,"sagemaker-mlops"\]/')
95-
fi
136+
echo "sagemaker-mlops changed - will add mlops"
137+
add_module_and_dependents "sagemaker-mlops"
96138
fi
97-
if echo "$CHANGES" | grep -q "^sagemaker-core/"; then
139+
140+
# Convert associative array to JSON array
141+
SUBMODULES='[]'
142+
for submodule in "${!SUBMODULES_SET[@]}"; do
98143
if [ "$SUBMODULES" = '[]' ]; then
99-
SUBMODULES='["sagemaker-core"]'
144+
SUBMODULES="[\"$submodule\"]"
100145
else
101-
SUBMODULES=$(echo $SUBMODULES | sed 's/\]$/,"sagemaker-core"\]/')
146+
SUBMODULES=$(echo $SUBMODULES | sed "s/\]$/,\"$submodule\"\]/")
102147
fi
103-
fi
148+
done
149+
104150
echo "Final SUBMODULES: $SUBMODULES"
105151
echo "submodules=$SUBMODULES" >> $GITHUB_OUTPUT
106152

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@ scratch/
1818
*.egg
1919
examples/tensorflow/distributed_mnist/data
2020
*.iml
21+
22+
# Sphinx documentation
23+
docs/_build/
2124
doc/_build
2225
doc/_static
2326
doc/_templates

‎.readthedocs.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Read the Docs configuration file
2+
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
3+
4+
version: 2
5+
6+
build:
7+
os: ubuntu-22.04
8+
tools:
9+
python: "3.10"
10+
11+
sphinx:
12+
configuration: docs/conf.py
13+
14+
formats:
15+
- pdf
16+
- epub
17+
18+
python:
19+
install:
20+
- requirements: docs/requirements.txt
21+
- method: pip
22+
path: .
23+
extra_requirements:
24+
- docs

‎CHANGELOG.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,40 @@
11
# Changelog
22

3+
## v3.3.1 (2026-01-12)
4+
### Bug fixes and Other Changes
5+
* ProcessingJob fix - Remove tags in Processor while Job creation
6+
* Telemetry Updates
7+
* sagemaker-mlops bug fix - Correct source code 'dependencies' parameter to 'requirements'
8+
* aws_batch bug fix - remove experiment config parameter as it Estimator is deprecated.
9+
10+
11+
## v3.3.0 (2025-12-19)
12+
13+
### Features
14+
* AWS_Batch: queueing of training jobs with ModelTrainer
15+
### Bug fixes and Other Changes
16+
* Fixes for model registry with ModelBuilder
17+
18+
## v3.2.0 (2025-12-18)
19+
20+
### Features
21+
* Evaluator handshake with trainer
22+
* Datasets Format validation
23+
### Bug fixes and Other Changes
24+
* Add xgboost 3.0-5 to release
25+
* Fix get_child_process_ids parsing issue
26+
27+
## v3.1.1 (2025-12-10)
28+
29+
### Bug fixes and Other Changes
30+
* Add validation to bedrock reward models
31+
* Hyperparameter issue fixes, Add validation s3 output path
32+
* Fix the recipe selection for multiple recipe scenario
33+
* Train wait() timeout exception handling
34+
* Update example notebooks to reflect recent code changes
35+
* Update `model_package_group_name` param to `model_package_group` in finetuning interfaces
36+
* remove `dataset` param for benchmark evaluator
37+
338
## v3.1.0 (2025-12-03)
439

540
### Features

‎README.rst‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,6 @@ You can also train and deploy models with **Amazon algorithms**,
3333
which are scalable implementations of core machine learning algorithms that are optimized for SageMaker and GPU training.
3434
If you have **your own algorithms** built into SageMaker compatible Docker containers, you can train and host models using these as well.
3535

36-
For detailed documentation, including the API reference, see `Read the Docs <https://sagemaker.readthedocs.io>`_.
37-
3836
To install SageMaker Python SDK, see `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`_.
3937

4038
❗🔥 SageMaker V3 Release

0 commit comments

Comments
 (0)