Skip to content

Commit dde808e

Browse files
aws-tianquawaws-tianquaw
committed
Merge branch 'main' into release-2.7.0
2 parents 238d491 + b5c4641 commit dde808e

280 files changed

Lines changed: 12657 additions & 135 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/build-image.yml‎

Lines changed: 31 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ jobs:
4242
- uses: actions/checkout@v4
4343
- uses: mamba-org/setup-micromamba@v1
4444
with:
45-
environment-file: ./environment.yml
45+
environment-file: ./environment.lock
4646
environment-name: sagemaker-distribution
4747
init-shell: bash
4848
- name: Free up disk space
@@ -54,30 +54,51 @@ jobs:
5454
run: |
5555
TARGET_VERSION=$(python -c 'import semver; print(semver.bump_${{ inputs.release-type }}("${{ inputs.base-version }}"))')
5656
echo "target_version=$TARGET_VERSION" >> $GITHUB_OUTPUT
57-
- name: Create new branch
57+
- name: Create or merge release branch
5858
# Note - CodeBuild depends on this branch name. Don't change without corresponding backend change.
59-
run: git checkout -b release-${{ steps.calc_target.outputs.target_version }}
59+
id: check_branch
60+
run: |
61+
git config --local user.email "github-actions[bot]@users.noreply.github.com"
62+
git config --local user.name "github-actions[bot]"
63+
if git ls-remote --exit-code --heads origin release-${{ steps.calc_target.outputs.target_version }}; then
64+
echo "Branch exists, merging latest changes in"
65+
git checkout release-${{ steps.calc_target.outputs.target_version }}
66+
git merge main
67+
git push
68+
else
69+
echo "Branch doesn't exist, creating now..."
70+
git checkout -b release-${{ steps.calc_target.outputs.target_version }}
71+
fi
6072
- name: Generate artifacts
6173
run: python ./src/main.py create-${{ inputs.release-type }}-version-artifacts --base-patch-version ${{ inputs.base-version }} --force
6274
- name: Commit .in artifacts to branch
6375
env:
6476
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
77+
# Generate the artifacts every time, only push if there are changes
6578
run: |
6679
git config --local user.email "github-actions[bot]@users.noreply.github.com"
6780
git config --local user.name "github-actions[bot]"
6881
git add ./build_artifacts
69-
git commit -m 'chore: Generate build artifacts for ${{ steps.calc_target.outputs.target_version }} release'
82+
git diff-index --quiet HEAD || git commit -m 'chore: Generate build artifacts for ${{ steps.calc_target.outputs.target_version }} release'
7083
git push --set-upstream origin release-${{ steps.calc_target.outputs.target_version }}
71-
- name: Open pull request
84+
- name: Create or fetch release PR
7285
id: get_pr_id
7386
env:
7487
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
75-
# Note - CodeBuild depends on this PR title. Don't change without corresponding backend change.
88+
# Create PR if it doesn't exist; fetch existing PR number if it does
7689
run: |
77-
URL=$(gh pr create -H release-${{ steps.calc_target.outputs.target_version }} \
78-
--title 'release: v${{ steps.calc_target.outputs.target_version }}' -F ./.github/workflows/PR_TEMPLATE.md)
79-
PR=$(echo $URL | sed 's:.*/::')
80-
echo "pr_id=$PR" >> $GITHUB_OUTPUT
90+
if gh pr list --head release-${{ steps.calc_target.outputs.target_version }} --state open --json number | grep -q '"number":'; then
91+
echo "PR exists already, just fetching ID"
92+
PR_ID=$(gh pr view release-${{ steps.calc_target.outputs.target_version }} --json number | jq -r .number)
93+
echo "pr_id=$PR_ID" >> $GITHUB_OUTPUT
94+
else
95+
echo "PR doesn't exist, creating now..."
96+
git push --set-upstream origin release-${{ steps.calc_target.outputs.target_version }}
97+
URL=$(gh pr create -H release-${{ steps.calc_target.outputs.target_version }} -B main \
98+
--title 'release: v${{ steps.calc_target.outputs.target_version }}' -F ./.github/workflows/PR_TEMPLATE.md)
99+
PR_ID=$(echo $URL | sed 's:.*/::')
100+
echo "pr_id=$PR_ID" >> $GITHUB_OUTPUT
101+
fi
81102
call-codebuild-project:
82103
runs-on: ubuntu-latest
83104
needs: open-pr

‎.github/workflows/check-image-size.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ jobs:
4141
- uses: actions/checkout@v4
4242
- uses: mamba-org/setup-micromamba@v1
4343
with:
44-
environment-file: ./environment.yml
44+
environment-file: ./environment.lock
4545
environment-name: sagemaker-distribution
4646
init-shell: bash
4747
- name: Free up disk space

‎.github/workflows/check_code_quality.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ jobs:
3131
- uses: actions/checkout@v4
3232
- uses: mamba-org/setup-micromamba@v1
3333
with:
34-
environment-file: ./environment.yml
34+
environment-file: ./environment.lock
3535
environment-name: sagemaker-distribution
3636
init-shell: bash
3737
- name: Free up disk space

.github/workflows/monthly-minor-release.yml renamed to .github/workflows/nightly-minor-build.yml

Lines changed: 23 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1-
name: Generate monthly minor release(s)
1+
name: Kickoff nightly Minor version builds
2+
run-name: Kickoff Minor nightly builds $(date +'%m-%d-%Y')
23
on:
3-
# Run first day of the month at 8AM PST / 9AM PDT
4-
#schedule:
5-
# - cron: '0 16 1 * *'
4+
# Run manually
5+
workflow_dispatch:
6+
# Run on 1st-10th day each month at 4PM PST / 5PM PDT
7+
schedule:
8+
- cron: '0 0 1-10 * *'
69
jobs:
710
generate-version-matrix:
811
name: Generate version matrix
@@ -12,14 +15,25 @@ jobs:
1215
matrix: ${{ steps.gen-mat.outputs.matrix }}
1316
steps:
1417
- uses: actions/checkout@v4
18+
- name: Get supported Major versions
19+
id: supported-major-versions
20+
# Check support_policy.md to determine active Major versions
21+
run: |
22+
major_versions=$(sed -n '/^### CPU Images/,/^### GPU Images/{
23+
/^### GPU Images/q
24+
/^| [0-9]/{
25+
s/^| \([0-9]\+\).*/\1/p
26+
}
27+
}' support_policy.md | sort -n | uniq | tr '\n' ',' | sed 's/,$//' | sed 's/,/","/g')
28+
echo "majors=[\"$major_versions\"]" >> $GITHUB_OUTPUT
1529
- name: Generate base patch version matrix
1630
id: gen-mat
17-
# Output looks like: matrix={"version":["0.11.0","1.4.1"]}
18-
# For each major, get highest minor version. Then get highest patch
19-
# for that minor. Use this patch as base version.
31+
# Output looks like: matrix={"version":["1.13.1","2.5.0","3.0.0"]}
32+
# For each active Major version, get highest Minor version. Then get highest Patch
33+
# for that Minor. Use this Patch as base version.
2034
run: |
2135
versions=("{\"version\":[")
22-
for major in build_artifacts/v*; do
36+
for major in build_artifacts/v${{ steps.supported-major-versions.outputs.majors }}; do
2337
highest_minor=$(ls $major | sort -t. -k2n | tail -n1)
2438
highest_patch=$(ls $major/$highest_minor | sort -t. -k2n | tail -n1)
2539
versions+="\"${highest_patch#v}\""
@@ -28,7 +42,7 @@ jobs:
2842
versions=${versions::-1}
2943
versions+="]}"
3044
echo "matrix=$versions" >> $GITHUB_OUTPUT
31-
start-monthly-minor:
45+
start-minor-build:
3246
name: Start monthly minor release
3347
needs: generate-version-matrix
3448
permissions:
@@ -43,15 +57,3 @@ jobs:
4357
with:
4458
release-type: "minor"
4559
base-version: ${{ matrix.version }}
46-
check-image-size:
47-
name: Check Image Size
48-
needs: start-monthly-minor
49-
permissions:
50-
pull-requests: write
51-
contents: write
52-
strategy:
53-
matrix: ${{ fromJson(needs.generate-version-matrix.outputs.matrix) }}
54-
fail-fast: false
55-
uses: aws/sagemaker-distribution/.github/workflows/check-image-size.yml@main
56-
with:
57-
base-version: ${{ matrix.version }}

‎.github/workflows/update-environment-lock.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: Update environment.lock
2+
# Update environment.lock weekly, so we can quickly install environment without having
3+
# to solve environment.yml every time.
4+
on:
5+
# Every Monday
6+
schedule:
7+
- cron: '0 0 * * MON'
8+
# Manually call
9+
workflow_dispatch:
10+
defaults:
11+
run:
12+
shell: bash -l {0}
13+
jobs:
14+
update-environment:
15+
name: Install environment.yml, generate environment.lock
16+
runs-on: ubuntu-latest
17+
permissions:
18+
contents: write
19+
steps:
20+
- uses: actions/checkout@v4
21+
- uses: mamba-org/setup-micromamba@v1
22+
with:
23+
environment-file: ./environment.yml
24+
environment-name: sagemaker-distribution
25+
init-shell: bash
26+
- name: Create environment-lock.yml
27+
run: micromamba env export --explicit > environment.lock
28+
- name: Run sagemaker-distribution unit tests to check for regressions
29+
run: pytest -m unit
30+
- name: Commit changes
31+
run: |
32+
git config --local user.email "github-actions[bot]@users.noreply.github.com"
33+
git config --local user.name "github-actions[bot]"
34+
git add environment.lock
35+
git commit -m 'Update environment.lock'
36+
git push

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ Follow these steps for sending out a pull request for adding new packages:
6161
```
6262
This project uses Conda to manage its dependencies. Run the following to setup your local environment:
6363
64-
conda env update --file environment.yml -n sagemaker-distribution
64+
conda env update --file environment.lock -n sagemaker-distribution
6565
6666
conda activate sagemaker-distribution
6767

‎DEVELOPMENT.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
This project uses Conda to manage its dependencies. Run the following to setup your local environment:
44

55
```shell
6-
conda env update --file environment.yml -n sagemaker-distribution
6+
conda env update --file environment.lock -n sagemaker-distribution
77
```
88

99
## Tests

‎build_artifacts/v2/v2.4/v2.4.3/CHANGELOG-cpu.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Change log: 2.4.3(cpu)
2+
3+
## Upgrades:
4+
5+
Package | Previous Version | Current Version
6+
---|---|---
7+
amazon-q-developer-jupyterlab-ext|3.4.7|3.4.8
8+
langchain|0.3.23|0.3.24
9+
uvicorn|0.34.0|0.34.2
10+
sagemaker-code-editor|1.4.2|1.4.3
11+
sagemaker-studio-analytics-extension|0.1.6|0.1.7

‎build_artifacts/v2/v2.4/v2.4.3/CHANGELOG-gpu.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Change log: 2.4.3(gpu)
2+
3+
## Upgrades:
4+
5+
Package | Previous Version | Current Version
6+
---|---|---
7+
amazon-q-developer-jupyterlab-ext|3.4.7|3.4.8
8+
langchain|0.3.23|0.3.24
9+
uvicorn|0.34.0|0.34.2
10+
sagemaker-code-editor|1.4.2|1.4.3
11+
sagemaker-studio-analytics-extension|0.1.6|0.1.7

0 commit comments

Comments
 (0)