Skip to content

Commit 721fb62

Browse files
[Backport] #1606 & #873 (#1750)
* ci: require tag-triggered artifacts for release uploads (#1606) Backport of #1606 to 12.9.x. Adapted for the 12.9.x branch workflow structure: - ci.yml: add tag push triggers (v*, cuda-core-v*, cuda-pathfinder-v*) so setuptools-scm resolves exact release versions from tag refs - release.yml: make run-id optional with auto-detection from tag-triggered CI runs via ci/tools/lookup-run-id; add wheel version validation via ci/tools/validate-release-wheels before publishing - Add ci/tools/lookup-run-id: finds the successful tag-triggered CI run for a given release tag - Add ci/tools/validate-release-wheels: rejects dev/local wheel versions and enforces version match against the release tag - release_checklist.yml: add reminder to wait for tag-triggered CI * ci: sync release-upload.yml with main and add download-wheels script Sync release-upload.yml to match main by adding run-id and component inputs and a 'Download and Upload Wheels' step that downloads wheels via ci/tools/download-wheels, validates them via validate-release-wheels, and uploads them to the GitHub Release. Also add the ci/tools/download-wheels helper script (from main) and wire up release.yml to pass run-id and component to the upload-archive job. Fixes #1120: the download-wheels script now exists on the backport branch, so the release-upload workflow no longer fails with 'No such file or directory'. * [pre-commit.ci] auto code formatting --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 1e0ae86 commit 721fb62

File tree

7 files changed

+383
-4
lines changed

7 files changed

+383
-4
lines changed

.github/ISSUE_TEMPLATE/release_checklist.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ body:
2626
- label: "Finalize the doc update, including release notes (\"Note: Touching docstrings/type annotations in code is OK during code freeze, apply your best judgement!\")"
2727
- label: Update the docs for the new version
2828
- label: Create a public release tag
29+
- label: Wait for the tag-triggered CI run to complete, and use that run ID for release workflows
2930
- label: If any code change happens, rebuild the wheels from the new tag
3031
- label: Update the conda recipe & release conda packages
3132
- label: Upload conda packages to nvidia channel

.github/workflows/ci.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,12 @@ on:
1515
branches:
1616
- "pull-request/[0-9]+"
1717
- "12.9.x"
18+
tags:
19+
# Build release artifacts from tag refs so setuptools-scm resolves exact
20+
# release versions instead of .dev+local variants.
21+
- "v*"
22+
- "cuda-core-v*"
23+
- "cuda-pathfinder-v*"
1824

1925
jobs:
2026
ci-vars:

.github/workflows/release-upload.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,14 @@ on:
1010
git-tag:
1111
type: string
1212
required: true
13+
run-id:
14+
description: "The GHA run ID that generated validated artifacts"
15+
type: string
16+
required: true
17+
component:
18+
description: "Component to download wheels for"
19+
type: string
20+
required: true
1321

1422
concurrency:
1523
# Concurrency group that uses the workflow name and PR number if available
@@ -63,3 +71,19 @@ jobs:
6371
--clobber "${{ inputs.git-tag }}"
6472
--repo "${{ github.repository }}"
6573
release/*
74+
75+
- name: Download and Upload Wheels
76+
env:
77+
GH_TOKEN: ${{ github.token }}
78+
run: |
79+
# Use the shared script to download wheels
80+
./ci/tools/download-wheels "${{ inputs.run-id }}" "${{ inputs.component }}" "${{ github.repository }}" "release/wheels"
81+
82+
# Validate that release wheels match the expected version from tag.
83+
./ci/tools/validate-release-wheels "${{ inputs.git-tag }}" "${{ inputs.component }}" "release/wheels"
84+
85+
# Upload wheels to the release
86+
if [[ -d "release/wheels" && $(ls -A release/wheels 2>/dev/null | wc -l) -gt 0 ]]; then
87+
echo "Uploading wheels to release ${{ inputs.git-tag }}"
88+
gh release upload --clobber "${{ inputs.git-tag }}" --repo "${{ github.repository }}" release/wheels/*
89+
fi

.github/workflows/release.yml

Lines changed: 42 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,10 @@ on:
2424
required: true
2525
type: string
2626
run-id:
27-
description: "The GHA run ID that generated validated artifacts"
28-
required: true
27+
description: "The GHA run ID that generated validated artifacts (optional - auto-detects successful tag-triggered CI run for git-tag)"
28+
required: false
2929
type: string
30+
default: ""
3031
build-ctk-ver:
3132
type: string
3233
required: true
@@ -43,6 +44,31 @@ defaults:
4344
shell: bash --noprofile --norc -xeuo pipefail {0}
4445

4546
jobs:
47+
determine-run-id:
48+
runs-on: ubuntu-latest
49+
outputs:
50+
run-id: ${{ steps.lookup-run-id.outputs.run-id }}
51+
steps:
52+
- name: Checkout Source
53+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
54+
with:
55+
fetch-depth: 0
56+
57+
- name: Determine Run ID
58+
id: lookup-run-id
59+
env:
60+
GH_TOKEN: ${{ github.token }}
61+
run: |
62+
if [[ -n "${{ inputs.run-id }}" ]]; then
63+
echo "Using provided run ID: ${{ inputs.run-id }}"
64+
echo "run-id=${{ inputs.run-id }}" >> $GITHUB_OUTPUT
65+
else
66+
echo "Auto-detecting successful tag-triggered run ID for tag: ${{ inputs.git-tag }}"
67+
RUN_ID=$(./ci/tools/lookup-run-id "${{ inputs.git-tag }}" "${{ github.repository }}")
68+
echo "Auto-detected run ID: $RUN_ID"
69+
echo "run-id=$RUN_ID" >> $GITHUB_OUTPUT
70+
fi
71+
4672
check-tag:
4773
runs-on: ubuntu-latest
4874
steps:
@@ -86,13 +112,14 @@ jobs:
86112
pull-requests: write
87113
needs:
88114
- check-tag
115+
- determine-run-id
89116
secrets: inherit
90117
uses: ./.github/workflows/build-docs.yml
91118
with:
92119
build-ctk-ver: ${{ inputs.build-ctk-ver }}
93120
component: ${{ inputs.component }}
94121
git-tag: ${{ inputs.git-tag }}
95-
run-id: ${{ inputs.run-id }}
122+
run-id: ${{ needs.determine-run-id.outputs.run-id }}
96123
is-release: true
97124

98125
upload-archive:
@@ -101,27 +128,34 @@ jobs:
101128
contents: write
102129
needs:
103130
- check-tag
131+
- determine-run-id
104132
secrets: inherit
105133
uses: ./.github/workflows/release-upload.yml
106134
with:
107135
git-tag: ${{ inputs.git-tag }}
136+
run-id: ${{ needs.determine-run-id.outputs.run-id }}
137+
component: ${{ inputs.component }}
108138

109139
publish-wheels:
110140
name: Publish wheels
111141
runs-on: ubuntu-latest
112142
needs:
113143
- check-tag
144+
- determine-run-id
114145
environment:
115146
name: ${{ inputs.wheel-dst }}
116147
url: https://${{ (inputs.wheel-dst == 'testpypi' && 'test.') || '' }}pypi.org/p/${{ inputs.component }}/
117148
permissions:
118149
id-token: write
119150
steps:
151+
- name: Checkout Source
152+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
153+
120154
- name: Download component wheels
121155
env:
122156
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
123157
run: |
124-
gh run download ${{ inputs.run-id }} -p "${{ inputs.component }}*" -R ${{ github.repository }}
158+
gh run download ${{ needs.determine-run-id.outputs.run-id }} -p "${{ inputs.component }}*" -R ${{ github.repository }}
125159
mkdir dist
126160
for p in ${{ inputs.component }}*
127161
do
@@ -133,6 +167,10 @@ jobs:
133167
done
134168
rm -rf ${{ inputs.component }}*
135169
170+
- name: Validate wheel versions for release tag
171+
run: |
172+
./ci/tools/validate-release-wheels "${{ inputs.git-tag }}" "${{ inputs.component }}" "dist"
173+
136174
- name: Publish package distributions to PyPI
137175
if: ${{ inputs.wheel-dst == 'pypi' }}
138176
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0

ci/tools/download-wheels

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
#
5+
# SPDX-License-Identifier: Apache-2.0
6+
7+
# A utility script to download component wheels from GitHub Actions artifacts.
8+
# This script reuses the same logic that was in release.yml to maintain consistency.
9+
10+
set -euo pipefail
11+
12+
# Check required arguments
13+
if [[ $# -lt 3 ]]; then
14+
echo "Usage: $0 <run-id> <component> <repository> [output-dir]" >&2
15+
echo " run-id: The GitHub Actions run ID containing the artifacts" >&2
16+
echo " component: The component name pattern to download (e.g., cuda-core, cuda-bindings)" >&2
17+
echo " repository: The GitHub repository (e.g., NVIDIA/cuda-python)" >&2
18+
echo " output-dir: Optional output directory (default: ./dist)" >&2
19+
exit 1
20+
fi
21+
22+
RUN_ID="$1"
23+
COMPONENT="$2"
24+
REPOSITORY="$3"
25+
OUTPUT_DIR="${4:-./dist}"
26+
27+
# Ensure we have a GitHub token
28+
if [[ -z "${GH_TOKEN:-}" ]]; then
29+
echo "Error: GH_TOKEN environment variable is required"
30+
exit 1
31+
fi
32+
33+
echo "Downloading wheels for component: $COMPONENT from run: $RUN_ID"
34+
35+
# Download component wheels using the same logic as release.yml
36+
if [[ "$COMPONENT" == "all" ]]; then
37+
# Download all component patterns
38+
gh run download "$RUN_ID" -p "cuda-*" -R "$REPOSITORY"
39+
else
40+
gh run download "$RUN_ID" -p "${COMPONENT}*" -R "$REPOSITORY"
41+
fi
42+
43+
# Create output directory
44+
mkdir -p "$OUTPUT_DIR"
45+
46+
# Process downloaded artifacts
47+
for p in cuda-*
48+
do
49+
if [[ ! -d "$p" ]]; then
50+
continue
51+
fi
52+
53+
# exclude cython test artifacts
54+
if [[ "${p}" == *-tests ]]; then
55+
echo "Skipping test artifact: $p"
56+
continue
57+
fi
58+
59+
# If we're not downloading "all", only process matching component
60+
if [[ "$COMPONENT" != "all" && "$p" != ${COMPONENT}* ]]; then
61+
continue
62+
fi
63+
64+
echo "Processing artifact: $p"
65+
# Move wheel files to output directory
66+
if [[ -d "$p" ]]; then
67+
find "$p" -name "*.whl" -exec mv {} "$OUTPUT_DIR/" \;
68+
fi
69+
done
70+
71+
# Clean up artifact directories
72+
rm -rf cuda-*
73+
74+
echo "Downloaded wheels to: $OUTPUT_DIR"
75+
ls -la "$OUTPUT_DIR"

ci/tools/lookup-run-id

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
#
5+
# SPDX-License-Identifier: Apache-2.0
6+
7+
# A utility script to find the GitHub Actions workflow run ID for a given git tag.
8+
# This script requires a successful CI run that was triggered by the tag push.
9+
10+
set -euo pipefail
11+
12+
# Check required arguments
13+
if [[ $# -lt 2 ]]; then
14+
echo "Usage: $0 <git-tag> <repository> [workflow-name]" >&2
15+
echo " git-tag: The git tag to find the corresponding workflow run for" >&2
16+
echo " repository: The GitHub repository (e.g., NVIDIA/cuda-python)" >&2
17+
echo " workflow-name: Optional workflow name to filter by (default: CI)" >&2
18+
echo "" >&2
19+
echo "Examples:" >&2
20+
echo " $0 v13.0.1 NVIDIA/cuda-python" >&2
21+
echo " $0 v13.0.1 NVIDIA/cuda-python \"CI\"" >&2
22+
exit 1
23+
fi
24+
25+
GIT_TAG="${1}"
26+
REPOSITORY="${2}"
27+
WORKFLOW_NAME="${3:-CI}"
28+
29+
# Ensure we have required tools
30+
if [[ -z "${GH_TOKEN:-}" ]]; then
31+
echo "Error: GH_TOKEN environment variable is required" >&2
32+
exit 1
33+
fi
34+
35+
if ! command -v jq >/dev/null 2>&1; then
36+
echo "Error: jq is required but not installed" >&2
37+
exit 1
38+
fi
39+
40+
if ! command -v gh >/dev/null 2>&1; then
41+
echo "Error: GitHub CLI (gh) is required but not installed" >&2
42+
exit 1
43+
fi
44+
45+
echo "Looking up run ID for tag: ${GIT_TAG} in repository: ${REPOSITORY}" >&2
46+
47+
# Resolve git tag to commit SHA
48+
if ! COMMIT_SHA=$(git rev-parse "${GIT_TAG}"); then
49+
echo "Error: Could not resolve git tag '${GIT_TAG}' to a commit SHA" >&2
50+
echo "Make sure the tag exists and you have fetched it" >&2
51+
exit 1
52+
fi
53+
54+
echo "Resolved tag '${GIT_TAG}' to commit: ${COMMIT_SHA}" >&2
55+
56+
# Find workflow runs for this commit
57+
echo "Searching for '${WORKFLOW_NAME}' workflow runs for commit: ${COMMIT_SHA} (tag: ${GIT_TAG})" >&2
58+
59+
# Get completed workflow runs for this commit.
60+
RUN_DATA=$(gh run list \
61+
--repo "${REPOSITORY}" \
62+
--commit "${COMMIT_SHA}" \
63+
--workflow "${WORKFLOW_NAME}" \
64+
--status completed \
65+
--json databaseId,workflowName,status,conclusion,headSha,headBranch,event,createdAt,url \
66+
--limit 50)
67+
68+
if [[ -z "${RUN_DATA}" || "${RUN_DATA}" == "[]" ]]; then
69+
echo "Error: No completed '${WORKFLOW_NAME}' workflow runs found for commit ${COMMIT_SHA}" >&2
70+
echo "Available workflow runs for this commit:" >&2
71+
gh run list --repo "${REPOSITORY}" --commit "${COMMIT_SHA}" --limit 10 || true
72+
exit 1
73+
fi
74+
75+
# Filter for successful push runs from the tag ref.
76+
RUN_ID=$(echo "${RUN_DATA}" | jq -r --arg tag "${GIT_TAG}" '
77+
map(select(.conclusion == "success" and .event == "push" and .headBranch == $tag))
78+
| sort_by(.createdAt)
79+
| reverse
80+
| .[0].databaseId // empty
81+
')
82+
83+
if [[ -z "${RUN_ID}" ]]; then
84+
echo "Error: No successful '${WORKFLOW_NAME}' workflow runs found for tag '${GIT_TAG}'." >&2
85+
echo "This release workflow now requires artifacts from a tag-triggered CI run." >&2
86+
echo "If you just pushed the tag, wait for CI on that tag to finish and retry." >&2
87+
echo "" >&2
88+
echo "Completed runs for commit ${COMMIT_SHA}:" >&2
89+
echo "${RUN_DATA}" | jq -r '.[] | "\(.databaseId): event=\(.event // "null"), headBranch=\(.headBranch // "null"), conclusion=\(.conclusion // "null"), status=\(.status // "null"), createdAt=\(.createdAt // "null")"' >&2
90+
exit 1
91+
fi
92+
93+
echo "Found workflow run ID: ${RUN_ID} for tag '${GIT_TAG}'" >&2
94+
95+
# Verify the run has the expected artifacts by checking if there are any artifacts
96+
echo "Verifying artifacts exist for run ${RUN_ID}..." >&2
97+
ARTIFACT_LIST=$(gh run view "${RUN_ID}" --repo "${REPOSITORY}" --json url || echo "")
98+
99+
if [[ -z "${ARTIFACT_LIST}" ]]; then
100+
echo "Warning: Could not verify artifacts for workflow run ${RUN_ID}" >&2
101+
fi
102+
103+
# Output the run ID (this is what gets used by calling scripts)
104+
echo "${RUN_ID}"

0 commit comments

Comments
 (0)