diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index aac13021866..183d215865b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,7 +11,15 @@ Thank you for your interest in contributing to CUDA Python! Based on the type of 2. You want to implement a feature, improvement, or bug fix: - Please refer to each component's guideline: - [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/contribute.html) - - [`cuda.bindings`](https://nvidia.github.io/cuda-python/cuda-bindings/latest/contribute.html) + - [`cuda.bindings`](https://nvidia.github.io/cuda-python/cuda-bindings/latest/contribute.html)[1](#footnote1) + - [`cuda.pathfinder`](https://nvidia.github.io/cuda-python/cuda-pathfinder/latest/contribute.html) + +## Table of Contents + +- [Pre-commit](#pre-commit) +- [Code signing](#code-signing) +- [Developer Certificate of Origin (DCO)](#developer-certificate-of-origin-dco) +- [CI infrastructure overview](#ci-infrastructure-overview) ## Pre-commit @@ -78,3 +86,127 @@ By making a contribution to this project, I certify that: maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ``` + +## CI infrastructure overview + +The CUDA Python project uses a comprehensive CI pipeline that builds, tests, and releases multiple components across different platforms. This section provides a visual overview of our CI infrastructure to help contributors understand the build and release process. + +### CI Pipeline Flow + +![CUDA Python CI Pipeline Flow](ci/ci-pipeline.svg) + +Alternative Mermaid diagram representation: + +```mermaid +flowchart TD + %% Trigger Events + subgraph TRIGGER["🔄 TRIGGER EVENTS"] + T1["• Push to main branch"] + T2["• Pull request
• Manual workflow dispatch"] + T1 --- T2 + end + + %% Build Stage + subgraph BUILD["🔨 BUILD STAGE"] + subgraph BUILD_PLATFORMS["Parallel Platform Builds"] + B1["linux-64
(Self-hosted)"] + B2["linux-aarch64
(Self-hosted)"] + B3["win-64
(GitHub-hosted)"] + end + BUILD_DETAILS["• Python versions: 3.9, 3.10, 3.11, 3.12, 3.13
• CUDA version: 13.0.0 (build-time)
• Components: cuda-core, cuda-bindings,
cuda-pathfinder, cuda-python"] + end + + %% Artifact Storage + subgraph ARTIFACTS["📦 ARTIFACT STORAGE"] + subgraph GITHUB_ARTIFACTS["GitHub Artifacts"] + GA1["• Wheel files (.whl)
• Test artifacts
• Documentation
(30-day retention)"] + end + subgraph GITHUB_CACHE["GitHub Cache"] + GC1["• Mini CTK cache"] + end + end + + %% Test Stage + subgraph TEST["🧪 TEST STAGE"] + subgraph TEST_PLATFORMS["Parallel Platform Tests"] + TS1["linux-64
(Self-hosted)"] + TS2["linux-aarch64
(Self-hosted)"] + TS3["win-64
(GitHub-hosted)"] + end + TEST_DETAILS["• Download wheels from artifacts
• Test against multiple CUDA runtime versions
• Run Python unit tests, Cython tests, examples"] + ARTIFACT_FLOWS["Artifact Flows:
• cuda-pathfinder: main → backport
• cuda-bindings: backport → main"] + end + + %% Release Pipeline + subgraph RELEASE["🚀 RELEASE PIPELINE"] + subgraph RELEASE_STAGES["Sequential Release Steps"] + R1["Validation
• Artifact integrity
• Git tag verification"] + R2["Publishing
• PyPI/TestPyPI
• Component or all releases"] + R3["Documentation
• GitHub Pages
• Release notes"] + R1 --> R2 --> R3 + end + RELEASE_DETAILS["• Manual workflow dispatch with run ID
• Supports individual component or full releases"] + end + + %% Main Flow + TRIGGER --> BUILD + BUILD -.->|"wheel upload"| ARTIFACTS + ARTIFACTS -.-> TEST + TEST --> RELEASE + + %% Artifact Flow Arrows (Cache Reuse) + GITHUB_CACHE -.->|"mini CTK reuse"| BUILD + GITHUB_CACHE -.->|"mini CTK reuse"| TEST + + %% Artifact Flow Arrows (Wheel Fetch) + GITHUB_ARTIFACTS -.->|"wheel fetch"| TEST + GITHUB_ARTIFACTS -.->|"wheel fetch"| RELEASE + + %% Styling + classDef triggerStyle fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#1976D2 + classDef buildStyle fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#7B1FA2 + classDef artifactStyle fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#F57C00 + classDef testStyle fill:#e8f5e8,stroke:#4CAF50,stroke-width:2px,color:#388E3C + classDef releaseStyle fill:#ffebee,stroke:#f44336,stroke-width:2px,color:#D32F2F + + class TRIGGER,T1,T2 triggerStyle + class BUILD,BUILD_PLATFORMS,B1,B2,B3,BUILD_DETAILS buildStyle + class ARTIFACTS,GITHUB_ARTIFACTS,GITHUB_CACHE,GA1,GC1 artifactStyle + class TEST,TEST_PLATFORMS,TS1,TS2,TS3,TEST_DETAILS,ARTIFACT_FLOWS testStyle + class RELEASE,RELEASE_STAGES,R1,R2,R3,RELEASE_DETAILS releaseStyle +``` + +### Pipeline Execution Details + +**Parallel Execution**: The CI pipeline leverages parallel execution to optimize build and test times: +- **Build Stage**: Different architectures/operating systems (linux-64, linux-aarch64, win-64) are built in parallel across their respective runners +- **Test Stage**: Different architectures/operating systems/CUDA versions are tested in parallel; documentation preview is also built in parallel with testing + +### Branch-specific Artifact Flow + +#### Main Branch +- **Build** → **Test** → **Documentation** → **Potential Release** +- Artifacts stored as `{component}-python{version}-{platform}-{sha}` +- Full test coverage across all platforms and CUDA versions +- **Artifact flow out**: `cuda-pathfinder` artifacts → backport branches + +#### Backport Branches +- **Build** → **Test** → **Backport PR Creation** +- Artifacts used for validation before creating backport pull requests +- Maintains compatibility with older CUDA versions +- **Artifact flow in**: `cuda-pathfinder` artifacts ← main branch +- **Artifact flow out**: older `cuda-bindings` artifacts → main branch + +### Key Infrastructure Details + +- **Self-hosted runners**: Used for Linux builds and GPU testing (more resources, faster builds) +- **GitHub-hosted runners**: Used for Windows builds and general tasks +- **Artifact retention**: 30 days for GitHub Artifacts (wheels, docs, tests) +- **Cache retention**: GitHub Cache for build dependencies and environments +- **Security**: All commits must be signed, untrusted code blocked +- **Parallel execution**: Matrix builds across Python versions and platforms +- **Component isolation**: Each component (core, bindings, pathfinder, python) can be built/released independently + +--- + +1: The `cuda-python` meta package shares the same license and the contributing guidelines as those of `cuda-bindings`. diff --git a/ci/.ci-pipeline-regen.md b/ci/.ci-pipeline-regen.md new file mode 100644 index 00000000000..7ddf9b970d5 --- /dev/null +++ b/ci/.ci-pipeline-regen.md @@ -0,0 +1,106 @@ +# CUDA Python CI Pipeline SVG Regeneration Instructions + +This file contains the prompt and requirements for regenerating `ci-pipeline.svg` with the same styling and content. + +## Styling Requirements + +- Hand-drawn Excalidraw-style design with rough, sketchy borders +- Comic Sans MS font family for all text +- Imperfect lines and curves that mimic hand-drawn aesthetics +- Canvas size: 900x800 pixels +- Color scheme: + - Trigger Events: #e8f4fd background, #2196F3 border, #1976D2 text + - Build Stage: #f3e5f5 background, #9C27B0 border, #7B1FA2 text + - Artifact Storage: #fff3e0 background, #FF9800 border, #F57C00 text + - Test Stage: #e8f5e8 background, #4CAF50 border, #388E3C text + - Release Pipeline: #ffebee background, #f44336 border, #D32F2F text + +## Content Structure + +1. **Title**: "CUDA Python CI Pipeline Flow" + +2. **Trigger Events** (top blue box): + - Push to main branch + - Pull request + - Manual workflow dispatch + +3. **Build Stage** (purple box): + - Three platform boxes: linux-64 (Self-hosted), linux-aarch64 (Self-hosted), win-64 (GitHub-hosted) + - Details: Python versions 3.9-3.13, CUDA 13.0.0 (build-time) + - Components: cuda-core, cuda-bindings, cuda-pathfinder, cuda-python + +4. **Artifact Storage** (orange box): + - GitHub Artifacts box: Wheel files (.whl), Test artifacts, Documentation (30-day retention) + - GitHub Cache box: Mini CTK cache + +5. **Test Stage** (green box): + - Three platform boxes: linux-64 (Self-hosted), linux-aarch64 (Self-hosted), win-64 (GitHub-hosted) + - Details: Download wheels from artifacts, Test against multiple CUDA runtime versions, Run Python unit tests, Cython tests, examples + - Artifact Flows (in red text): + • cuda-pathfinder: main → backport + • cuda-bindings: backport → main + +6. **Release Pipeline** (red box): + - Three sequential boxes: Validation → Publishing → Documentation + - Validation: Artifact integrity, Git tag verification + - Publishing: PyPI/TestPyPI, Component or all releases + - Documentation: GitHub Pages, Release notes + - Details: Manual workflow dispatch with run ID, Supports individual component or full releases + +## Arrow Requirements + +- Main flow arrows: Trigger → Build → Artifact → Test → Release +- Additional artifact flow arrows (dashed, orange #FF9800): + - From GitHub Cache (mini CTK) back to Build Stage with "mini CTK reuse" label + - From GitHub Artifacts (wheels) to Release Pipeline with "wheel fetch" label + - **NEW**: From GitHub Cache (mini CTK) to Test Stage with "mini CTK reuse" label + - **NEW**: From GitHub Artifacts (wheels) to Test Stage with "wheel fetch" label +- Arrow marker definition with hand-drawn style (orange arrow heads, not black) +- Use stroke-dasharray="5,3" for artifact flow arrows + +## Critical Arrow Positioning Requirements (UPDATED) + +**IMPORTANT**: Arrows must NOT overlap with stage boxes. Ensure proper clearance: + +1. **Mini CTK reuse arrow** (GitHub Cache → Build Stage): + - Arrow endpoint Y coordinate must be BELOW the Build Stage box edge (y=292) + - Use y=295 or greater for the endpoint to ensure no overlap + - Position "mini CTK reuse" text to the RIGHT of the arrow (not left) for less visual clutter + - Text color should be orange (#FF9800) to match arrow + +2. **Wheel fetch arrow** (GitHub Artifacts → Release Pipeline): + - Arrow endpoint Y coordinate must be ABOVE the Release Pipeline box edge (y=652) + - Use y=645 or smaller for the endpoint to provide proper margin + - Position "wheel fetch" text between Test Stage and Release Pipeline boxes + - Text should be to the LEFT of the arrow for better spacing + +## Font Size Requirements (UPDATED) + +- ALL text labels must use consistent 12pt font size for readability +- No 9pt text - this is too small and hard to read +- Title: 16pt, Stage headers: 14pt, All other text: 12pt + +## Key Features + +- All boxes use rough, hand-drawn paths (not perfect rectangles) +- Text should be properly sized and positioned within boxes +- Platform boxes within each stage should be clearly separated +- Maintain consistent spacing and alignment +- Orange arrow heads must match the orange arrow color + +## Text Positioning + +- Use text-anchor="middle" for centered headers +- Use text-anchor="start" for left-aligned bullet points +- Ensure all text fits within their enclosing boxes +- Use transforms for angled text labels on artifact flow arrows +- Artifact flow arrow text positioning is critical - follow positioning requirements above + +## Recent Manual Adjustments Applied + +- Fixed arrow endpoint positioning to prevent overlap with stage boxes +- Moved mini CTK reuse arrow endpoint from y=285 to y=295 +- Moved wheel fetch arrow endpoint from y=650 to y=645 +- Repositioned text labels for better visual separation +- Standardized all text to 12pt font size for consistency +- Changed arrow heads from black to orange to match arrow color diff --git a/ci/ci-pipeline.svg b/ci/ci-pipeline.svg new file mode 100644 index 00000000000..eeff4c69fd1 --- /dev/null +++ b/ci/ci-pipeline.svg @@ -0,0 +1,172 @@ + + + + + + + + + + + + + + + + + + CUDA Python CI Pipeline Flow + + + + TRIGGER EVENTS + • Push to main branch + • Pull request + • Manual workflow dispatch + + + + + + BUILD STAGE + + + + linux-64 + Self-hosted + + + linux-aarch64 + Self-hosted + + + win-64 + GitHub-hosted + + + • Python versions: 3.9, 3.10, 3.11, 3.12, 3.13 + • CUDA version: 13.0.0 (build-time) + • Components: cuda-core, cuda-bindings, + cuda-pathfinder, cuda-python + + + + wheel upload + + + + ARTIFACT STORAGE + + + + GitHub Artifacts + • Wheel files (.whl) + • Test artifacts + • Documentation + (30-day retention) + + + GitHub Cache + • Mini CTK cache + + + + + + + + mini CTK reuse + + + + wheel fetch + + + + + mini CTK reuse + + + + wheel fetch + + + + TEST STAGE + + + + linux-64 + Self-hosted + + + linux-aarch64 + Self-hosted + + + win-64 + GitHub-hosted + + + • Download wheels from artifacts + • Test against multiple CUDA runtime versions + • Run Python unit tests, Cython tests, examples + Artifact Flows: + • cuda-pathfinder: main → backport + • cuda-bindings: backport → main + + + + + + + RELEASE PIPELINE + + + + Validation + • Artifact integrity + • Git tag verification + + + + + + Publishing + • PyPI/TestPyPI + • Component or all releases + + + + + + Documentation + • GitHub Pages + • Release notes + + + • Manual workflow dispatch with run ID + • Supports individual component or full releases +