Skip to content

Commit 18e024d

Browse files
RyAuldRyAuld
committed
Add ADO publish pipeline and setup guide
1 parent 2de45ae commit 18e024d

File tree

5 files changed

+544
-0
lines changed

5 files changed

+544
-0
lines changed

‎.Pipelines/ADO-PUBLISH-SETUP.md‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# ADO Pipeline Setup Guide — MSAL Python → PyPI
2+
3+
This document describes every step needed to create an Azure DevOps (ADO)
4+
pipeline that checks out the GitHub repo, runs tests, builds distributions,
5+
and publishes to test.pypi.org (via the MSAL-Python environment) and PyPI.
6+
7+
The `.Pipelines/` folder follows the same template convention as [MSAL.NET](https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/tree/main/build):
8+
9+
| File | Purpose |
10+
|------|---------|
11+
| [`pipeline-publish.yml`](pipeline-publish.yml) | Top-level orchestrator — triggers, variables, stage wiring |
12+
| [`template-run-tests.yml`](template-run-tests.yml) | Reusable step template — pytest across Python version matrix |
13+
| [`template-build-package.yml`](template-build-package.yml) | Reusable step template — `python -m build` + `twine check` + artifact publish |
14+
| [`template-publish-package.yml`](template-publish-package.yml) | Reusable step template — `TwineAuthenticate` + `twine upload` (parameterized for MSAL-Python/PyPI) |
15+
16+
---
17+
18+
## Overview
19+
20+
This pipeline is **manually triggered only** — no automatic branch or tag triggers.
21+
Every publish requires explicitly entering a version and selecting a destination.
22+
23+
| Stage | Trigger | Target |
24+
|-------|---------|--------|
25+
| **Validate** | always | asserts `packageVersion` matches `msal/sku.py` |
26+
| **CI** (tests on Py 3.9–3.13) | after Validate | — |
27+
| **Build** (sdist + wheel) | after CI | dist artifact |
28+
| **PublishMSALPython** | `publishTarget = MSAL-Python` | test.pypi.org |
29+
| **PublishPyPI** | `publishTarget = pypi` | PyPI (production) |
30+
31+
---
32+
33+
## Step 1 — Prerequisites
34+
35+
| Requirement | Notes |
36+
|-------------|-------|
37+
| ADO Organization | [Create one](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/create-organization) if you don't have one |
38+
| ADO Project | Under the org; enable **Pipelines** and **Artifacts** |
39+
| GitHub account with admin rights | Needed to authorize the ADO GitHub App |
40+
| PyPI API token | Scoped to the `msal` project — generate at <https://pypi.org/manage/account/token/> |
41+
| MSAL-Python (test.pypi.org) API token | Scoped to the `msal` project on test.pypi.org |
42+
43+
---
44+
45+
## Step 2 — Connect ADO to the GitHub Repository
46+
47+
1. In your ADO project go to **Project Settings → Service connections → New service connection**.
48+
2. Choose **GitHub** and click **Next**.
49+
3. Select **GitHub App** (recommended) or **Personal Access Token**.
50+
- GitHub App: follow the OAuth flow to install the ADO GitHub App on the
51+
`AzureAD` organization and grant repository access to
52+
`microsoft-authentication-library-for-python`.
53+
- PAT: create a GitHub PAT with `repo` scope and paste it here.
54+
4. Set **Service connection name**: `github-msal-python`
55+
5. Check **Grant access permission to all pipelines**, click **Save**.
56+
57+
---
58+
59+
## Step 3 — Create PyPI Service Connections (Twine)
60+
61+
The `TwineAuthenticate@1` task uses "Python package upload" service connections
62+
for external registries.
63+
64+
### 3a — MSAL-Python (test.pypi.org) connection
65+
66+
1. **Project Settings → Service connections → New service connection**
67+
2. Choose **Python package upload**, click **Next**.
68+
3. Fill in:
69+
| Field | Value |
70+
|-------|-------|
71+
| **Twine repository URL** | `https://test.pypi.org/legacy/` |
72+
| **EndpointName** (`-r` value) | `MSAL-Test-Python-Upload` |
73+
| **Username** | `__token__` |
74+
| **Password** | *(your test.pypi.org API token, e.g. `pypi-AgA...`)* |
75+
| **Service connection name** | `MSAL-Test-Python-Upload` |
76+
4. Check **Grant access permission to all pipelines**, click **Save**.
77+
78+
### 3b — PyPI (production) connection
79+
80+
1. **Project Settings → Service connections → New service connection**
81+
2. Choose **Python package upload**, click **Next**.
82+
3. Fill in:
83+
| Field | Value |
84+
|-------|-------|
85+
| **Twine repository URL** | `https://upload.pypi.org/legacy/` |
86+
| **EndpointName** (`-r` value) | `MSAL-Prod-Python-Upload` |
87+
| **Username** | `__token__` |
88+
| **Password** | *(your PyPI API token)* |
89+
| **Service connection name** | `MSAL-Prod-Python-Upload` |
90+
4. Check **Grant access permission to all pipelines**, click **Save**.
91+
92+
> **Security note:** Never commit API tokens to source control. All secrets
93+
> are stored in ADO service connections and injected by `TwineAuthenticate@1`
94+
> via the ephemeral `$(PYPIRC_PATH)` file at pipeline runtime.
95+
96+
---
97+
98+
## Step 4 — Create ADO Environments
99+
100+
Environments let you add approval gates before the deployment jobs run.
101+
102+
1. Go to **Pipelines → Environments → New environment**.
103+
2. Create two environments:
104+
105+
| Name | Description |
106+
|------|-------------|
107+
| `MSAL-Python` | Staging — test.pypi.org uploads |
108+
| `MSAL-Python-Release` | Production — PyPI uploads (**add approval check**) |
109+
110+
3. For the `MSAL-Python-Release` environment:
111+
- Click the `MSAL-Python-Release` environment → **Approvals and checks → +**
112+
- Add **Approvals** → add the release approver(s) (e.g., release manager).
113+
- This ensures a human must approve before the wheel is pushed to production.
114+
115+
---
116+
117+
## Step 5 — Create the Pipeline in ADO
118+
119+
1. Go to **Pipelines → New pipeline**.
120+
2. Select **GitHub** as the code source.
121+
3. Pick the repository **AzureAD/microsoft-authentication-library-for-python**.
122+
- ADO will use the `github-msal-python` service connection created in Step 2.
123+
4. Choose **Existing Azure Pipelines YAML file**.
124+
5. Set the path to: `/.Pipelines/pipeline-publish.yml`
125+
6. Click **Continue** → review the YAML → click **Save** (not *Run*).
126+
7. Rename the pipeline to something descriptive, e.g.
127+
`msal-python · publish`.
128+
129+
> **Note:** The existing `azure-pipelines.yml` (CI-only, runs on `dev`) is a
130+
> separate pipeline and is not affected.
131+
132+
---
133+
134+
## Step 6 — Authorize Pipelines to use Service Connections
135+
136+
When the pipeline first uses a service connection you may be prompted to
137+
authorize it. To pre-authorize:
138+
139+
1. **Project Settings → Service connections** → click a connection →
140+
**Security** tab.
141+
2. Set the **Pipeline permissions** to include the new publish pipeline.
142+
143+
Repeat for all three connections: `github-msal-python`, `MSAL-Test-Python-Upload`,
144+
`MSAL-Prod-Python-Upload`.
145+
146+
---
147+
148+
## Step 7 — Pipeline Parameters (Run Pipeline UI)
149+
150+
This pipeline is **always manually queued**. Both fields are required — the Validate stage fails if either is missing or the version doesn’t match `msal/sku.py`:
151+
152+
| Parameter | Required | Description | Example values |
153+
|-----------|----------|-------------|----------------|
154+
| **Package version to publish** | ✅ Yes | Must exactly match `msal/sku.py __version__`. | `1.36.0` (release), `1.36.0rc1` (preview) |
155+
| **Publish target** | ✅ Yes | Explicit destination — no auto-routing. | `MSAL-Python` (test.pypi.org) or `pypi` (production) |
156+
157+
---
158+
159+
## Step 8 — End-to-End Release Walkthrough
160+
161+
### Publishing a preview / release candidate to test.pypi.org
162+
163+
1. Set `msal/sku.py __version__ = "1.36.0rc1"`
164+
2. Go to **Pipelines → Run pipeline**
165+
3. Enter `packageVersion = 1.36.0rc1`, select `publishTarget = MSAL-Python`
166+
4. Click **Run** — pipeline runs: Validate → CI → Build → PublishMSALPython
167+
5. Verify at <https://test.pypi.org/project/msal/>
168+
169+
### Publishing a production release to PyPI
170+
171+
1. Set `msal/sku.py __version__ = "1.36.0"` and merge to the release branch
172+
2. Go to **Pipelines → Run pipeline**
173+
3. Enter `packageVersion = 1.36.0`, select `publishTarget = pypi`
174+
4. Click **Run** — pipeline runs: Validate → CI → Build → PublishPyPI (approval gate)
175+
5. Go to **Pipelines → Environments → MSAL-Python-Release** and approve the deployment
176+
6. Verify: `pip install msal==1.36.0` or check <https://pypi.org/project/msal/>
177+
178+
## Pipeline Trigger Reference
179+
180+
```
181+
Manual queue (publishTarget = MSAL-Python)
182+
└─► Validate ─► CI ─► Build ─► PublishMSALPython
183+
(test.pypi.org, auto)
184+
185+
Manual queue (publishTarget = pypi)
186+
└─► Validate ─► CI ─► Build ─► PublishPyPI
187+
(PyPI, requires approval)
188+
```
189+
190+
---
191+
192+
## Troubleshooting
193+
194+
| Symptom | Likely cause | Fix |
195+
|---------|-------------|-----|
196+
| `403` on twine upload | Token expired or wrong scope | Regenerate API token on pypi.org; update the service connection |
197+
| `File already exists` error | Version already published; PyPI does not allow overwriting | Bump version in `msal/sku.py` |
198+
| Pipeline not triggered by tag | ADO only picks up tags after the pipeline is saved with the `tags:` trigger | Re-save the pipeline in ADO after adding the trigger |
199+
| `TwineAuthenticate` says endpoint not found | Service connection name mismatch | Ensure `pythonUploadServiceConnection` value exactly matches the service connection name |
200+
201+
---
202+
203+
## References
204+
205+
- [Publish Python packages with Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/pypi?view=azure-devops)
206+
- [TwineAuthenticate@1 task reference](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/twine-authenticate-v1?view=azure-devops)
207+
- [Publish and download Python packages with Azure Artifacts](https://learn.microsoft.com/en-us/azure/devops/artifacts/quickstarts/python-packages?view=azure-devops)
208+
- [Python package upload service connection](https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints#python-package-upload-service-connection)
209+
- [ADO Environments – approvals and checks](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops)

‎.Pipelines/pipeline-publish.yml‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# pipeline-publish.yml
2+
#
3+
# Publish pipeline for the msal Python package.
4+
# Source: https://github.com/AzureAD/microsoft-authentication-library-for-python
5+
#
6+
# Composes reusable templates from this folder:
7+
# template-run-tests.yml - pytest across Python version matrix
8+
# template-build-package.yml - sdist + wheel build + twine check
9+
# template-publish-package.yml - TwineAuthenticate + twine upload (parameterized)
10+
#
11+
# Trigger logic:
12+
# This pipeline is MANUALLY TRIGGERED ONLY.
13+
# Both packageVersion and publishTarget must be explicitly set at queue time.
14+
#
15+
# One-time ADO setup: see ADO-PUBLISH-SETUP.md
16+
17+
# ── Pipeline parameters ────────────────────────────────────────────────────────
18+
# Both fields are shown as required inputs in the ADO "Run pipeline" UI.
19+
# Neither has a default — the Validate stage will fail if either is empty or
20+
# if packageVersion does not match msal/sku.py __version__.
21+
parameters:
22+
- name: packageVersion
23+
displayName: 'Package version to publish (must match msal/sku.py, e.g. 1.36.0 or 1.36.0rc1)'
24+
type: string
25+
26+
- name: publishTarget
27+
displayName: 'Publish target'
28+
type: string
29+
values:
30+
- MSAL-Python # publishes to test.pypi.org (staging / preview)
31+
- pypi # publishes to PyPI (production)
32+
33+
trigger: none # manual runs only — no automatic branch or tag triggers
34+
pr: none
35+
36+
variables:
37+
pythonBuildVersion: '3.12' # single version used for build + publish jobs
38+
39+
# ══════════════════════════════════════════════════════════════════════════════
40+
# Stage 1 · Validate — verify packageVersion matches msal/sku.py before
41+
# anything else runs.
42+
# ══════════════════════════════════════════════════════════════════════════════
43+
stages:
44+
- stage: Validate
45+
displayName: 'Validate version'
46+
jobs:
47+
- job: ValidateVersion
48+
displayName: 'Assert packageVersion matches msal/sku.py'
49+
pool:
50+
vmImage: ubuntu-latest
51+
steps:
52+
- task: UsePythonVersion@0
53+
inputs:
54+
versionSpec: '3.12'
55+
displayName: 'Use Python 3.12'
56+
57+
- bash: |
58+
PARAM_VER="${{ parameters.packageVersion }}"
59+
SKU_VER=$(python -c "import sys; sys.path.insert(0,'msal'); from sku import __version__; print(__version__)")
60+
61+
if [ -z "$PARAM_VER" ]; then
62+
echo "##vso[task.logissue type=error]packageVersion is required. Enter the version to publish (must match msal/sku.py __version__)."
63+
exit 1
64+
elif [ "$PARAM_VER" != "$SKU_VER" ]; then
65+
echo "##vso[task.logissue type=error]Version mismatch: parameter '$PARAM_VER' != msal/sku.py '$SKU_VER'"
66+
echo "Update msal/sku.py __version__ to match the packageVersion parameter, or correct the parameter."
67+
exit 1
68+
else
69+
echo "Version validated: $PARAM_VER"
70+
fi
71+
displayName: 'Validate version matches msal/sku.py'
72+
73+
# ══════════════════════════════════════════════════════════════════════════════
74+
# Stage 2 · CI — run the full test matrix
75+
# ══════════════════════════════════════════════════════════════════════════════
76+
- stage: CI
77+
displayName: 'Test'
78+
dependsOn: Validate
79+
condition: succeeded()
80+
jobs:
81+
- job: Test
82+
displayName: 'pytest – Python $(python.version)'
83+
pool:
84+
vmImage: ubuntu-latest
85+
strategy:
86+
matrix:
87+
Python39:
88+
python.version: '3.9'
89+
Python310:
90+
python.version: '3.10'
91+
Python311:
92+
python.version: '3.11'
93+
Python312:
94+
python.version: '3.12'
95+
Python313:
96+
python.version: '3.13'
97+
steps:
98+
- template: template-run-tests.yml # python.version resolved from matrix
99+
100+
# ══════════════════════════════════════════════════════════════════════════════
101+
# Stage 3 · Build — compile sdist + wheel (single Python version)
102+
# ══════════════════════════════════════════════════════════════════════════════
103+
- stage: Build
104+
displayName: 'Build distribution'
105+
dependsOn: CI
106+
condition: succeeded()
107+
jobs:
108+
- job: BuildDist
109+
displayName: 'Build sdist + wheel'
110+
pool:
111+
vmImage: ubuntu-latest
112+
steps:
113+
- template: template-build-package.yml
114+
parameters:
115+
pythonVersion: '3.12' # must be a literal — template params resolve at compile time
116+
artifactName: python-dist
117+
118+
# ══════════════════════════════════════════════════════════════════════════════
119+
# Stage 4a · Publish to MSAL-Python (test.pypi.org)
120+
# Runs when: publishTarget == 'MSAL-Python'
121+
# ══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
122+
- stage: PublishMSALPython
123+
displayName: 'Publish → MSAL-Python (test.pypi.org)'
124+
dependsOn: Build
125+
condition: >
126+
and(
127+
succeeded(),
128+
eq('${{ parameters.publishTarget }}', 'MSAL-Python')
129+
)
130+
jobs:
131+
- deployment: DeployMSALPython
132+
displayName: 'Upload to MSAL-Python (test.pypi.org)'
133+
pool:
134+
vmImage: ubuntu-latest
135+
# Optional: add approval checks in ADO → Pipelines → Environments → MSAL-Python
136+
environment: MSAL-Python
137+
strategy:
138+
runOnce:
139+
deploy:
140+
steps:
141+
- template: template-publish-package.yml
142+
parameters:
143+
serviceConnectionName: MSAL-Test-Python-Upload
144+
repositoryName: MSAL-Test-Python-Upload
145+
artifactName: python-dist
146+
pythonVersion: '3.12' # must be a literal — template params resolve at compile time
147+
skipExisting: true
148+
149+
# ══════════════════════════════════════════════════════════════════════════════
150+
# Stage 4b · Publish to PyPI
151+
# Runs when: publishTarget == 'pypi'
152+
# ══════════════════════════════════════════════════════════════════════════════
153+
- stage: PublishPyPI
154+
displayName: 'Publish → PyPI'
155+
dependsOn: Build
156+
condition: >
157+
and(
158+
succeeded(),
159+
eq('${{ parameters.publishTarget }}', 'pypi')
160+
)
161+
jobs:
162+
- deployment: DeployPyPI
163+
displayName: 'Upload to PyPI'
164+
pool:
165+
vmImage: ubuntu-latest
166+
# IMPORTANT: configure a required manual approval on this environment in
167+
# ADO → Pipelines → Environments → MSAL-Python-Release → Approvals and checks.
168+
environment: MSAL-Python-Release
169+
strategy:
170+
runOnce:
171+
deploy:
172+
steps:
173+
- template: template-publish-package.yml
174+
parameters:
175+
serviceConnectionName: MSAL-Prod-Python-Upload
176+
repositoryName: MSAL-Prod-Python-Upload
177+
artifactName: python-dist
178+
pythonVersion: '3.12' # must be a literal — template params resolve at compile time
179+
skipExisting: false

0 commit comments

Comments
 (0)