ci: automate release process and harden test workflow by wowi42 · Pull Request #1691 · pyinfra-dev/pyinfra

wowi42 · 2026-04-24T09:39:47Z

Automates the release pipeline and tightens the existing test workflow.

Release automation

New release.yml: pushing a v* tag builds with uv build, checks the artifact versions match the tag, publishes to PyPI via trusted publishing (OIDC, release environment), then creates a GitHub release.
The release body comes from scripts/generate_changelog.py, run against the previous tag, with gh attribution for external contributors. It falls back to Release vX.Y.Z if the script produces nothing.
New Release Drafter setup keeps a draft of the next release updated from merged PRs. Categories and version resolution map to the labels that already exist in this repo (API, CLI, operations, facts, connectors, documentation, dependency issue, new feature, bug). A breaking label needs creating before it can drive major bumps.

Test workflow

Concurrency groups cancel superseded runs, for pull requests only, so branch and tag pushes always produce a full result.
Tests also trigger on v* tags.
fail-fast: false on both test matrices, so one failing job does not hide the results of the others.

Security

Every action introduced by this PR is pinned to a commit SHA at its latest release: checkout v6.0.3, setup-python v6.2.0, setup-uv v8.2.0, pypi-publish v1.14.0, action-gh-release v3.0.0, release-drafter v7.3.1.
The Dependabot config was dropped after review (supply chain trade-off, see discussion above).

Setup needed before this works

Create the release GitHub environment and configure the PyPI trusted publisher for this repo and workflow.
Optionally create a breaking label for major version resolution.

Pull request is based on the default branch (3.x at this time)
Tests pass (see scripts/dev-test.sh)
Type checking & code style passes (see scripts/dev-lint.sh)

- Add concurrency groups to cancel redundant test runs - Trigger tests on v* tags in addition to branches - Add fail-fast: false to test matrices - Create automated release workflow (tag → build → PyPI → GitHub release) - Add Release Drafter for auto-generated changelogs from PR labels - Add Dependabot for weekly dependency updates (grouped)

Fizzadar

Quick drive by review, two security related concerns. Release stuff looks neat will play around with it over the weekend!

Fizzadar · 2026-04-24T11:10:00Z

@@ -0,0 +1,41 @@
+version: 2


With all the supply chain attacks right now (and beyond, this feels like the beginning) I feel automating dependency update is high risk, low gain.

Historically legacy setup py meant users got the latest matching major dependency versions but uv changed that to specific versions.

I wonder if we do the delay thing, but that also sucks because if everyone does no one gains anything.

I think this needs discussion, I recommend moving it out of scope for this PR do now and opening a new issue just for that.

- Drop .github/dependabot.yml; defer to a separate discussion/issue per review (supply-chain concerns). - Pin new GitHub Actions references in release.yml and release-drafter.yml to exact commit SHAs (with # version comments) to mitigate tag-hijack attacks.

wowi42 · 2026-04-24T17:57:26Z

Thanks for the quick review @Fizzadar! Addressed both security concerns in 26cdc4c3:

Dependabot : Dropped .github/dependabot.yml from this PR. Agreed on the supply-chain risk trade-off; happy to open a separate issue to discuss a policy (delay windows, manual triage, or scoping to specific ecosystems) without blocking the release-automation work.

Pinning new actions to commit SHAs : Pinned every new action reference introduced by this PR to an exact commit, with a trailing # v<tag> comment for readability:

.github/workflows/release.yml
- actions/checkout@93cb6efe… # v5
- actions/setup-python@a26af69b… # v5
- astral-sh/setup-uv@d0cc045d… # v6
- pypa/gh-action-pypi-publish@cef22109… # release/v1
- softprops/action-gh-release@3bb12739… # v2
.github/workflows/release-drafter.yml
- release-drafter/release-drafter@6a93d829… # v6

I left the pre-existing actions in test.yml / docs.yml alone since they weren't introduced by this PR, happy to do a follow-up that sweeps the whole repo if you'd like that tracked as a separate effort.

DonDebonair · 2026-04-25T15:45:25Z

I wonder if we should first decide on if/how we want to adopt Conventional Commits and see how this changes the use of Release Drafter.

The tag-triggered run had HEAD on the tagged commit, so the script's own tag detection found the new tag itself and produced an empty commit range. The output redirect also always created the file, so the -f check could never fall back and the release body ended up empty. Pass the previous tag explicitly, check for non-empty output and provide GH_TOKEN so the script's gh api attribution lookups work. Also sanity check that the built artifacts carry the tag's version before publishing, guarding against dynamic-versioning misfires.

The category and version-resolver labels mostly did not exist in the repo (api, core, cli, docs, enhancement, bugfix, ...), so nothing would ever be categorised and every release resolved to patch. Use the actual label set (API, CLI, operations, facts, connectors, documentation, dependency issue, new feature, bug); 'breaking' still needs creating. The pull_request trigger does nothing without an autolabeler config and fails on fork PRs where GITHUB_TOKEN is read-only, so remove it along with the now-unneeded pull-requests permission.

Branch and tag pushes should always produce a full test result, release tags especially - a cancelled tag run would leave a release with no test signal.

checkout v5.0.1 -> v6.0.3, setup-python v5.6.0 -> v6.2.0, setup-uv v6.8.0 -> v8.2.0, action-gh-release v2.6.2 -> v3.0.0, release-drafter v6.4.0 -> v7.3.1. All new majors are node24 runtime bumps, drop-in on GitHub-hosted runners; setup-uv v8 also stops publishing major tags and drafter v7 reads the token input (defaults to github.token) so the env block goes. pypi-publish was already at the latest v1.14.0.

wowi42 · 2026-06-11T12:21:29Z

Pushed a few follow-up commits to harden the release flow and align it with the repo. Summary of what changed:

Release changelog extraction and artifact check (0c1033de)

The changelog step ran generate_changelog.py from the tagged commit. The script's own tag detection found the current tag and produced an empty range, so release notes came out blank. It now resolves the previous tag from the tag's parent (git describe --tags --abbrev=0 "${VERSION}^") and passes that to the script, so the range is correct.
Renamed the heredoc marker to CHANGELOG_EOF and switched the empty check to [ -s ... ] so an empty file falls back to a plain Release <version> body instead of emitting nothing.
Added a step that lists dist/pyinfra-<version>-py3-none-any.whl and the sdist before publishing, so a tag/artifact version mismatch fails the job before it reaches PyPI.
Quoted $GITHUB_OUTPUT and used GITHUB_REF_NAME instead of stripping refs/tags/ by hand.

Release Drafter labels (63d53eca)

The categories and version-resolver referenced labels that do not exist in this repo (core, cli, docs, enhancement, bugfix, ...). They now map to the actual labels: API, CLI, documentation, dependency issue, new feature (minor), bug (patch). Added a comment noting that breaking still needs creating before it can drive a major bump.
Dropped the pull_request trigger and the pull-requests: write permission from the release-drafter workflow. The draft only needs to update on push to the version branches, so this removes write access it did not need.

Test workflow concurrency (a3926518)

cancel-in-progress was always true, which could cancel a branch or release-tag run mid-way. It is now ${{ github.event_name == 'pull_request' }}, so only superseded PR runs get cancelled and branch/tag pushes always run to completion.

Action pins (df983a54)

Bumped the pinned SHAs to current releases (checkout v6.0.3, setup-python v6.2.0, setup-uv v8.2.0, release-drafter v7.3.1, action-gh-release v3.0.0) and corrected the version comments on the pins.

Fizzadar requested changes Apr 24, 2026

View reviewed changes

DonDebonair mentioned this pull request Apr 24, 2026

meta: add PR review skill #1683

Merged

wowi42 mentioned this pull request Apr 24, 2026

docs: expand PR template and document PR labels #1697

Closed

5 tasks

Merge branch '3.x' into feat/ci-cd-release-improvements

a272e11

wowi42 added the new feature label May 7, 2026

wowi42 added 7 commits May 7, 2026 10:29

Merge branch '3.x' into feat/ci-cd-release-improvements

360b02b

Merge branch '3.x' into feat/ci-cd-release-improvements

df1e6d2

Merge branch '3.x' into feat/ci-cd-release-improvements

b96e868

ci: only cancel in-progress test runs for pull requests

a392651

Branch and tag pushes should always produce a full test result, release tags especially - a cancelled tag run would leave a release with no test signal.

wowi42 marked this pull request as ready for review June 11, 2026 07:06

wowi42 changed the title ~~ci: improve CI/CD pipeline and automate release process~~ ci: automate release process and harden test workflow Jun 11, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

ci: automate release process and harden test workflow#1691

ci: automate release process and harden test workflow#1691
wowi42 wants to merge 10 commits into
pyinfra-dev:3.xfrom
KalvadTech:feat/ci-cd-release-improvements

wowi42 commented Apr 24, 2026 •

edited

Loading

Uh oh!

Fizzadar left a comment

Uh oh!

Fizzadar Apr 24, 2026

Uh oh!

Uh oh!

wowi42 commented Apr 24, 2026 •

edited

Loading

Uh oh!

DonDebonair commented Apr 25, 2026

Uh oh!

wowi42 commented Jun 11, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

wowi42 commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release automation

Test workflow

Security

Setup needed before this works

Uh oh!

Fizzadar left a comment

Choose a reason for hiding this comment

Uh oh!

Fizzadar Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wowi42 commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonDebonair commented Apr 25, 2026

Uh oh!

wowi42 commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wowi42 commented Apr 24, 2026 •

edited

Loading

wowi42 commented Apr 24, 2026 •

edited

Loading

wowi42 commented Jun 11, 2026 •

edited

Loading