Skip to content

ci(release): verify selected prepare artifacts - #25

Merged
farhan-syah merged 2 commits into
NodeDB-Lab:mainfrom
presempathy-awb:codex/upstream/release-artifact-authentication
Jul 28, 2026
Merged

ci(release): verify selected prepare artifacts#25
farhan-syah merged 2 commits into
NodeDB-Lab:mainfrom
presempathy-awb:codex/upstream/release-artifact-authentication

Conversation

@presempathy-awb

@presempathy-awb presempathy-awb commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Reject a manually selected Release Prepare run unless it is a successful run of
.github/workflows/release-prepare.yml for the exact commit referenced by the
requested release tag, and unless the downloaded artifacts are exactly the five
versioned pagedb-fsck archives the release publishes.

This is the deliberately narrow follow-up to the useful portion of #24. It does
not add a provenance document, a shared prepare gate, locked publication,
transactional version stamping, release documentation, action pinning, or
literal workflow-structure tests.

Problem

The Release workflow accepts a maintainer-supplied prepare_run_id and uses
actions/download-artifact to retrieve every fsck-* artifact from that run.
The action already scopes the run lookup to the current repository, so an
additional repository field would not be an independent check. The version in
each expected archive name already covers the requested tag version.

Two independent facts remain worth checking:

  1. The selected run must have prepared the same commit that the release tag now
    resolves to.
  2. The selected run must contain the complete release attachment set and
    nothing else matching the release artifact pattern.

Without those checks, a valid run ID can point to a different successful
workflow or tag build. fail_on_unmatched_files only proves that at least one
downloaded file matches artifacts/*; it does not prove that all five target
archives are present or that they came from the tag commit being released.

Change

The two facts are checked by two scripts, ordered so the provenance decision is
made before any archive reaches the runner.

scripts/ci/verify_prepare_run.sh <run.json> <commit> runs first. The
github-release job reads the selected run through GitHub's
repository-scoped GET /repos/{owner}/{repo}/actions/runs/{run_id} endpoint via
gh api, then requires the returned run to have:

  • head_sha equal to the commit checked out from the requested tag;
  • workflow path .github/workflows/release-prepare.yml;
  • event push;
  • status completed;
  • conclusion success.

Fields are compared one at a time rather than as a single boolean, so a
rejection names the field that disagreed — the maintainer needs to know whether
they picked the wrong run, the wrong tag, or a run that has not finished yet.
Because this step precedes actions/download-artifact, a wrong, unfinished, or
foreign run is rejected without its archives ever being fetched or unpacked.

scripts/ci/verify_release_artifacts.sh <artifact-dir> <version> runs after
the download and requires exactly these five regular, non-symlink files for the
validated version:

  • Linux x64 tarball;
  • Linux arm64 tarball;
  • macOS arm64 tarball;
  • macOS x64 tarball;
  • Windows x64 zip.

Both directions are checked by name: every expected archive must be present, and
every present entry must be expected. A cardinality comparison would be shorter
but would let one absence and one intruder cancel out, and it reports "wrong
number of files" where the useful message is which file.

A missing target, an extra downloaded attachment, a different version, a
different workflow, an unsuccessful prepare run, or a different tag commit
fails before softprops/action-gh-release runs.

The existing job-level actions: read permission is sufficient; no new
repository permission or secret is introduced. gh and jq are both preinstalled
on the runner image.

Scope of the guarantee

These checks establish provenance and completeness, not content integrity.
Nothing re-derives a digest for the archives. The archives are trusted on the
strength of a run pinned to the tag commit, the prepare workflow file, and a
successful conclusion. That boundary is stated at the top of
verify_prepare_run.sh so a future reader does not mistake it for an integrity
check.

Independence and retry behavior

The crates.io stage is unchanged. It still depends only on the existing
validate-version job, builds from the tag checkout, and can be retried after
the prepare artifacts expire.

Only github-release consumes the prepared binaries, so only that stage
performs this artifact check. The existing manual stage controls remain intact:
a crates.io outage does not block GitHub Release creation, and a GitHub Release
failure does not require republishing the crate.

The prepare workflow is also unchanged. It continues to build and retain the
same five artifacts for 14 days; no extra upload job or provenance artifact is
added.

Tests

scripts/ci/test_release_verifiers.sh is a compact generated-data regression
test rather than tracked fixture directories or workflow-text greps. It performs
no network access and asserts observable verifier behavior only — never YAML
layout, job names, needs syntax, or literal workflow strings.

Every negative case asserts the reason for the rejection, not merely a nonzero
exit, so a verifier that breaks in an unrelated way — bad arity, a typo in a
guard, a missing dependency — cannot pass by failing for the wrong reason.
Twenty cases cover:

  • prepare run: the successful tag build; a differing head_sha; an unsuccessful
    conclusion; a run still in progress; a different workflow path; a
    workflow_dispatch event; an absent field; malformed JSON; an absent metadata
    file; a malformed commit argument;
  • artifacts: the exact five-archive set; an extra attachment; a hidden dotfile
    attachment; a missing archive at full attachment count; an absent archive; a
    symlinked archive; a directory in place of an archive; a version mismatch; a
    malformed version argument; an absent artifact directory.

The missing-archive case renames rather than deletes, keeping the attachment
count at five so the by-name check is genuinely exercised.

The suite was mutation-tested to confirm it detects the failures it claims to.
Removing the missing-archive check produces three failures; removing the
head_sha provenance pin is caught by the differing-commit case.

Both scripts are held to shellcheck by a new lint step, so they stay clean
rather than relying on a one-time local run.

Local verification:

bash scripts/ci/test_release_verifiers.sh
  20 cases, all ok
  release artifact verification: tests passed

shellcheck scripts/ci/*.sh
  exit 0

actionlint .github/workflows/release.yml .github/workflows/test.yml
  exit 0

cargo fmt --all --check
  exit 0

git diff --check
  exit 0

The verifiers were also exercised against the repository's real successful
Release Prepare run
29916913669
for v0.1.0-beta.3. GitHub reported head
6e01fcbad579af61657c9f5367b6cbf425e5b976, and the run still retains exactly
the five expected platform archives. Passing the live API response to
verify_prepare_run.sh produced prepare run: ok, and it correctly rejected the
same response against a mismatched tag commit. Downloading those archives and
reproducing the job's merge-multiple: true flat layout produced
release artifacts: ok, and adding a stray attachment to that directory was
rejected. This read-only proof did not dispatch a workflow, create a release, or
publish a crate.

Zizmor's pedantic, low-or-higher comparison reports the same 48 pre-existing
findings on current main and this branch: 33 unpinned action uses, 10
artipacked, four template-injection findings, and one undocumented-permission
finding. This PR introduces no additional finding. The unpinned action uses are
intentionally reserved for a separate SHA-pinning PR with Dependabot ownership.

Scope

Base: 031db94b4184f464d2dac1c901da05688fd2f7de

Head: 452fc883031de8aa1fa27e32bf36225b294b9dc0

Five files change, with no Rust source, package manifest, lockfile, public API,
on-disk format, benchmark, or release-documentation change. Runtime benchmark
comparisons are not applicable because the change executes only in the manual
GitHub Release workflow and its shell regression test.

Split release verification into two focused checks: verify_prepare_run.sh
confirms the selected Release Prepare run is a successful, tag-pushed build
of the exact release commit before its artifacts are ever downloaded, and
verify_release_artifacts.sh now only checks the downloaded attachment set
against the expected archive list. Add regression tests for both verifiers
and run them with shellcheck in CI.
@farhan-syah
farhan-syah merged commit c9aa941 into NodeDB-Lab:main Jul 28, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants