Skip to content

ci(release): authenticate prepared artifacts - #24

Closed
presempathy-awb wants to merge 1 commit into
NodeDB-Lab:mainfrom
presempathy-awb:codex/upstream/release-provenance-current
Closed

ci(release): authenticate prepared artifacts#24
presempathy-awb wants to merge 1 commit into
NodeDB-Lab:mainfrom
presempathy-awb:codex/upstream/release-provenance-current

Conversation

@presempathy-awb

Copy link
Copy Markdown
Contributor

Summary

This PR makes PageDB's two-stage release process fail closed around the exact
source and binary artifacts that were prepared for a tag.

The current workflow already separates an expensive tag-triggered preparation
run from a manually dispatched distribution run. That is the right overall
shape, but the preparation run ID is currently treated as sufficient evidence
that the downloaded files belong to the requested release. Crate publication
also validates the tag without depending on the preparation run at all. A
maintainer can therefore select a successful but unrelated run, a partial
artifact set can reach distribution, and a prerelease version stamp can leave
Cargo.lock inconsistent with Cargo.toml.

The new contract is:

  1. A pushed v* tag is validated against the package base version and resolved
    to its peeled commit.
  2. Release Prepare runs the complete reusable CI workflow, builds the five
    supported pagedb-fsck archives once, and records a small immutable
    provenance document.
  3. Manual distribution accepts a tag and preparation run ID, downloads that
    run's provenance and archives, and requires exact agreement on repository,
    tag, version, commit, and run ID.
  4. The verifier also requires exactly the five expected archive names, with no
    missing, extra, renamed, or non-file entries.
  5. Both crates.io publication and GitHub Release creation depend on that one
    shared acceptance gate, while remaining independently retryable.

The change is intentionally confined to release automation, executable release
contracts, the tracked lockfile metadata needed by locked packaging, and
maintainer documentation. It does not modify PageDB runtime Rust code, storage
algorithms, database behavior, file formats, browser code, or benchmark
implementations.

Why this is needed

A run ID is a locator, not provenance

actions/download-artifact can select another workflow run by numeric ID, but
the successful download only proves that the run exists and contains artifacts
matching the requested pattern. It does not prove that the run was created from
this repository, for this tag, from this commit, with this package version.

Release Prepare now uploads release-provenance.json with:

  • schema version;
  • repository identity;
  • tag;
  • parsed package version;
  • peeled tag commit;
  • preparation run ID.

The distribution workflow checks every field against values independently
derived from the requested release. The run ID still locates the artifacts, but
the content establishes whether those artifacts are acceptable.

Publication must not bypass preparation

Previously, the GitHub Release stage consumed prepared artifacts while the
crates.io stage depended only on tag validation. That allowed the two outputs of
one release to cross different trust boundaries.

This PR introduces a shared validate-prepare job. Both publication jobs
declare:

needs: [validate-version, validate-prepare]

The crate and binary stages remain independent after that point. A crates.io
outage does not require rebuilding binaries, and a GitHub Release-body retry
does not require republishing the crate. A retry reuses the same accepted tag
and preparation run.

Locked prerelease stamping must update both files

The current version stamper edits the first version = "..." line in
Cargo.toml. There are two problems with that approach:

  • textual position is weaker than explicitly selecting the root [package]
    field;
  • a prerelease stamp changes the package version while the workspace package
    entry in Cargo.lock still carries the base version.

That second case is observable with cargo package --locked: Cargo requests a
lockfile update and the release fails after the tag has already been prepared.

The replacement stamper treats Cargo.toml and Cargo.lock as one
transactional release unit. It:

  • accepts only X.Y.Z or X.Y.Z-label.N;
  • rewrites only the root [package] version;
  • preserves the manifest's file mode;
  • uses a targeted offline Cargo update to align the root lock entry;
  • is idempotent when the requested version is already present;
  • restores both files if lockfile alignment fails;
  • rejects malformed and command-shaped input before changing a file.

--allow-dirty remains necessary for the deliberate release-time manifest and
lock edits. It does not disable Cargo's package verification. Build, metadata,
package, and publish operations remain locked.

Artifact acceptance contract

The provenance verifier expects exactly these versioned files:

  • pagedb-fsck-${VERSION}-linux-x64.tar.gz;
  • pagedb-fsck-${VERSION}-linux-arm64.tar.gz;
  • pagedb-fsck-${VERSION}-macos-arm64.tar.gz;
  • pagedb-fsck-${VERSION}-macos-x64.tar.gz;
  • pagedb-fsck-${VERSION}-windows-x64.zip.

The directory must contain five entries, each entry must be a regular file, and
each expected name must exist. This rejects both an incomplete matrix and an
over-broad artifact glob that introduces unexpected release attachments.

The macOS x64 archive continues to cross-compile on the current Apple Silicon
macos-latest runner. No dependency on the retired Intel runner is
reintroduced.

Workflow and supply-chain hardening

All third-party actions touched by the release path and its transitively invoked
test workflow are pinned to immutable commits. Each pin was checked against the
corresponding official ref immediately before preparing this PR:

  • actions/checkout v7:
    3d3c42e5aac5ba805825da76410c181273ba90b1;
  • actions/upload-artifact v7:
    043fb46d1a93c77aae656e7c1c64a875d1fc6a0a;
  • actions/download-artifact v8:
    3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c;
  • softprops/action-gh-release v3 peeled commit:
    3d0d9888cb7fd7b750713d6e236d1fcb99157228;
  • dtolnay/rust-toolchain stable:
    4cda84d5c5c54efe2404f9d843567869ab1699d4;
  • Swatinem/rust-cache v2 peeled commit:
    e18b497796c12c097a38f9edb9d0641fb99eee32;
  • taiki-e/install-action v2.85.2:
    41049aa56687c35e0afa74eed4f09cec4f9afabf.

Every checkout disables persisted credentials. The release artifact build does
not restore the Rust build cache, avoiding cache state in the bytes promoted to
a release.

Matrix values are passed through environment variables and a fixed
configuration case rather than interpolated directly into shell source. This
removes template-injection findings without changing the target or feature
coverage.

Permissions are job-scoped and documented inline. The preparation workflows are
read-only. Artifact validation receives actions: read; GitHub Release
creation receives contents: write; crates.io publication continues to use the
protected crates.io environment and its existing registry token.

The release workflows do not install or depend on RocksDB, SQLite, redb,
Clang, libclang, or any external comparison engine. The benchmark job still
installs its own compilation prerequisite inside the reusable test workflow,
where that dependency belongs; it is not a prerequisite of release packaging
or publication.

Executable regression coverage

Three shell contract suites are added to the reusable CI workflow.

scripts/test-stamp-version.sh covers:

  • same-version idempotence;
  • prerelease stamping;
  • root [package] selection without modifying metadata versions;
  • Cargo's observed package version;
  • lockfile alignment;
  • locked package creation after stamping;
  • manifest mode preservation;
  • a repeated no-op stamp;
  • rollback of both files when Cargo cannot align a corrupt lockfile;
  • malformed, missing, and command-shaped inputs;
  • absence of shell execution from rejected input.

scripts/test-release-provenance.sh creates an independent fixture and covers:

  • acceptance of a complete, matching preparation record;
  • rejection of mismatched repository, tag, version, commit, and run ID;
  • rejection of an unsupported schema;
  • rejection of a missing platform archive;
  • rejection of an unexpected extra attachment.

scripts/test-release-workflow-contract.sh covers the workflow wiring itself:

  • tag-triggered preparation and manual-only distribution;
  • reusable version validation;
  • use of the full reusable CI workflow;
  • exact tag-commit output;
  • shared preparation validation for both publication stages;
  • use of the tested provenance verifier;
  • locked build and publish commands;
  • durable execution of all three release contract suites in CI;
  • current macOS x64 runner selection;
  • immutable action pins;
  • disabled persisted checkout credentials;
  • no release-build cache;
  • no external-engine or native-toolchain prerequisite in release workflows.

These are behavior and boundary assertions rather than snapshots of complete
YAML files. They are narrow enough to explain a failure and broad enough to
prevent the release trust boundary from being removed by a later workflow
refactor.

Documentation

RELEASING.md is the operator runbook for the two-stage process. It documents:

  • pre-tag checks and accepted tag forms;
  • local release-contract commands;
  • repository-controlled credential and environment requirements;
  • preparation outputs and retention;
  • exact manual distribution inputs;
  • provenance and archive validation;
  • independent retry commands;
  • post-publication verification;
  • the rule that moving or deleting a published release tag is not a retry
    mechanism.

The README links to the runbook and states the prepare-then-distribute model.

Verification

The candidate is based directly on
NodeDB-Lab/pagedb@f9e6cfaee12a0ebb2436f5cdf81f88273ad582d2.
That hash was rechecked against the upstream main ref immediately before the
final review.

Release contracts and workflow analysis

  • bash scripts/test-stamp-version.sh: passed.
  • bash scripts/test-release-provenance.sh: passed.
  • bash scripts/test-release-workflow-contract.sh: passed.
  • actionlint YAML/expression/workflow validation: passed on all four changed
    workflows.
  • shellcheck on all five changed shell scripts: passed.
  • shellcheck on the inline release validation and distribution shell blocks:
    passed.
  • Zizmor 1.28 auditor persona at low severity: 0 findings.
  • git diff --check: passed.
  • No new SHORTCUT: markers are present.

Native Rust regression and quality gates

  • cargo nextest run -p pagedb --all-features --no-fail-fast:
    589 passed, 5 skipped, 0 failed.
  • The same command with PAGEDB_INVARIANT_CHECKS=1:
    589 passed, 5 skipped, 0 failed.
  • cargo fmt --all --check: passed.
  • cargo clippy -p pagedb --all-targets --all-features -- -D warnings:
    passed.
  • cargo doc -p pagedb --no-deps --all-features: passed.
  • cargo test -p pagedb --doc --all-features: passed.
  • cargo deny check: advisories, bans, licenses, and sources passed. Its
    existing unmatched-license and duplicate-version notices remain warnings.
  • cargo package -p pagedb --locked --allow-dirty: passed and verified the
    packaged crate; 214 files, approximately 1.8 MiB unpacked and 398.3 KiB
    compressed.

The first local Clippy attempt exposed a machine-level Homebrew-versus-rustup
compiler cache mismatch. No source was changed to hide it. Re-running with the
rustup toolchain directory first in PATH rebuilt the affected metadata and
passed. This is recorded here so the successful result is not mistaken for an
unexplained retry.

Feature and target matrix

All four feature configurations passed for library, binaries, tests, and
benches:

  • default;
  • no default features;
  • compression;
  • all features.

Cross-target library checks passed for:

  • x86_64-pc-windows-gnu;
  • aarch64-apple-darwin;
  • x86_64-apple-darwin;
  • aarch64-apple-ios;
  • aarch64-linux-android;
  • armv7-linux-androideabi;
  • wasm32-unknown-unknown with opfs;
  • wasm32-wasip1.

The two WebAssembly targets emit existing unused/dead-code warnings for
native-only members, but both checks exit successfully. This PR does not change
those Rust modules.

Dependency isolation passed for both default and all-feature PageDB graphs:
neither RocksDB nor rusqlite appears in the PageDB package's dependency tree.

Benchmark availability and performance disposition

All benchmark executables still build:

  • PageDB segment;
  • isolated engine-comparison btree;
  • isolated engine-comparison comparison, including its RocksDB and SQLite
    comparison dependencies.

This PR has no production Rust or benchmark diff, so it introduces no runtime
performance path to compare against upstream. Re-running wall-clock benchmarks
would compare the same PageDB Rust sources with local system noise and would not
measure the release workflow changes. The compile-only benchmark gate is the
relevant regression proof for this lane.

Scope boundaries

This PR does not:

  • publish a crate, create a release, push a tag, or alter repository secrets;
  • add RocksDB, browser harnesses, virtual-machine requirements, or release host
    packages to PageDB;
  • change database semantics, storage format, cryptography, VFS behavior, or
    public APIs;
  • change benchmark implementations or claim a new performance result;
  • merge or depend on any divergent Gitea history.

The work was prepared in an isolated checkout. The original dirty PageDB
checkout and its uncommitted work were not modified.

Suggested review order

  1. scripts/ci/verify_release_provenance.sh and its fixture test for the
    acceptance boundary.
  2. .github/workflows/release.yml for the shared gate and independently
    retryable publication stages.
  3. .github/workflows/release-prepare.yml and
    .github/workflows/release-validate.yml for the values authenticated by that
    gate.
  4. scripts/ci/stamp_version.sh and its transactional regression test.
  5. scripts/test-release-workflow-contract.sh and the release-contract entry in
    .github/workflows/test.yml.
  6. RELEASING.md, README, and the two-line Cargo.lock registry metadata fix
    required by locked package verification.

Bind crate and GitHub publication to one verified prepare run, align release-time manifest and lockfile stamping transactionally, and pin the transitive release action surface.

Add executable release contracts plus the maintainer runbook so provenance, artifact completeness, retry behavior, and dependency boundaries remain enforced.
@farhan-syah

Copy link
Copy Markdown
Member

Reviewed this rebased onto current main. Closing without action — the release pipeline it replaces is not broken, and the change is far larger than the problem it addresses.

The provenance document does not add what the description claims. actions/download-artifact defaults repository to the current repo, so a run ID is already same-repo by construction. Of the six recorded fields, repository and prepare_run_id cannot mismatch — prepare_run_id is compared against the run ID the artifacts were downloaded from. tag and version are already covered by the version-stamped archive filenames. Only commit carries independent signal, and only for a moved tag, which RELEASING.md in this same PR declares is not a supported operation.

The genuinely useful part is the exact-five-archive filename assertion in verify_release_provenance.sh. That alone catches the realistic failure mode — a maintainer pasting the wrong prepare run ID into a manual dispatch — without the JSON document, the extra upload job, the extra download step, or the fixture suite.

Two changes are regressions:

  1. publish-crate gaining needs: validate-prepare. The crate is built from the tag by a verifying cargo publish and never consumes the binaries. Coupling them means a single flaky runner in the build matrix blocks crates.io, and once the 14-day artifact retention lapses the crate cannot be published at all without cutting a new tag. The two stages were independent on purpose.

  2. The test.yml comment reverts an earlier cleanup and reintroduces a reference to an internal-only file in a published workflow.

The --locked framing is inverted. main publishes with cargo publish -p pagedb --allow-dirty and no --locked, so the lockfile-desync failure the description presents as observable does not occur today. This PR introduces --locked, then adds a transactional stamper and a 143-line suite to satisfy that new requirement. Locked publishing is a reasonable goal, but it is a separate change and should be argued on its own terms.

scripts/test-release-workflow-contract.sh asserts that workflow files contain particular literal strings. It verifies no behaviour and fails on any legitimate refactor. It is also wired into the lint job, so release-workflow greps would gate every ordinary pull request.

Action SHA pinning is a legitimate practice and the pins are correctly annotated with their tags. It is not urgent here, it is a standing maintenance commitment that wants Dependabot or Renovate alongside it, and it does not belong bundled with 900 lines of unrelated machinery. Happy to take it as a standalone change.

If this is worth revisiting, the shape is a small commit carrying the archive-name assertion and the tag-commit check, and a separate one for pinning.

@presempathy-awb

Copy link
Copy Markdown
Contributor Author

Reworked into the two requested independent lanes:

  • ci(release): verify selected prepare artifacts #25 keeps only the selected-run tag-commit check and exact-five-archive assertion. Crates.io remains independent; there is no provenance JSON, shared prepare gate, locked publish, stamping rewrite, workflow-layout grep test, Cargo.lock edit, or release runbook. It is 19/19 green and the verifier also passed against the real retained v0.1.0-beta.3 prepare run and its five artifacts.
  • ci: pin GitHub Actions and enable updates #26 pins every third-party workflow action to a full commit SHA and adds weekly Dependabot maintenance with a seven-day cooldown. It is independently 19/19 green.

The closed branch remains untouched as review history; neither replacement depends on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants