Skip to content

e2e-gate --status success filter hides re-run-to-green Runtime Live E2E runs from the release gate#460

Merged
clkao merged 2 commits into
mainfrom
spacedock-ensign/e2e-gate-rerun-status-filter
Jul 2, 2026
Merged

e2e-gate --status success filter hides re-run-to-green Runtime Live E2E runs from the release gate#460
clkao merged 2 commits into
mainfrom
spacedock-ensign/e2e-gate-rerun-status-filter

Conversation

@clkao

@clkao clkao commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

A Runtime Live E2E run re-run to green was invisible to the release e2e-gate, blocking the v0.23.0 cut on a genuinely green matrix.

What changed

  • Drop --status success from the e2e-gate's gh run list query
  • Add shim-driven tests exercising the real query builder
  • Pin --workflow "Runtime Live E2E" as the live-matrix binding
  • Rewrite the parked-run doc comment; document re-run-to-green in docs/releasing.md

Evidence

  • go test ./... 15/15 packages passed; -race green on both touched packages
  • Detached adversarial audit: all claim-breaking mutants caught; fail-open --workflow hole closed in cycle 1

q3

Spike Test added 2 commits July 2, 2026 10:36
At the v0.23.0 cut, gh run list --status success returned [] for a run
re-run to green minutes earlier (conclusion success, attempt 2), while
the same query without --status already saw the success -- the filter
lags conclusion right after a re-run, blocking a genuinely green cut.
The parked-run defense the flag was meant to provide already lives in
EvaluateE2EGate's predicate (empty conclusion never matches), so the
flag was redundant and only cost re-run-to-green cuts.

T1 drives the real ghRunListForCommit against a fake gh shim pinning
the gate-time observation (RED under --status, GREEN once dropped).
T2 confirms the parked-run defense survives the flag's removal.
…matrix binding

Cycle-1 audit finding: workflowName is not among ghRunListForCommit's
fetched --json fields, so the --workflow "Runtime Live E2E" argv token
is the ONLY thing binding the release gate to the live matrix. Dropping
it left the whole suite green -- any green run on the release commit
(e.g. ordinary push CI) would then satisfy the gate.

The gh shim now returns [] unless argv carries that exact --workflow
"Runtime Live E2E" pair, so the token's removal fails the gate closed
in the test. Verified locally: removing the token from
ghRunListForCommit flips TestE2EGateCommandPassesOnRerunToGreenLiveRun
RED (exit 1, "no conclusion:success..."); reverting restores GREEN.
@clkao
clkao merged commit a88d4f6 into main Jul 2, 2026
4 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant