ci: E2E suite Phase 1 — reporting + tagging

[EnderOfWorlds007]

Summary

Phase 1 of the E2E suite redesign. Today's 27 E2E tests are unchanged — better-instrumented around. Spec: docs-internal/2026-05-02-e2e-test-suite-design.md.

What changed

• E2E Report job posts a sticky PR comment with a per-cell summary table, failures detail block, and a Sentry traces link. Marker <!-- e2e-pr-report --> ensures the comment updates in place across pushes (verified during PR development — count stays at 1).
• nick-fields/retry@v3 wraps the test invocation (max 2 attempts, 30s wait, 25-min per-attempt budget). Job timeout raised to 55 min so the retry budget can actually fit. Catches Paseo testnet flakes.
• DOT_TAG=e2e-ci-{pr|nightly|dispatch} propagated to the test run; CLI's existing cli.tag root-span attribute (src/telemetry-config.ts:191) picks it up automatically.
• JUnit XML (e2e-reports/junit.xml) + forensic dot-runs.log uploaded as workflow artefacts (7-day retention).
• src/index.test.ts — source-level guard against the documented Bun SEA stdin warm-up regression. Negative-lookbehind regex distinguishes the live call at src/index.ts:87 from a // line comment that mentions it.
• Auto-issue on schedule/release failure (no de-dup by design — cadence of new issues is the signal).
• Sticky-PR-comment + auto-issue verified end-to-end before merge: a temporary simulate_failure workflow_dispatch input was added, fired once to produce real issue Nightly E2E failure: 2026-05-02 #83 (closed), then removed in a separate commit so the verification path doesn't ship.

Out of scope (future phases)

• Per-cell SIGNER derivation, per-cell domain registrations (Phase 3)
• Matrix split into per-cell jobs with max-parallel: 1 on publish cells (Phase 4)
• Hardhat/multi-contract/modable/chaos cells (Phase 5–6)
• Published-artefact surface + release: prereleased trigger (Phase 7)
• Branch protection / required status checks — currently main has no protection rule at all; flipping to gating is a separate post-Phase-4 decision (see spec §7f for the one-shot gh api snippet)
• Sentry dashboard creation for E2E health (manual one-time bootstrap, spec §9a)

Phase 1 + 2 fold note

The spec listed DOT_TAG propagation + Sentry traces link as Phase 2; this PR folds them into Phase 1 because the Sentry link is useless until tags are set, and both touch the same workflow file in tightly-coupled ways.

Test plan

• Local: pnpm test — all unit tests pass (439, including new src/index.test.ts)
• Local: pnpm exec vitest run e2e/cli/install.test.ts — JUnit XML produced, dot-runs.log populated
• CI: sticky PR comment posts on first push, updates in place on subsequent pushes (count stayed at 1 across 4+ pushes during PR dev)
• CI: deliberate test failure renders correctly in the failures detail block (verified by injecting a regex-based failure during PR dev, then reverting)
• CI: simulate_failure dispatch path opened real issue Nightly E2E failure: 2026-05-02 #83 (closed); scaffolding removed in b106f66
• CI: if: always() on report job correctly renders even when test cancels (verified — see runs 25260814880 and 25260866303 where test was cancelled by concurrency but report succeeded)
• First post-merge nightly (cron 06:00 UTC) — will produce the first real auto-issue if testnet flakes; auto-issue path verified pre-merge but real-cron path is exercised first time then

Related

• Spec: docs-internal/2026-05-02-e2e-test-suite-design.md (Phase 1 of 8)
• Plan: docs-internal/2026-05-02-e2e-phase-1-plan.md
• Reference patterns: docs-internal/e2e-testing-spec.md §M1–M9

[github-actions]

Dev build ready — try this branch:

curl -fsSL https://raw.githubusercontent.com/paritytech/playground-cli/main/install.sh | VERSION=dev/ci/e2e-phase-1-reporting bash

[github-actions]

E2E Test Pass · ✅ PASS

Tag: e2e-ci-pr · Branch: ci/e2e-phase-1-reporting · Commit: b1b0fe7 · Run logs

Cell	Result	Time
current	✅ PASS	12m49s

Sentry traces: view spans for this run