test(e2e): make Playwright browser/UI testing a mandatory CI gate

[mastermanas805]

Why

A dashboard change could previously ship having silently broken login, claim, the dashboard, or the payments/upgrade flow — the Playwright suite covered none of those money-critical user paths. This makes browser/UI testing a hard, hermetic CI gate so a UI regression fails before it can merge.

Coverage gaps found & closed

Audited the existing 23-test Playwright suite against the Chrome-MCP CHROME-MCP-TEST-PLAN-2026-05-19.md (S1–S8). Critical gaps: S5 Payments had zero automation (and the Chrome-MCP S5 run is itself blocked in prod — Razorpay isn't recurring-enabled), S3 Claim had zero coverage, and S4 had no 429 / deployment-count-consistency checks.

New specs (e2e/):

• upgrade-journey.spec.ts (S5, headline) — checkout success → Razorpay redirect; checkout failure (500 / 503 billing_not_configured / 409 already_on_plan); Change-plan modal (immediate / short_url / already_on_plan / 5xx → support-fallback); invoice rendering with null/pending/unknown-status rows (the S5-F3 $NaN / Invalid Date regression); Annual toggle; no-self-serve-cancel policy.
• claim-flow.spec.ts (S3) — preview, single-use 409 replay, post-claim payment funnel + countdown, expired token, funnel→Razorpay CTA.
• dashboard-trust.spec.ts (S4/S8) — dashboard render, empty state, the 429 retry-hint UI, and Overview-tile ⇄ Billing-panel deployment-count consistency (the S5-F4 drift).
• auth-guards.spec.ts (S2) — every /app/* route bounces an unauthed visitor to /login; the checkout deep-link intent survives the login bounce.

e2e/fixtures.ts gains billing fixtures + installBillingAPIFake / mockCheckout* / mockChangePlan helpers.

Hermetic — creates nothing

All API + Razorpay traffic is page.route()-mocked (VITE_NO_PROXY=1, per CLAUDE.md convention 10). The suite is deterministic, creates nothing on any backend, and needs no teardown. The Razorpay short_url is a fake and is itself intercepted — the browser never loads a real checkout page. No test hits a real backend; if one ever did it would have to teardown.

CI gate: advisory → blocking

• The playwright job is renamed to playwright (mandatory UI gate) and runs npm run test:e2e:ci (the exact local equivalent — new script).
• Uploads the HTML report + traces as an artifact on failure so a red gate is debuggable.
• Operator action: for the gate to block merges, add playwright (mandatory UI gate) to the branch-protection rule for main (Settings → Branches → Require status checks). The workflow makes the check available; the rule makes it required.

Suites: Playwright vs Chrome-MCP

CHROME-MCP-TEST-PLAN-2026-05-19.md is updated to a living two-layer map — Playwright = the automated must-pass gate, Chrome-MCP = the manual exploratory layer for what Playwright can't do hermetically (real Razorpay/OTP/decline, real provisioning, Lighthouse).

Gate output

npx tsc --noEmit clean · npm run build (incl. prerender) OK · npx vitest run 662 pass · npx playwright test --project=chromium 57 pass (was 23).

🤖 Generated with Claude Code

[mastermanas805]

Rebase attempt 2026-05-20 hit conflicts. main already has .github/workflows/ci.yml + deploy-pages.yml. Whether the mandatory UI gate added by this PR is still wanted (vs. today's heavier integration-test layer) needs an operator decision.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

[mastermanas805]

Status check 2026-05-20 — still adds value

Audit against current main (61a89cd):

Main's e2e/ has 8 specs / 31 tests: admin-customers, auth, navigation, pause-resume, persona-bugs, resources, settings-pat, vault. None of them cover:

• S2 auth-guard route enumeration (every /app/* bounces unauthed → login + checkout-intent survival)
• S3 claim flow (preview, single-use 409, payment funnel, expired token)
• S4 dashboard trust (render, empty state, 429 retry hint, Overview ⇄ Billing deployment-count consistency)
• S5 upgrade journey / billing (checkout success/failure/already-on-plan, change-plan modal, invoice null/pending/unknown-status rows — the $NaN / Invalid Date regression)

Main's .github/workflows/ci.yml runs Playwright but the job is unnamed (just playwright) and uploads no artifact on failure. This PR renames it to playwright (mandatory UI gate), switches to npm run test:e2e:ci (matches local), and uploads HTML report + traces.

Not replaced by INTEGRATION-TESTS-2026-05-20.md — that work covers api/worker/provisioner Go integration (backup/restore, Brevo webhook, propagation runner, deep /readyz). Disjoint from UI testing.

Mergeability: MERGEABLE / CLEAN. CI green (build-and-test, playwright (mandatory UI gate) both SUCCESS as of 2026-05-20 17:01 UTC).

Branch-protection operator action (carry-over from PR body): for the playwright (mandatory UI gate) check to block merge, it must be added to the branch-protection rule for main (Settings → Branches → Require status checks). The workflow makes the check available; the rule makes it required.

Recommendation: merge. PR holds up against current main; coverage 23 → 57 tests is exactly the money-critical UI surface not otherwise gated.

[mastermanas805]

CI is green. Cannot auto-merge via this session — the PR touches .github/workflows/ci.yml which requires workflow OAuth scope (current token has repo only). Manual merge needed: the 4 new e2e spec files (auth-guards, claim-flow, dashboard-trust, upgrade-journey) are real new coverage and should land. The workflow change is a cosmetic rename of the playwright job to 'mandatory UI gate' + artifact upload.