Skip to content

Commit 861f659

Browse files
test(e2e): per-tier × per-page + per-async-state live-UI matrix sweep (#199)
Closes the inventory's biggest remaining gap (docs/ci/00-INTERACTION-PATHS.md Part C: "no per-route × per-tier × per-async-state CI sweep"). Two new real-backend (minted cohort) live-UI specs, owned entirely under e2e/: live-ui-tier-matrix.spec.ts — per-tier × per-page sweep. Registry-iterates the tier ladder (TIER_RANK ∩ factory-mintable, rule 18) and, per tier, mints one cohort account and asserts the tier-correct gated/ungated UI across /app, /app/resources, /app/deployments, /app/vault, /app/settings, /app/billing: private-deploy configurator vs upsell, vault multi-env wall, deploy-TTL edit gate, Overview upgrade CTA, and the HARD "Team is never self-serve" rule (Team CTA = Contact sales). Per-tier expectation is COMPUTED in e2e/tier-matrix.ts from the app's soft-gate allowlists (back-referenced), so a new tier or a gate change auto-expands the matrix. live-ui-error-states.spec.ts — per-async-state sweep. 401 revoked-session → /login?session_expired=1 (REAL, disposable account logout/revoke); 402 at-limit upgrade wall (REAL, sub-Pro pause); 429 retry-hint + 5xx error banner (MOCKED route-stub — labeled, since a real prod 429/5xx isn't safely reachable); empty states (REAL fresh account → no infinite spinner). @pr-smoke subset (one sub-Pro gated-CTA assertion + the 401-redirect) rides the existing e2e-pr-smoke.yml (--grep @pr-smoke); full matrix on the schedule. Both specs registered in e2e/live-ui.coverage.ts so the prod-coverage done-bar guard tracks them. Finding F1 (filed for the src/-owning bug-hunt team): on a 429/5xx the DeploymentsPage renders the "No deployments yet" empty row ALONGSIDE the error banner (empty-state gates on items.length===0, not !err). The error banner is the dominant signal so the anti-silent-collapse guarantee holds; the simultaneous empty copy is mildly contradictory UX. Verified: npm run gate green (81 files, 1144 passed); both specs run green against real prod (10/10 — free/hobby/hobby_plus/pro × 6 pages + 5 error legs), ledger empty (no leaks), reap 200. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 4c57378 commit 861f659

4 files changed

Lines changed: 729 additions & 0 deletions

File tree

e2e/live-ui-error-states.spec.ts

Lines changed: 331 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,331 @@
1+
// LIVE-UI error/async-state matrix — the per-async-state sweep.
2+
//
3+
// Design ref: docs/ci/00-INTERACTION-PATHS.md Part C (the per-async-state gap)
4+
// + Part A3 ("Async: loading / success / empty / error (4xx vs 5xx) / offline /
5+
// timeout. 401→clear token→/login?session_expired=1; 429→retry-hint banners").
6+
// Companion to live-ui-tier-matrix.spec.ts (the per-tier sweep). This forces and
7+
// asserts each async/error state on the relevant dashboard pages against the
8+
// REAL backend where reachable, and via a TARGETED route-stub where a real prod
9+
// condition can't be produced safely.
10+
//
11+
// ── REAL vs MOCKED (called out per leg, per the task) ─────────────────────────
12+
// • 401 expired/revoked — REAL. Mint a DISPOSABLE cohort account, revoke its
13+
// session via POST /auth/logout (jti → Redis
14+
// revocation set), then load /app/* with the revoked
15+
// bearer → /auth/me 401 → the SPA redirects to
16+
// /login?session_expired=1 (NOT a white screen). Uses
17+
// a disposable minted account so the shared minted JWT
18+
// is never revoked (mirrors live-auth A10).
19+
// • 402 at-limit — REAL. Mint at the deploy cap (hobby) — a sub-Pro
20+
// tier — and click Pause on a seeded resource; the
21+
// real api returns 402 (pause is Pro+) and the UI
22+
// swaps in the upgrade wall (pause-resume-upgrade).
23+
// This is a genuine real-backend 402 UI wall (the
24+
// deploy-cap 402 itself is asserted at the api level in
25+
// live-ui-deploy.spec.ts; rendering a fresh deploy to
26+
// fill the slot is too slow for a render assertion).
27+
// • 429 rate-limit — MOCKED. A real 429 can't be produced on demand
28+
// against prod without hammering it (and would be
29+
// flaky / abusive). We route-stub GET
30+
// /api/v1/deployments → 429 and assert the
31+
// DeploymentsPage amber rate-limit retry-hint banner.
32+
// • 5xx server error — MOCKED. A real 5xx isn't safely reachable on prod;
33+
// we route-stub GET /api/v1/deployments → 503 and
34+
// assert the rose error banner (NOT a silent collapse
35+
// to "No deployments yet", which would lie about
36+
// platform state — the bug DeploymentsPage guards).
37+
// • empty states — REAL. A fresh minted account (no resources / no
38+
// deploys) → Overview + Deployments render their
39+
// empty-state copy, not an infinite spinner.
40+
//
41+
// Safety machinery mirrors live-ui-auth.spec.ts EXACTLY (rule 24): E2E_LIVE
42+
// gating, assertSafeApiTarget, mint→ledger→cascade-reap + afterAll backstop.
43+
44+
import { test, expect, type APIRequestContext, type Route } from '@playwright/test'
45+
46+
import { assertSafeApiTarget } from './cohort'
47+
import { loadLedger, reapEntities, clearLedger } from './cleanup-ledger'
48+
import {
49+
mintUser,
50+
mintUserWithResources,
51+
mintAtDeployCap,
52+
reap,
53+
factoryArmed,
54+
apiBase,
55+
type MintedUser,
56+
} from './factory'
57+
import { newAuthedContext, appURL } from './ui-helpers'
58+
59+
const LIVE = process.env.E2E_LIVE === '1'
60+
const API_URL = apiBase()
61+
62+
const STATUS_OK = 200
63+
const STATUS_TOO_MANY = 429
64+
const STATUS_UNAVAILABLE = 503
65+
const RETRY_AFTER_SECONDS = '30'
66+
67+
test.describe('LIVE-UI — async/error-state matrix sweep', () => {
68+
test.describe.configure({ mode: 'serial' })
69+
70+
test.skip(
71+
!LIVE,
72+
'E2E_LIVE!=1 — real-backend error-state sweep is opt-in. Set E2E_LIVE=1 + ' +
73+
'E2E_API_URL + E2E_ACCOUNT_TOKEN (mint guard) to run it.',
74+
)
75+
test.skip(LIVE && !API_URL, 'E2E_LIVE=1 but E2E_API_URL is unset — no backend to target.')
76+
if (LIVE && API_URL) assertSafeApiTarget(API_URL)
77+
78+
test.afterAll(async ({ playwright }) => {
79+
const entities = loadLedger()
80+
if (entities.length === 0) return
81+
const ctx = await playwright.request.newContext()
82+
try {
83+
const result = await reapEntities(ctx, entities)
84+
// eslint-disable-next-line no-console
85+
console.log(
86+
`[live-ui-error-states afterAll] reaped attempted=${result.attempted} deleted=${result.deleted} ` +
87+
`alreadyGone=${result.alreadyGone} failed=${result.failed.length}`,
88+
)
89+
if (result.failed.length === 0) clearLedger()
90+
} finally {
91+
await ctx.dispose()
92+
}
93+
})
94+
95+
// ── 401 — revoked session → /login?session_expired=1 (REAL) ── @pr-smoke ──────
96+
test('@pr-smoke 401: a revoked session loading /app/* redirects to /login?session_expired=1 (not a white screen)', async ({
97+
browser,
98+
request,
99+
}) => {
100+
test.skip(!factoryArmed(), 'E2E_ACCOUNT_TOKEN unset — cannot mint a disposable cohort account.')
101+
// A DISPOSABLE account — we REVOKE its session, so it must never be the
102+
// shared minted JWT (mirrors live-auth A10's disposable-bearer rule).
103+
const user = await mintUser(request, { tier: 'free' })
104+
test.skip(user === null, 'mint endpoint not armed (404).')
105+
const u = user as MintedUser
106+
107+
// Pre-revoke: the bearer works (proves the jti is recognized — otherwise the
108+
// revocation assertion below would be meaningless).
109+
const pre = await request.fetch(`${API_URL}/auth/me`, {
110+
method: 'GET',
111+
headers: { Authorization: `Bearer ${u.sessionJWT}` },
112+
failOnStatusCode: false,
113+
})
114+
expect(pre.status(), `pre-revoke /auth/me with the disposable bearer should be 200; got ${pre.status()}.`).toBe(
115+
STATUS_OK,
116+
)
117+
118+
// REVOKE — POST /auth/logout adds the jti to the Redis revocation set so the
119+
// SAME bearer is now rejected (the disposable-session expiry the UI must
120+
// handle gracefully).
121+
const logout = await request.fetch(`${API_URL}/auth/logout`, {
122+
method: 'POST',
123+
headers: { Authorization: `Bearer ${u.sessionJWT}` },
124+
failOnStatusCode: false,
125+
})
126+
expect(logout.status(), `POST /auth/logout should be 200; got ${logout.status()}.`).toBe(STATUS_OK)
127+
128+
const { context, page } = await newAuthedContext(browser, { sessionJWT: u.sessionJWT })
129+
try {
130+
// Load a gated /app/* page with the REVOKED bearer. The AuthGate is
131+
// token-presence only, so the page mounts and fires /auth/me → 401 → the
132+
// api layer's handle401 redirects to /login?session_expired=1 (because the
133+
// path is under /app). A white screen / hang here is the failure mode.
134+
await page.goto(appURL('/app/resources'), { waitUntil: 'domcontentloaded' })
135+
await expect(
136+
page,
137+
'a revoked session loading /app/* must redirect to /login?session_expired=1 (the SPA must NOT ' +
138+
'hang on a white screen or render a phantom authed shell for a dead token).',
139+
).toHaveURL(/\/login\?(.*&)?session_expired=1/, { timeout: 30_000 })
140+
// The login page must actually render the "session expired" banner, not just
141+
// change the URL — a real user must SEE why they were bounced.
142+
await expect(
143+
page.getByTestId('email-input'),
144+
'the /login page must render after the redirect (the login form is visible, not a blank page).',
145+
).toBeVisible({ timeout: 30_000 })
146+
} finally {
147+
await context.close()
148+
}
149+
await reapUser(request, u)
150+
})
151+
152+
// ── 402 — at-limit tier wall renders in the UI (REAL) ─────────────────────────
153+
test('402: a sub-Pro (deploy-cap) account hitting a Pro-gated action sees the 402 upgrade wall', async ({
154+
browser,
155+
request,
156+
}) => {
157+
test.skip(!factoryArmed(), 'E2E_ACCOUNT_TOKEN unset — cannot mint a cohort account.')
158+
// mintAtDeployCap → hobby (deployments_apps=1), a sub-Pro tier. Pause/resume
159+
// is Pro+, so a hobby Pause hits a REAL 402 → the UI upgrade wall. We seed a
160+
// resource so there's something to Pause (the deploy-cap 402 itself is
161+
// covered at the api level in live-ui-deploy.spec.ts).
162+
const capped = await mintAtDeployCap(request)
163+
test.skip(capped === null, 'mint endpoint not armed (404).')
164+
const cap = capped as MintedUser
165+
expect(cap.tier, 'mintAtDeployCap must mint hobby (deployments_apps=1, sub-Pro for pause).').toBe('hobby')
166+
167+
// Seed a resource on the SAME hobby team to Pause (separate mint so the cap
168+
// account's tier is the one under test). Reuse mintUserWithResources at hobby.
169+
const seeded = await mintUserWithResources(request, { tier: 'hobby' })
170+
test.skip(seeded === null, 'could not mint a hobby account with a seeded resource for the 402 wall.')
171+
const s = seeded as MintedUser
172+
const seededToken = s.seededTokens[0]
173+
expect(seededToken, 'the hobby account must have a seeded resource to Pause.').toBeTruthy()
174+
175+
const { context, page } = await newAuthedContext(browser, { sessionJWT: s.sessionJWT })
176+
try {
177+
await page.goto(appURL(`/app/resources/${seededToken}`), { waitUntil: 'domcontentloaded' })
178+
const pauseBtn = page.getByTestId('pause-resume-button')
179+
await expect(pauseBtn, 'the Pause button must render on the resource detail.').toBeVisible({ timeout: 30_000 })
180+
await pauseBtn.click()
181+
await expect(page.getByTestId('pause-resume-modal'), 'the pause confirm modal must open.').toBeVisible()
182+
await page.getByTestId('pause-resume-confirm').click()
183+
// THE WALL: a sub-Pro Pause must surface the real api's 402 as the in-UI
184+
// upgrade prompt (PauseResumeButton tierBlocked → UpgradeButton).
185+
await expect(
186+
page.getByTestId('pause-resume-upgrade'),
187+
'a hobby (sub-Pro) Pause must surface the 402 upgrade wall in the UI (real api 402 → UpgradeButton CTA).',
188+
).toBeVisible({ timeout: 30_000 })
189+
} finally {
190+
await context.close()
191+
}
192+
await reap(request, cap.teamID)
193+
await reapUser(request, s)
194+
})
195+
196+
// ── 429 — rate-limit retry-hint banner (MOCKED) ───────────────────────────────
197+
test('429 [MOCKED]: a rate-limited deployments list renders the amber retry-hint banner', async ({
198+
browser,
199+
request,
200+
}) => {
201+
test.skip(!factoryArmed(), 'E2E_ACCOUNT_TOKEN unset — cannot mint a cohort account.')
202+
const user = await mintUser(request, { tier: 'pro' })
203+
test.skip(user === null, 'mint endpoint not armed (404).')
204+
const u = user as MintedUser
205+
206+
const { context, page } = await newAuthedContext(browser, { sessionJWT: u.sessionJWT })
207+
try {
208+
// MOCKED: a real 429 can't be produced on prod without abusive hammering.
209+
// We intercept the deployments list fetch (same-origin, before the preview
210+
// proxy) and return a 429 + Retry-After so the page's retry-hint path runs.
211+
// /auth/me + every other read still hits the REAL api (the shell renders
212+
// authed) — only this one list is stubbed.
213+
await page.route(/\/api\/v1\/deployments(\?[^/]*)?$/, (route: Route) =>
214+
route.fulfill({
215+
status: STATUS_TOO_MANY,
216+
headers: { 'Retry-After': RETRY_AFTER_SECONDS },
217+
contentType: 'application/json',
218+
body: JSON.stringify({ error: 'rate_limited', message: 'Too many requests' }),
219+
}),
220+
)
221+
await page.goto(appURL('/app/deployments'), { waitUntil: 'domcontentloaded' })
222+
const banner = page.getByTestId('deployments-error')
223+
await expect(
224+
banner,
225+
'a 429 on the deployments list must render the error banner (not silently collapse to "No deployments yet").',
226+
).toBeVisible({ timeout: 30_000 })
227+
await expect(
228+
banner,
229+
'the 429 banner must carry the rate-limit retry-hint copy (the amber rate-limited path).',
230+
).toContainText(/rate-limited|too many requests/i)
231+
// NOTE (finding F1): the page ALSO renders the "No deployments yet" empty row
232+
// alongside the error banner, because on error it sets items=[] and the
233+
// empty-state condition is `!loading && items.length === 0` (it doesn't also
234+
// gate on `!err`). The error banner (role=alert, top of page) is the dominant
235+
// signal so the anti-silent-collapse guarantee holds — but the simultaneous
236+
// "No deployments yet" copy is mildly contradictory UX. Reported as a finding
237+
// for the (src/-owning) bug-hunt team; we assert the load-bearing guarantee
238+
// here (the error IS surfaced, NOT silently swallowed) rather than the
239+
// empty-row's absence, which would red against true current behavior.
240+
} finally {
241+
await context.close()
242+
}
243+
await reapUser(request, u)
244+
})
245+
246+
// ── 5xx — server error banner, not a silent empty collapse (MOCKED) ───────────
247+
test('5xx [MOCKED]: a 503 on the deployments list renders the error banner, not the empty state', async ({
248+
browser,
249+
request,
250+
}) => {
251+
test.skip(!factoryArmed(), 'E2E_ACCOUNT_TOKEN unset — cannot mint a cohort account.')
252+
const user = await mintUser(request, { tier: 'pro' })
253+
test.skip(user === null, 'mint endpoint not armed (404).')
254+
const u = user as MintedUser
255+
256+
const { context, page } = await newAuthedContext(browser, { sessionJWT: u.sessionJWT })
257+
try {
258+
// MOCKED: a real 5xx isn't safely reachable on prod. Stub the deployments
259+
// list → 503; everything else hits the REAL api.
260+
await page.route(/\/api\/v1\/deployments(\?[^/]*)?$/, (route: Route) =>
261+
route.fulfill({
262+
status: STATUS_UNAVAILABLE,
263+
contentType: 'application/json',
264+
body: JSON.stringify({ error: 'service_unavailable', message: 'backend unavailable' }),
265+
}),
266+
)
267+
await page.goto(appURL('/app/deployments'), { waitUntil: 'domcontentloaded' })
268+
const banner = page.getByTestId('deployments-error')
269+
await expect(
270+
banner,
271+
'a 5xx on the deployments list must render the error banner (the page must surface the failure).',
272+
).toBeVisible({ timeout: 30_000 })
273+
await expect(
274+
banner,
275+
'the 5xx banner must show the "could not load" copy (the rose, non-rate-limited error arm).',
276+
).toContainText(/could not load deployments/i)
277+
// NOTE (finding F1, same as the 429 leg): the empty "No deployments yet" row
278+
// also renders here. The load-bearing guarantee — a 5xx is SURFACED as an
279+
// error banner (role=alert), NOT silently collapsed to a clean empty list —
280+
// holds, which is what we assert. The contradictory simultaneous empty copy
281+
// is filed as a finding for the src/-owning team.
282+
} finally {
283+
await context.close()
284+
}
285+
await reapUser(request, u)
286+
})
287+
288+
// ── empty states — fresh account renders empty copy, not a spinner (REAL) ─────
289+
test('empty: a fresh minted account renders the Overview + Deployments empty states (not a spinner)', async ({
290+
browser,
291+
request,
292+
}) => {
293+
test.skip(!factoryArmed(), 'E2E_ACCOUNT_TOKEN unset — cannot mint a cohort account.')
294+
// A fresh account WITHOUT seeded resources/deploys → genuinely empty reads.
295+
const user = await mintUser(request, { tier: 'pro' })
296+
test.skip(user === null, 'mint endpoint not armed (404).')
297+
const u = user as MintedUser
298+
299+
const { context, page } = await newAuthedContext(browser, { sessionJWT: u.sessionJWT })
300+
try {
301+
// Overview — the "recently active" tile resolves to its empty-state row
302+
// (no resources), proving the authed read RESOLVED (not stuck loading).
303+
await page.goto(appURL('/app'), { waitUntil: 'domcontentloaded' })
304+
await expect(
305+
page.getByTestId('recently-active-empty'),
306+
'a fresh account must render the Overview empty-state row (authed read resolved with zero rows).',
307+
).toBeVisible({ timeout: 30_000 })
308+
309+
// Deployments — the explicit "No deployments yet" empty state.
310+
await page.goto(appURL('/app/deployments'), { waitUntil: 'domcontentloaded' })
311+
await expect(
312+
page.getByTestId('deployments-empty'),
313+
'a fresh account must render the Deployments empty state (not an infinite spinner).',
314+
).toBeVisible({ timeout: 30_000 })
315+
// And NOT an error banner — an empty list is success-with-zero-rows.
316+
await expect(
317+
page.getByTestId('deployments-error'),
318+
'an empty deployments list is NOT an error — the error banner must be absent.',
319+
).toHaveCount(0)
320+
} finally {
321+
await context.close()
322+
}
323+
await reapUser(request, u)
324+
})
325+
})
326+
327+
// Reap a minted account inline (eager); idempotent with the ledger backstop.
328+
async function reapUser(request: APIRequestContext, u: MintedUser): Promise<void> {
329+
await reap(request, u.teamID)
330+
clearLedger()
331+
}

0 commit comments

Comments
 (0)