Skip to content

Commit 4cf37ca

Browse files
mastermanas805claude
mastermanas805
and
claude
authored
test(e2e): PROD-COVERAGE done-bar guard — every prod-feasible flow live-tested or exempt (#187)
* test(e2e): PROD-COVERAGE done-bar guard — enforce every prod-feasible flow is live-tested or exempt Final piece of "every user control/flow covered by a live PROD integration test". A network-free vitest guard that reds CI whenever a prod-feasible flow is added without either a live-prod spec or a justified exemption. - e2e/prod-coverage-manifest.ts: in-repo mirror of the PROD-COVERAGE-MATRIX — 88 `live` route-legs + ~33 `exempt` legs (each with a reason mirroring the matrix: Brevo-gated email, Razorpay charge, real-GitHub-OAuth, full-Kaniko-build-deferred, team-tier-gated, OPTIONS/CORS, static content, operator/admin, live-DNS domains, destructive team purge). - e2e/prod-coverage-donebar.test.ts: registry-iterating guard (rule 18) — (a) every `live` flow is covered by >=1 live-*.spec, (b) every `exempt` flow has a non-empty reason, (c) no flow untagged/duplicated, plus reverse-drift (no spec-covered route escapes the inventory). Failure messages name the offending flow and the fix. - e2e/live-*.coverage.ts: each live spec's covered-route manifest extracted to a playwright-free sibling the spec re-exports and the guard imports — so the guard never pulls the @playwright/test runtime into vitest. - vite.config.ts: include e2e/**/*.test.ts (vitest-only guards), still exclude e2e/*.spec.ts (playwright runtime). Guard PASSES locally (current coverage complete, all exemptions justified); npm run gate green; new files 100% covered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(e2e): pin Playwright testMatch to *.spec.ts so it skips the vitest guard The default Playwright config's implicit testMatch also globs *.test.ts, so it tried to run e2e/prod-coverage-donebar.test.ts (a vitest-only guard) under the Playwright runtime and crashed (`Cannot read properties of undefined (reading 'config')` — vitest describe/it under Playwright). Pin testMatch to *.spec.ts; the live/auth-contract/auth-roundtrip configs already use explicit per-spec testMatch and are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 33457bc commit 4cf37ca

18 files changed

Lines changed: 599 additions & 100 deletions

‎e2e/live-anon-provision.coverage.ts‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Coverage manifest for live-anon-provision.spec.ts (W2 — the six remaining
2+
// anonymous resource-provision flows + authed reap).
3+
//
4+
// The spec drives one identical test per service from the PROVISION_FLOWS
5+
// registry; this sibling lists the canonical route string each flow exercises
6+
// (matrix §1.B). Playwright-free so the vitest done-bar guard can import it
7+
// without the @playwright/test runtime. `POST /db/new` is NOT here — it lives in
8+
// live-provision-smoke.coverage.ts.
9+
//
10+
// The spec re-exports this as `coveredRoutes`.
11+
export const coveredRoutes: string[] = [
12+
'POST /vector/new',
13+
'POST /cache/new',
14+
'POST /nosql/new',
15+
'POST /queue/new',
16+
'POST /storage/new',
17+
'POST /webhook/new',
18+
'DELETE /api/v1/resources/:id',
19+
]

‎e2e/live-anon-provision.spec.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -380,3 +380,8 @@ test.describe('LIVE — every anonymous provision flow → backend-assert → re
380380
})
381381
}
382382
})
383+
384+
// Covered-route manifest (rule 18), defined in the playwright-free sibling so the
385+
// vitest prod-coverage done-bar guard can union it without the @playwright/test
386+
// runtime. One route per PROVISION_FLOWS entry + the authed reap (matrix §1.B).
387+
export { coveredRoutes } from './live-anon-provision.coverage'

‎e2e/live-auth.coverage.ts‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Coverage manifest for live-auth.spec.ts (W1 — auth/session/github/cli legs).
2+
//
3+
// live-auth.spec.ts tags its legs by matrix W1 leg-ID (A1-start, A4, …). This
4+
// sibling translates those leg-IDs into the canonical route strings the
5+
// prod-coverage matrix (§0.2 / §1.D) attributes to this spec, so the vitest
6+
// done-bar guard can union one shared route vocabulary across every live-*.spec.
7+
// Playwright-free so the guard never imports the @playwright/test runtime.
8+
//
9+
// The spec re-exports this as `coveredRoutes`.
10+
export const coveredRoutes: string[] = [
11+
'GET /auth/github/start',
12+
'GET /auth/github/callback',
13+
'OPTIONS /auth/exchange',
14+
'GET /auth/me',
15+
'POST /auth/logout',
16+
'POST /auth/cli',
17+
'GET /auth/cli/:id',
18+
'POST /auth/email/start',
19+
]

‎e2e/live-auth.spec.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -699,3 +699,9 @@ test.describe('LIVE — auth/login seams (W1: OAuth, logout-revocation, CLI, /au
699699
// that wants to assert this file's CohortEntity usage compiles against the ledger
700700
// contract without importing the spec body.
701701
export type { CohortEntity }
702+
703+
// Covered-route manifest (rule 18), defined in the playwright-free sibling so the
704+
// vitest prod-coverage done-bar guard can union it without the @playwright/test
705+
// runtime. The spec tags legs by W1 leg-ID; the sibling maps them to route
706+
// strings (matrix §0.2 / §1.D).
707+
export { coveredRoutes } from './live-auth.coverage'

‎e2e/live-claim-deploy.coverage.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
// Coverage manifest for live-claim-deploy.spec.ts (W3 — claim + env-switch +
2+
// deploy lifecycle legs).
3+
//
4+
// The spec tags its legs by matrix W3 leg-ID (claim-conversion, deploy-create-202,
5+
// …); this sibling translates them into the canonical route strings the matrix
6+
// (§0.2 / §1.C / §1.E / §1.K) attributes to this spec. Playwright-free so the
7+
// vitest done-bar guard can import it without the @playwright/test runtime.
8+
//
9+
// `deploy-build-to-live-url` is intentionally NOT here — the full Kaniko build is
10+
// PROD-EXEMPT (full-Kaniko-build-deferred); only the accepted-contract + events +
11+
// delete legs run live.
12+
//
13+
// The spec re-exports this as `coveredRoutes`.
14+
export const coveredRoutes: string[] = [
15+
'POST /cache/new',
16+
'POST /claim',
17+
'GET /api/v1/resources',
18+
'POST /deploy/new',
19+
'GET /api/v1/deployments/:id/events',
20+
'DELETE /api/v1/deployments/:id',
21+
]

‎e2e/live-claim-deploy.spec.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -747,3 +747,9 @@ function buildUstarTar(name: string, content: Buffer): Buffer {
747747

748748
// Re-export for a potential future registry-iterating coverage test (rule 18).
749749
export type { CohortEntity }
750+
751+
// Covered-route manifest (rule 18), defined in the playwright-free sibling so the
752+
// vitest prod-coverage done-bar guard can union it without the @playwright/test
753+
// runtime. The spec tags legs by W3 leg-ID; the sibling maps them to route
754+
// strings (matrix §0.2 / §1.C / §1.E / §1.K).
755+
export { coveredRoutes } from './live-claim-deploy.coverage'

‎e2e/live-provision-smoke.coverage.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
// Coverage manifest for live-provision-smoke.spec.ts (WS1-P1 — the anonymous
2+
// /db/new hot-pool provision smoke). Playwright-free sibling so the vitest
3+
// prod-coverage done-bar guard can import the covered-route set without the
4+
// @playwright/test runtime. The spec re-exports this as `coveredRoutes`.
5+
export const coveredRoutes: string[] = [
6+
'POST /db/new',
7+
'GET /api/v1/resources',
8+
]

‎e2e/live-provision-smoke.spec.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,3 +192,8 @@ test.describe('LIVE smoke — anonymous provision → backend-assert → reap',
192192
clearLedger()
193193
})
194194
})
195+
196+
// Covered-route manifest (rule 18), defined in the playwright-free sibling so the
197+
// vitest prod-coverage done-bar guard can union it without the @playwright/test
198+
// runtime (matrix §1.B — POST /db/new).
199+
export { coveredRoutes } from './live-provision-smoke.coverage'

‎e2e/live-reads.coverage.ts‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
// Coverage manifest for live-reads.spec.ts (Batch A — W-OBS/RES/VAULT/APIKEYS/
2+
// BILLING/AUDIT read legs). Extracted into a playwright-free sibling so the
3+
// vitest prod-coverage done-bar guard (e2e/prod-coverage-donebar.test.ts) can
4+
// import the covered-route set WITHOUT pulling in the @playwright/test runtime.
5+
//
6+
// The spec re-exports this as `coveredRoutes`; the guard unions it with the
7+
// other live-*.coverage.ts manifests (rule 18 — registry-iterating, no
8+
// hand-typed duplicate list).
9+
export const coveredRoutes: string[] = [
10+
// W-OBS
11+
'GET /livez',
12+
'GET /healthz',
13+
'GET /readyz',
14+
'GET /openapi.json',
15+
'GET /api/v1/capabilities',
16+
'GET /api/v1/status',
17+
'GET /.well-known/oauth-protected-resource',
18+
'GET /api/v1/incidents',
19+
'GET /llms.txt',
20+
'GET /security.txt',
21+
// W-RES
22+
'GET /api/v1/whoami',
23+
'GET /api/v1/resources',
24+
'GET /api/v1/resources/:id',
25+
'GET /api/v1/resources/:id/credentials',
26+
'GET /api/v1/resources/:id/metrics',
27+
'DELETE /api/v1/resources/:id',
28+
'GET /api/v1/resources/families',
29+
'GET /api/v1/resources/:id/family',
30+
'GET /api/v1/resources/:id/backups',
31+
'GET /api/v1/resources/:id/restores',
32+
// W-VAULT
33+
'PUT /api/v1/vault/:env/:key',
34+
'GET /api/v1/vault/:env/:key',
35+
'GET /api/v1/vault/:env',
36+
// W-APIKEYS
37+
'POST /api/v1/auth/api-keys',
38+
'GET /api/v1/auth/api-keys',
39+
'DELETE /api/v1/auth/api-keys/:id',
40+
// W-BILLING
41+
'GET /api/v1/billing',
42+
'GET /api/v1/billing/invoices',
43+
'GET /api/v1/billing/usage',
44+
'GET /api/v1/usage/wall',
45+
// W-AUDIT
46+
'GET /api/v1/audit',
47+
'GET /api/v1/audit.csv',
48+
]

‎e2e/live-reads.spec.ts‎

Lines changed: 5 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -727,39 +727,8 @@ test.describe('LIVE — Batch A read flows (W-OBS / W-RES / W-VAULT / W-APIKEYS
727727
})
728728
})
729729

730-
// Exported so a future registry-iterating prod-coverage done-bar (matrix §4
731-
// Option B) can union manifests across live-*.spec.ts without a hand-typed list.
732-
export const coveredRoutes: string[] = [
733-
'GET /livez',
734-
'GET /healthz',
735-
'GET /readyz',
736-
'GET /openapi.json',
737-
'GET /api/v1/capabilities',
738-
'GET /api/v1/status',
739-
'GET /.well-known/oauth-protected-resource',
740-
'GET /api/v1/incidents',
741-
'GET /llms.txt',
742-
'GET /security.txt',
743-
'GET /api/v1/whoami',
744-
'GET /api/v1/resources',
745-
'GET /api/v1/resources/:id',
746-
'GET /api/v1/resources/:id/credentials',
747-
'GET /api/v1/resources/:id/metrics',
748-
'DELETE /api/v1/resources/:id',
749-
'GET /api/v1/resources/families',
750-
'GET /api/v1/resources/:id/family',
751-
'GET /api/v1/resources/:id/backups',
752-
'GET /api/v1/resources/:id/restores',
753-
'PUT /api/v1/vault/:env/:key',
754-
'GET /api/v1/vault/:env/:key',
755-
'GET /api/v1/vault/:env',
756-
'POST /api/v1/auth/api-keys',
757-
'GET /api/v1/auth/api-keys',
758-
'DELETE /api/v1/auth/api-keys/:id',
759-
'GET /api/v1/billing',
760-
'GET /api/v1/billing/invoices',
761-
'GET /api/v1/billing/usage',
762-
'GET /api/v1/usage/wall',
763-
'GET /api/v1/audit',
764-
'GET /api/v1/audit.csv',
765-
]
730+
// Re-exported from the playwright-free sibling so the vitest prod-coverage
731+
// done-bar guard (e2e/prod-coverage-donebar.test.ts) can union this spec's
732+
// covered-route manifest without importing the @playwright/test runtime
733+
// (rule 18 — registry-iterating, single source of the route list).
734+
export { coveredRoutes } from './live-reads.coverage'

0 commit comments

Comments
 (0)