Skip to content

Commit 295900e

Browse files
mastermanas805claude
mastermanas805
and
claude
authored
ci(e2e): prod LIVE E2E via on-the-fly minted cohort account (#174)
Add the production E2E CI workflow that mints an ephemeral, cohort-scoped test account against prod, runs the real-backend live specs with it, and reaps the account + any spec-created resources. - .github/workflows/e2e-prod.yml: workflow_dispatch + schedule (every 30m) + repository_dispatch(e2e-prod-from-deploy). No-ops cleanly when secrets.E2E_ACCOUNT_TOKEN is empty. MINT (POST /internal/e2e/account, X-E2E-Token, {"tier":"pro"}) -> mask+export session_jwt/team_id -> RUN (E2E_LIVE=1, E2E_API_URL=prod, E2E_SESSION_JWT) -> REAP (always: DELETE the account + npm run reap:live; reaper exits non-zero on leak). Prod sibling of e2e-live.yml (staging); that file is kept. - e2e/cohort.ts: mintedSession() surfaces the workflow-minted account (E2E_SESSION_JWT + companion identity env) so authed legs use a real cohort account instead of self-minting from E2E_JWT_SECRET. assertSafeApiTarget() relaxes the prod-refusal: prod is ALLOWED only for a sanctioned minted-account run (E2E_ACCOUNT_TOKEN/E2E_SESSION_JWT present), still REFUSED otherwise so a stray run can't hammer prod. - live-auth A8/A10 prefer mintedSession() when set (assert minted email/tier; reap only spec-created resources, account reaped by the workflow); anon legs unchanged. All four live specs now call assertSafeApiTarget() at module load. npm run gate green (build + 1115 vitest pass / 3 skip). Workflow no-ops until the api mint endpoint deploys + E2E_ACCOUNT_TOKEN secret is set. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 7a33cd7 commit 295900e

6 files changed

Lines changed: 390 additions & 42 deletions

File tree

‎.github/workflows/e2e-prod.yml‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
# Real-backend (LIVE) E2E against PRODUCTION (api.instanode.dev) using an
2+
# ephemeral, cohort-scoped account minted on the fly. This is the prod sibling
3+
# of e2e-live.yml (which targets STAGING) — DO NOT delete that one.
4+
#
5+
# Plan: docs/sessions/2026-06-04 TEST-ACCOUNTS-AND-NR-SYNTHETICS-PLAN.md.
6+
#
7+
# WHY this is safe to run against prod (and e2e-live.yml is not):
8+
# - The api guards a mint endpoint (PR #260) that creates an account with
9+
# is_test_cohort=true. The live worker skip-guards neuter
10+
# billing/churn/email/quota for that team, so a LIVE run can never charge a
11+
# card, burn a real quota budget, send a "we miss you" email, or churn a
12+
# real customer.
13+
# - The account + every resource it creates is reaped: this job DELETEs the
14+
# minted account (DELETE /internal/e2e/account/{team_id}) AND runs the
15+
# per-run ledger reaper (npm run reap:live) in an `if: always()` teardown.
16+
# The reaper exits non-zero on any leak, failing the job loudly (rule 24).
17+
# - cohort.ts assertSafeApiTarget() ALLOWS a prod E2E_API_URL only when a mint
18+
# token / minted session is present (a sanctioned run); an un-tokened prod
19+
# target is still refused, so a stray invocation can never hammer prod.
20+
#
21+
# HOW it mints/runs/reaps:
22+
# 1. MINT — POST https://api.instanode.dev/internal/e2e/account with header
23+
# X-E2E-Token: $E2E_ACCOUNT_TOKEN and body {"tier":"pro"} →
24+
# {team_id, user_id, email, tier, session_jwt, expires_at}. The
25+
# session_jwt + team_id are masked and exported to later steps.
26+
# 2. RUN — E2E_LIVE=1 E2E_API_URL=https://api.instanode.dev
27+
# E2E_SESSION_JWT=<minted> npx playwright test
28+
# --config=playwright.live.config.ts. The authed legs use the
29+
# minted account (cohort.ts mintedSession()); anon legs run as-is.
30+
# 3. REAP — (always) DELETE the minted account, then npm run reap:live to
31+
# sweep any spec-created resources from the on-disk ledger.
32+
#
33+
# Triggers:
34+
# - workflow_dispatch (operator on demand).
35+
# - schedule every 30 min (continuous prod integration signal).
36+
# - repository_dispatch type `e2e-prod-from-deploy` (post-deploy hook the api
37+
# repo can fire after a prod rollout).
38+
#
39+
# Guard: if secrets.E2E_ACCOUNT_TOKEN is empty (not yet configured) the job
40+
# no-ops cleanly with a ::notice:: — it NEVER reds when unconfigured. The
41+
# workflow ships before the secret exists and goes green only once the operator
42+
# sets E2E_ACCOUNT_TOKEN and the api mint endpoint is deployed.
43+
44+
name: E2E LIVE (prod, minted account)
45+
46+
on:
47+
workflow_dispatch: {}
48+
schedule:
49+
# Every 30 minutes — continuous prod integration signal.
50+
- cron: '*/30 * * * *'
51+
repository_dispatch:
52+
types: [e2e-prod-from-deploy]
53+
54+
concurrency:
55+
# One prod LIVE run at a time: they mint a real cohort account + create real
56+
# resources; overlapping runs could interleave ledger writes / dedup state.
57+
group: e2e-prod-${{ github.workflow }}
58+
cancel-in-progress: false
59+
60+
permissions:
61+
contents: read
62+
63+
jobs:
64+
e2e-prod:
65+
name: LIVE against prod via minted account + reap
66+
runs-on: ubuntu-latest
67+
timeout-minutes: 15
68+
env:
69+
# Fixed prod target — this workflow is prod-only by design.
70+
E2E_API_URL: https://api.instanode.dev
71+
E2E_LIVE_RUN_ID: ${{ github.run_id }}
72+
# The mint-endpoint guard token. Empty until the operator configures it →
73+
# the gate step below no-ops the job cleanly.
74+
E2E_ACCOUNT_TOKEN: ${{ secrets.E2E_ACCOUNT_TOKEN }}
75+
steps:
76+
- name: Gate on configured mint token
77+
# No token configured → no-op cleanly (never a false red). Sets RUN=0
78+
# so every later step is skipped.
79+
run: |
80+
set -euo pipefail
81+
if [ -z "${E2E_ACCOUNT_TOKEN:-}" ]; then
82+
echo "::notice::secrets.E2E_ACCOUNT_TOKEN not configured — skipping prod LIVE E2E (no-op)."
83+
echo "RUN=0" >> "$GITHUB_ENV"
84+
else
85+
echo "RUN=1" >> "$GITHUB_ENV"
86+
fi
87+
88+
- uses: actions/checkout@v6
89+
if: env.RUN == '1'
90+
91+
- uses: actions/setup-node@v6
92+
if: env.RUN == '1'
93+
with:
94+
node-version: '22'
95+
cache: 'npm'
96+
97+
- name: Install deps
98+
if: env.RUN == '1'
99+
run: npm ci
100+
101+
- name: Install Chromium
102+
if: env.RUN == '1'
103+
run: npx playwright install --with-deps chromium
104+
105+
- name: Mint ephemeral cohort account
106+
id: mint
107+
if: env.RUN == '1'
108+
# POST the guarded mint endpoint → capture session_jwt + team_id, mask
109+
# them, and export to later steps. Fails the job (non-2xx) so a broken
110+
# mint endpoint surfaces immediately rather than running un-authed.
111+
run: |
112+
set -euo pipefail
113+
resp="$(curl -sS -w '\n%{http_code}' \
114+
-X POST "${E2E_API_URL}/internal/e2e/account" \
115+
-H "X-E2E-Token: ${E2E_ACCOUNT_TOKEN}" \
116+
-H 'Content-Type: application/json' \
117+
-d '{"tier":"pro"}')"
118+
code="$(printf '%s' "$resp" | tail -n1)"
119+
body="$(printf '%s' "$resp" | sed '$d')"
120+
if [ "$code" != "200" ]; then
121+
echo "::error::mint endpoint returned HTTP $code (expected 200). Body: $body"
122+
exit 1
123+
fi
124+
jwt="$(printf '%s' "$body" | jq -r '.session_jwt // empty')"
125+
team="$(printf '%s' "$body" | jq -r '.team_id // empty')"
126+
email="$(printf '%s' "$body" | jq -r '.email // empty')"
127+
tier="$(printf '%s' "$body" | jq -r '.tier // empty')"
128+
if [ -z "$jwt" ] || [ -z "$team" ]; then
129+
echo "::error::mint response missing session_jwt or team_id. Body: $body"
130+
exit 1
131+
fi
132+
# Mask the secrets so they never appear in logs.
133+
echo "::add-mask::$jwt"
134+
echo "::add-mask::$team"
135+
# session_jwt + team_id are secret-ish → env only (not step outputs).
136+
# team_id is also a non-secret output for the reap step's `if`.
137+
{
138+
echo "MINTED_SESSION_JWT=$jwt"
139+
echo "MINTED_TEAM_ID=$team"
140+
echo "MINTED_EMAIL=$email"
141+
echo "MINTED_TIER=$tier"
142+
} >> "$GITHUB_ENV"
143+
echo "minted=1" >> "$GITHUB_OUTPUT"
144+
echo "Minted cohort account (tier=$tier) — session + team_id masked."
145+
146+
- name: Run LIVE E2E against prod (minted account)
147+
if: env.RUN == '1' && steps.mint.outputs.minted == '1'
148+
env:
149+
E2E_LIVE: '1'
150+
# The minted account drives the authed legs (cohort.ts mintedSession);
151+
# anon legs run as-is. assertSafeApiTarget() permits the prod target
152+
# because E2E_SESSION_JWT is present (a sanctioned run).
153+
E2E_SESSION_JWT: ${{ env.MINTED_SESSION_JWT }}
154+
E2E_TEAM_ID: ${{ env.MINTED_TEAM_ID }}
155+
E2E_ACCOUNT_EMAIL: ${{ env.MINTED_EMAIL }}
156+
E2E_ACCOUNT_TIER: ${{ env.MINTED_TIER }}
157+
run: npm run test:e2e:live
158+
159+
- name: Reap minted account (teardown)
160+
# ALWAYS runs (even on test failure/cancel) so the minted account + its
161+
# resources are deleted out-of-band. Idempotent: 404 == already gone.
162+
if: always() && env.RUN == '1' && env.MINTED_TEAM_ID != ''
163+
run: |
164+
set -euo pipefail
165+
code="$(curl -sS -o /dev/null -w '%{http_code}' \
166+
-X DELETE "${E2E_API_URL}/internal/e2e/account/${MINTED_TEAM_ID}" \
167+
-H "X-E2E-Token: ${E2E_ACCOUNT_TOKEN}")"
168+
case "$code" in
169+
200|202|204|404|410)
170+
echo "Reaped minted account (HTTP $code)." ;;
171+
*)
172+
echo "::error::DELETE minted account returned HTTP $code — possible leak."
173+
exit 1 ;;
174+
esac
175+
176+
- name: Reap cohort resources from ledger (teardown)
177+
# The per-run ledger reaper sweeps any resource a spec created. Exits
178+
# non-zero on any leak, failing the job loudly (rule 24).
179+
if: always() && env.RUN == '1'
180+
run: npm run reap:live
181+
182+
- name: Upload LIVE trace + ledger on failure
183+
if: failure() && env.RUN == '1'
184+
uses: actions/upload-artifact@v4
185+
with:
186+
name: e2e-prod-trace-${{ github.run_id }}
187+
path: |
188+
test-results/
189+
playwright-report-live/
190+
e2e/.cleanup-ledger.json
191+
if-no-files-found: ignore
192+
retention-days: 14

‎e2e/cohort.ts‎

Lines changed: 94 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,12 @@
1212
// real quota budget, or attempt a real charge.
1313
//
1414
// This is the instanode-web side ONLY. The backend `is_test_cohort` column +
15-
// the guards that read it are intentionally NOT in this PR (one-tree
16-
// discipline) — see the PR body's follow-up note. Until those guards exist,
17-
// LIVE runs MUST target STAGING, never prod.
15+
// the guards that read it ship in the api/worker tree (PR #260): a minted team
16+
// is `is_test_cohort=true`, and the live worker skip-guards neuter
17+
// billing/churn/email/quota for it. With those guards + the mint endpoint
18+
// (cohort-scoped) + the reaper all live, a SANCTIONED minted-account run MAY
19+
// target prod (see assertSafeApiTarget below). An un-sanctioned/un-tokened run
20+
// against prod is still REFUSED so a stray invocation can never hammer prod.
1821
//
1922
// The contract the backend guards will key on (kept here as the single source
2023
// of truth for the string the two repos share):
@@ -76,3 +79,91 @@ export function cohortName(label = 'res'): string {
7679
export function isCohortBranded(value: string | null | undefined): boolean {
7780
return !!value && value.includes(COHORT_MARKER)
7881
}
82+
83+
// ── Prod-target safety (item 3) ──────────────────────────────────────────────
84+
// LIVE specs create REAL backend resources. Originally cohort.ts refused any
85+
// prod E2E_API_URL outright (only STAGING was safe). Now that the backend
86+
// `is_test_cohort` skip-guards (PR #260), the cohort-scoped mint endpoint, and
87+
// the reaper are all live, a prod run is safe IFF it is a SANCTIONED
88+
// minted-account run — i.e. it carries a mint token (E2E_ACCOUNT_TOKEN, used by
89+
// the CI workflow to mint/reap the account) or an already-minted session JWT
90+
// (E2E_SESSION_JWT). A prod target WITHOUT either is still refused, so a stray /
91+
// mis-configured invocation can never provision-and-leak against prod.
92+
93+
/** The prod api host. A prod target is only allowed for a sanctioned minted run. */
94+
export const PROD_API_HOST = 'api.instanode.dev'
95+
96+
/** True when the resolved api base points at the prod api host. */
97+
export function isProdApiTarget(apiUrl: string): boolean {
98+
if (!apiUrl) return false
99+
try {
100+
return new URL(apiUrl).host.toLowerCase() === PROD_API_HOST
101+
} catch {
102+
// Not a parseable URL — be conservative and substring-match the host so a
103+
// malformed-but-prod-looking value can't slip past as "not prod".
104+
return apiUrl.toLowerCase().includes(PROD_API_HOST)
105+
}
106+
}
107+
108+
/**
109+
* True when this process is a SANCTIONED minted-account run: it holds a mint
110+
* token (the workflow mints/reaps the account out-of-band) or an already-minted
111+
* session JWT. Either proves the run is the cohort-scoped, reaped, skip-guarded
112+
* path rather than a stray prod invocation.
113+
*/
114+
export function isSanctionedMintedRun(): boolean {
115+
return !!(process.env.E2E_ACCOUNT_TOKEN || process.env.E2E_SESSION_JWT)
116+
}
117+
118+
/**
119+
* Guard a LIVE spec's resolved api target. Throws (failing the spec loudly,
120+
* never silently passing) when E2E_API_URL points at prod WITHOUT a sanctioned
121+
* minted-account run. Staging targets and sanctioned prod runs pass through.
122+
* Specs call this once at module load (before any provision) via topGuard().
123+
*/
124+
export function assertSafeApiTarget(apiUrl: string): void {
125+
if (isProdApiTarget(apiUrl) && !isSanctionedMintedRun()) {
126+
throw new Error(
127+
`Refusing to run LIVE E2E against prod (${PROD_API_HOST}) without a sanctioned ` +
128+
`minted-account run. Set E2E_ACCOUNT_TOKEN (CI mints+reaps a cohort account) ` +
129+
`or E2E_SESSION_JWT (a pre-minted cohort session), or point E2E_API_URL at staging. ` +
130+
`This guard exists so a stray run can never provision-and-leak real prod resources.`,
131+
)
132+
}
133+
}
134+
135+
// ── Workflow-minted account (item 2) ─────────────────────────────────────────
136+
// The prod E2E workflow mints an ephemeral cohort account up front
137+
// (POST /internal/e2e/account) and exports its session JWT + identity into the
138+
// env. When E2E_SESSION_JWT is set, the authed legs use THAT account's bearer
139+
// instead of self-minting from E2E_JWT_SECRET — so the authed flow runs against
140+
// prod as a real, skip-guarded cohort team. Anon legs are unaffected.
141+
142+
/** The minted account's identity + bearer, surfaced from the workflow env. */
143+
export interface MintedSession {
144+
/** Bearer token for authed requests (the api session JWT). */
145+
token: string
146+
/** The minted team's id (the workflow reaps the account by this out-of-band). */
147+
teamID: string
148+
/** The minted user's email, when the workflow exported it. */
149+
email: string
150+
/** The minted tier (e.g. 'pro'), when the workflow exported it. */
151+
tier: string
152+
}
153+
154+
/**
155+
* Returns the workflow-minted session when E2E_SESSION_JWT is set, else null.
156+
* Authed legs prefer this over self-minting so a prod run uses a real cohort
157+
* account. E2E_TEAM_ID / E2E_ACCOUNT_EMAIL / E2E_ACCOUNT_TIER are the companion
158+
* fields the workflow exports from the mint response.
159+
*/
160+
export function mintedSession(): MintedSession | null {
161+
const token = process.env.E2E_SESSION_JWT
162+
if (!token) return null
163+
return {
164+
token,
165+
teamID: process.env.E2E_TEAM_ID ?? '',
166+
email: process.env.E2E_ACCOUNT_EMAIL ?? '',
167+
tier: process.env.E2E_ACCOUNT_TIER ?? '',
168+
}
169+
}

‎e2e/live-anon-provision.spec.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@
4242

4343
import { expect, test, type APIRequestContext } from '@playwright/test'
4444

45-
import { cohortName, COHORT_MARKER } from './cohort'
45+
import { cohortName, COHORT_MARKER, assertSafeApiTarget } from './cohort'
4646
import {
4747
recordEntity,
4848
loadLedger,
@@ -157,6 +157,10 @@ test.describe('LIVE — every anonymous provision flow → backend-assert → re
157157
'E2E_LIVE=1 but E2E_API_URL/AGENT_API_URL is unset — no backend to target.',
158158
)
159159

160+
// Prod-target safety (item 3): refuse an un-sanctioned prod target; allow it
161+
// only for a minted-account run (E2E_ACCOUNT_TOKEN/E2E_SESSION_JWT present).
162+
if (LIVE && API_URL) assertSafeApiTarget(API_URL)
163+
160164
// Backstop reaper (rule 24): even if a per-service test throws before its
161165
// inline reap, afterAll reaps every still-ledgered entity. The standalone
162166
// reap-cohort.ts re-runs this same path in CI teardown if the whole process

0 commit comments

Comments
 (0)