plan: SESSION HANDOFF for fresh conversation (PR-5.1.5 CLOSED; PR-5.1 next, prod-write gate noted)

PR-5.1.5 fully closed/deployed/accepted; new handoff orients a cold reader to
PR-5.1 (promote v4 ingest to required + drop v3-write + ship psql-bench.sh),
with the exact CI edit sites, the harness-gated prod cross-check gate, and the
reusable parallel-executor gauntlet flow.

Signed-off-by: "Connor Tsui" <connor@spiraldb.com>

Author: connortsui20

.big-plans/ct__bench-v4.md

@@ -26,9 +26,30 @@ last_commit: b20904ae0
 last_cycle_commits: []
 ```
 
-## SESSION HANDOFF — 2026-06-11 (PR-5.1.5 read-path-perf DEPLOYED; fresh-conversation takeover)
+## SESSION HANDOFF — 2026-06-12 (PR-5.1.5 CLOSED; PR-5.1 next — READ THIS FIRST)
 
-**READ THIS FIRST on resume.** Resume via `/spiral:big-plans` in the `vortex4` worktree. **PR-5.1.5 is now CLOSED** (Step 2.5 done; gauntlet accepted at the project ~3-cycle cap; deployed + live + working; full close-out in its Implementation status entry). `Current State` now routes you to `current_pr: PR-5.1` (promote v4 ingest to required + drop v3-write from CI). This handoff section and the cold-render decision section below are **HISTORICAL** (the cold-render decision resolved as recursive-CTE skip-scan; all listed REMAINING items are done). The older "## SESSION HANDOFF — 2026-06-11 (PR-5.0 + read-path perf)" section is doubly superseded.
+**Resume via `/spiral:big-plans` in the `vortex4` worktree.** `Current State` routes you to `current_pr: PR-5.1`. PR-5.1.5 (read-path perf) is fully CLOSED — do NOT redo it; its complete record is the `### PR-5.1.5:` Implementation status entry. HEAD = `origin/ct/bench-v4` = `4575b1d3d`, working tree clean.
+
+### PR-5.1.5 outcome (one paragraph, so you don't re-open it)
+Shipped + deployed + live + gauntlet-accepted (3 cycles, executor=parallel Claude+Codex). Recursive-CTE skip scans replaced the three whole-table read-path scans (`/api/groups` cold 6.0s -> ~1s; tpch chart 13.6s -> 0.094s; all byte-identical vs the replaced queries on testcontainer + full prod seed). Write-path `commit_timestamp` stamping in `post-ingest.py` + the Rust loader. Migrations 006/007 applied to prod as master + VACUUM ANALYZE. PR-5.0 deferred data-checks resolved (chart-count sets match live v2 `bench.vortex.dev`). Commit trail `93ccf970e`..`4575b1d3d`. All suites green (web vitest 214, migrate Rust 100, python 154).
+
+### PR-5.1 — what's next (NOT started)
+Scope (spine PR-enumeration row PR-5.1): **promote the v4 `--postgres` ingest to required and drop the v3 `--server` write** from the 3 ingest workflows; `post-ingest.py` runs only with `--postgres`; **ship + document `scripts/psql-bench.sh`** (phase-4 end-review fold-in — file does NOT exist yet).
+- **Pre-promotion GATE (do FIRST, before removing `continue-on-error`):** re-run the PR-3.5 cross-check `scripts/cross_check_python_writer.py --postgres "$DSN" --envelopes <real_envelopes.json>` against accumulated prod soak data and confirm clean (Python writer UPDATEs seeded rows, 0 duplicate INSERTs, value columns round-trip). **This is a prod RDS WRITE via the `bench_ingest` IAM role** — harness auto-mode-gated AND likely needs operator AWS creds (the session so far only had `bench_read` password reads; bench_ingest is IAM-token auth via boto3/OIDC). Confirm with the user how to run it (operator-run vs. agent-run with creds) before proceeding — this is the genuine externalized-side-effect gate, NOT a Class A pause.
+- **Exact CI edit sites** (all verified this session): each of `bench.yml`, `sql-benchmarks.yml`, `v3-commit-metadata.yml` has (a) one `Ingest results to v3 server` step using `--server "${{ vars.V3_INGEST_URL }}"` + `INGEST_BEARER_TOKEN` (REMOVE per "drop v3-write"), and (b) three `continue-on-error: true` v4 steps (`Configure AWS credentials for v4 ingest` / `Install uv for v4 ingest` / `Ingest results to v4 Postgres (best-effort)`) — REMOVE the `continue-on-error` so v4 gates CI. bench.yml v3 step ~L108-118, v4 steps ~L128-156; sql-benchmarks.yml v3 ~L493-503, v4 ~L506-534; v3-commit-metadata.yml v3 ~L28-35, v4 ~L38-69. Then remove `--server` mode from `scripts/post-ingest.py` (+ its tests). NOTE the decommission inventory (Table F) lists `INGEST_BEARER_TOKEN` removal — the PR-5.1 v3-step removal drops the `secrets.INGEST_BEARER_TOKEN` references; `git grep -n INGEST_BEARER_TOKEN` must reach 0 by PR-5.3.
+- **Verify**: `yamllint --strict -c .yamllint.yaml` on the 3 changed workflows (REQUIRED per .github/AGENTS.md); `ruff check scripts/` + `pytest scripts/` (post-ingest.py + its tests); `shellcheck`/`bash -n` on psql-bench.sh.
+- **Review**: this is cross-bundle (3 workflows + script + py) -> gauntlet **pr-3** (fresh+correctness+maint). Then Step 2.5 close + advance to PR-5.2 (DNS flip).
+
+### KEY OPERATIONAL FACTS (carried forward)
+- **Docker must be running** for testcontainer suites (web vitest + migrate pytest + the migrate Rust e2e). `python` is not on PATH — use `python3`/`uv run`.
+- **Prod master DSN** (further migrations/VACUUM): `AWS_PROFILE=bench-prod aws rds describe-db-instances ... MasterUserSecret.SecretArn` -> `aws secretsmanager get-secret-value` -> `PGUSER`/`PGPASSWORD` + `PGHOST=vortex-bench-prod.c4f8qygk4xdp.us-east-1.rds.amazonaws.com PGDATABASE=vortex_bench PGSSLMODE=verify-full PGSSLROOTCERT=~/rds-ca.pem`. bench_read pw: `~/.bench-read-pw`; CA: `~/rds-ca.pem`. **Every prod write is harness auto-mode-gated** even under broad approval — surface + get per-action approval. The user approved prod READS this session.
+- **Gauntlet flow used this session** (reuse it): `executor=parallel` via `compose_prompts.py --reference-dir <gauntlet>/reference --executor-routing fresh=parallel,correctness=parallel`; dispatch Claude lenses via `Agent`, Codex lenses via `node ~/.config/claude/plugins/cache/openai-codex/codex/1.0.4/scripts/codex-companion.mjs task --model gpt-5.5 --effort xhigh --prompt-file <f>` (background); then a synthesizer subagent over the 4 executor-tagged blobs. The project **~3-cycle cap** (spine Key-decision) governs: fix-and-accept or defer at cycle 3, never spiral.
+- migrations dir is repo-root `migrations/` (NOT benchmarks-website/migrations/). `schema-deploy.yml` is develop-only so ct/bench-v4 pushes only fire `web-deploy.yml`. Live URL: `https://benchmarks-web.vercel.app`.
+- **OUTSTANDING (unrelated to PR-5.1):** the pre-squash backup ref `backup/bench-v4-pre-squash-*` is still MISSING — recreate it before the PR-5.3 / final squash.
+
+## SESSION HANDOFF — 2026-06-11 (PR-5.1.5 read-path-perf DEPLOYED) [HISTORICAL — PR-5.1.5 CLOSED; superseded by the 2026-06-12 handoff above]
+
+**HISTORICAL.** PR-5.1.5 is closed (see the 2026-06-12 handoff above + the `### PR-5.1.5:` Implementation status entry). This section and the cold-render decision section below describe PR-5.1.5 mid-flight; the cold-render decision resolved as recursive-CTE skip-scan and every listed REMAINING item is done.
 
 ### COLD-RENDER DECISION — RESOLVED 2026-06-11 (user chose (a) skip-scan)
 The site is deployed + fast-cached but **cold render of `/api/groups` is ~6s** (the ~13 per-group summaries are each an index-only scan of the group's full ~1.8M-row history at ~2.4s, run 8-concurrently). For this low-traffic dashboard the 5-min CDN cache often expires between visits, so cold is common. **User decided: (a) recursive-CTE skip-scan** on the summaries (jump to each series' latest -> ~ms -> cold render ~1s); apply the same pattern to discovery + DISTINCT-engine where it falls out naturally (handoff item 4). Options (b) concurrency bump and (c) accept were declined.