plan: skip-scan + write-path landed, deployed, live-verified (PR-5.1.5)

Cold render /api/groups 6.0s -> 0.64-1.07s live; gauntlet + data-checks + close remain.

Signed-off-by: "Connor Tsui" <connor@spiraldb.com>

Author: connortsui20

.big-plans/ct__bench-v4.md

@@ -12,8 +12,8 @@ current_pr: PR-5.1.5
 pr_index: 3
 outstanding_must_fix: 0
 deferred_items_total: 17
-last_user_touchpoint: 2026-06-11T22:30:00Z
-last_user_touchpoint_what: "COLD-RENDER DECISION MADE: user chose (a) recursive-CTE skip-scan for the per-group summaries (jump to each series' latest -> ~ms -> cold render ~1s), covering discovery + DISTINCT-engine as well where natural. Remaining PR-5.1.5 work in order: skip-scan rewrite, (c) write-path commit_timestamp, gauntlet, PR-5.0 deferred data-checks, Step 2.5 close. PRIOR touchpoint (2026-06-11T21:15Z): PR-5.1.5 DEPLOYED + LIVE-VERIFIED (user authorized 'run everything'). Committed + Docker-verified + PUSHED (origin ct/bench-v4 @ a4834ba1f): sargable (2e637401e), parallelize e (629b5b0b6), denormalize c + migration 006 + d indexes (680b30e6e), covering index migration 007 + DISTINCT ON summary (a4834ba1f). Prod: instance upsized db.t4g.medium + vortex-bench-pg16(work_mem 32MB); migrations 006+007 APPLIED to prod as master + VACUUM ANALYZE; web-deploy.yml deploy SUCCEEDED incl the CDN probe (the push-unblock the handoff waited for). LIVE measurements: tpch chart 13.6s->0.094s; /api/groups 38s->0.079s cached / ~6.0s cache-cold server render (x-vercel-cache MISS); landing / ~6.9s cold. Prod SQL: chart 75ms, tpcds summary 2.4s (Index Only Scan, Heap Fetches 0 via idx_query_measurements_summary covering INCLUDE value_ns), DISTINCT engine 458ms, discovery 1.3s. REMAINING for PR-5.1.5 close-out: (1) cold render 6s->faster (concurrency bump band-aid OR recursive-CTE skip-scan for ~ms summaries — user to choose); (2) (c) WRITE-PATH still TODO (post-ingest.py + migrate/src/postgres.rs populate commit_timestamp — needed before develop merge; existing data backfilled by 006 so deployed site is correct now, NULLS LAST + a re-backfill cover the transient); (3) gauntlet review of the PR; (4) PR-5.0 deferred data-checks (slug-match + ~5 chart slugs vs v2) now verifiable on the fast site; (5) Step 2.5 close. (b)/(d) folded into the ~6s, acceptable. Backfill UPDATE created bloat -> VACUUM ANALYZE was required post-006 (lesson). schema-deploy.yml is develop-only so the ct/bench-v4 push only fired web-deploy; at the develop merge schema-deploy no-ops 006/007 (already in prod ledger, master-pre-applied like 005)."
+last_user_touchpoint: 2026-06-11T23:30:00Z
+last_user_touchpoint_what: "SKIP-SCAN + WRITE-PATH DONE, DEPLOYED, LIVE-VERIFIED. Skip-scans (15b778b01): collectQuerySummary 3-branch successor walk (prod tpcds 2796ms->63ms), collectQueryGroups 15-branch NULL-aware walk (2333ms->20ms), collectFilterUniverse single-col skips (565ms->0.2ms); all byte-identical vs replaced queries on testcontainer + full prod seed; web vitest 211 green; NO migration 008 needed (NULLS LAST handled via IS NOT NULL descent + IS NULL fallback probe). LIVE: /api/groups cache-cold 0.64-1.07s (was ~6.0s), landing cold 1.2s (was ~6.9s), x-vercel-cache MISS verified, payload sane (16 groups / 13 summaries / 372 charts). Write-path (e3861734e): post-ingest.py upsert stamps commit_timestamp via scalar subquery both paths + new pytest (100 green); migrate loader post-COPY 006-backfill UPDATE in same txn + e2e asserts 0 NULL/0 drift (100 green); e2e init now applies 001+006. User approved prod reads this session (EXPLAIN-verified on prod as bench_read). REMAINING: gauntlet review (Step 2.3), PR-5.0 deferred data-checks (slug-match + ~5 chart slugs vs v2), Step 2.5 close. PRIOR touchpoint (2026-06-11T22:30Z): user chose (a) recursive-CTE skip-scan. Committed + Docker-verified + PUSHED (origin ct/bench-v4 @ a4834ba1f): sargable (2e637401e), parallelize e (629b5b0b6), denormalize c + migration 006 + d indexes (680b30e6e), covering index migration 007 + DISTINCT ON summary (a4834ba1f). Prod: instance upsized db.t4g.medium + vortex-bench-pg16(work_mem 32MB); migrations 006+007 APPLIED to prod as master + VACUUM ANALYZE; web-deploy.yml deploy SUCCEEDED incl the CDN probe (the push-unblock the handoff waited for). LIVE measurements: tpch chart 13.6s->0.094s; /api/groups 38s->0.079s cached / ~6.0s cache-cold server render (x-vercel-cache MISS); landing / ~6.9s cold. Prod SQL: chart 75ms, tpcds summary 2.4s (Index Only Scan, Heap Fetches 0 via idx_query_measurements_summary covering INCLUDE value_ns), DISTINCT engine 458ms, discovery 1.3s. REMAINING for PR-5.1.5 close-out: (1) cold render 6s->faster (concurrency bump band-aid OR recursive-CTE skip-scan for ~ms summaries — user to choose); (2) (c) WRITE-PATH still TODO (post-ingest.py + migrate/src/postgres.rs populate commit_timestamp — needed before develop merge; existing data backfilled by 006 so deployed site is correct now, NULLS LAST + a re-backfill cover the transient); (3) gauntlet review of the PR; (4) PR-5.0 deferred data-checks (slug-match + ~5 chart slugs vs v2) now verifiable on the fast site; (5) Step 2.5 close. (b)/(d) folded into the ~6s, acceptable. Backfill UPDATE created bloat -> VACUUM ANALYZE was required post-006 (lesson). schema-deploy.yml is develop-only so the ct/bench-v4 push only fired web-deploy; at the develop merge schema-deploy no-ops 006/007 (already in prod ledger, master-pre-applied like 005)."
 subagent_invocations_this_pr: 2
 subagent_invocations_total: 147
 review_cycles_this_pr: 0
@@ -22,7 +22,7 @@ phase_end_cycle: 0
 phase_end_reject_cycles: 0
 last_phase_end_verdict: null
 current_pr_is_ci_reopen: null
-last_commit: 2e637401e
+last_commit: e3861734e
 last_cycle_commits: []
 ```