Add GPT-5.6 family to PolicyBench

[MaxGhenis]

Publishes dashboard-data-20260709b (sha256 cdb99bfb3e518180…, 64,159,293 bytes; full 23-model predictions.csv.gz attached).

What changed

• Adds GPT-5.6 Sol, Terra, and Luna across the evaluator, model cards, pricing, default roster, runbook, and dashboard metadata.
• Runs all three models over the complete 100-household US benchmark and folds 5,952 new prediction rows into the 20-model v1.1 board. Household-weighted bounded scores are 95.323% (Sol), 93.661% (Terra), and 92.539% (Luna).
• Audits all 489 wrong GPT-5.6 rows across 239 distinct cases and publishes decisive row-level diagnoses. The eight-way audit completed with 0 missing, hedged, or reference-suspect cases after adjudication.
• Preserves cache-write prompt tokens end to end so GPT-5.6 costs reproduce exactly, rejects adapter/model mismatches and connection-only fields in OpenAI Batch requests, and preserves additive columns when folding boards.
• Corrects the pre-existing scenario_109 / snap reference explanation: PolicyEngine uses $661.30 monthly from January through September and $689.62 from October through December. The $8,020.55 reference value and all scores remain unchanged.

Run results

• GPT-5.6 Sol: 1,984/1,984 rows, $8.952271
• GPT-5.6 Terra: 1,984/1,984 rows, $3.893235
• GPT-5.6 Luna: 1,984/1,984 rows, $1.865676
• Total provider cost: $14.711183
• OpenAI Batch was probed and rejected these launch models with model_not_found, so the full runs used the resumable synchronous supervisor.

Verification

• Python: 490 passed, 5 skipped; Ruff and git diff --check clean.
• App: 84 passed; ESLint clean; production build succeeds against the served 20260709b artifact.
• Strict dashboard validation passes for 23 models and 45,632 rows.
• All 39,680 incumbent predictions, metrics, and audit annotations are unchanged; the annotation CSV is an append-only 489-row update. Only the corrected SNAP reference explanation differs from the prior live payload.
• Downloaded release assets match local bytes exactly; decompressing the served predictions.csv.gz reproduces the staged CSV hash.
• Frozen manuscript artifact pins remain untouched; only live_dashboard_artifact advances.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
policybench-site	Ready	Preview, Comment	Jul 9, 2026 8:25pm