Add GPT-5.6 family to PolicyBench

RESULTS.md

@@ -18,64 +18,10 @@ truth.
 
 ## Full runbook
 
-Use a dated batch directory and keep model outputs per country and model so
-interrupted runs can resume independently.
-
-```bash
-RUN_DIR=results/full_batch_20260501
-
-policybench reference-outputs -n 1000 --seed 42 --country us --program-set headline \
-  -o "$RUN_DIR/us/reference_outputs.csv" \
-  --scenario-manifest-output "$RUN_DIR/us/scenarios.csv"
-
-policybench reference-outputs -n 1000 --seed 42 --country uk --program-set headline \
-  -o "$RUN_DIR/uk/reference_outputs.csv" \
-  --scenario-manifest-output "$RUN_DIR/uk/scenarios.csv"
-
-# Claude models MUST run serially (--parallel 1 --model-parallel 1): the
-# main-thread wall timeout and one-output-per-request contract are not
-# thread-safe. Running Claude with --parallel > 1 aborts. See docs/runbook.md
-# for the canonical, provider-grouped procedure.
-for country in us uk; do
-  for model in claude-opus-4.7 claude-sonnet-4.6 claude-haiku-4.5; do
-    policybench eval-no-tools-chunked \
-      --scenario-manifest "$RUN_DIR/$country/scenarios.csv" \
-      --output-dir "$RUN_DIR/$country/no_tools_chunked" \
-      --country "$country" \
-      --model "$model" \
-      --program-set headline \
-      --chunk-size 50 \
-      --parallel 1 \
-      --model-parallel 1
-  done
-done
-
-# Non-Claude models can run with higher concurrency.
-for country in us uk; do
-  for model in grok-4.3 grok-4.20 grok-4.1-fast gpt-5.5 gpt-5.4-mini gpt-5.4-nano \
-    gemini-3.1-pro-preview gemini-3.5-flash gemini-3-flash-preview \
-    gemini-3.1-flash-lite-preview; do
-    policybench eval-no-tools-chunked \
-      --scenario-manifest "$RUN_DIR/$country/scenarios.csv" \
-      --output-dir "$RUN_DIR/$country/no_tools_chunked" \
-      --country "$country" \
-      --model "$model" \
-      --program-set headline \
-      --chunk-size 50 \
-      --parallel 4
-  done
-done
-
-for country in us uk; do
-  mkdir -p "$RUN_DIR/$country/by_model"
-  cp "$RUN_DIR/$country/no_tools_chunked/by_model/"*.csv "$RUN_DIR/$country/by_model/"
-done
-
-policybench export-full-run --run-dir "$RUN_DIR"
-```
-
-For first-pass cost control, run the same commands with `-n 100` in a separate
-scratch directory before launching the 1,000-household batch.
+Use the [canonical benchmark runbook](docs/runbook.md) for provider grouping,
+resumable output layout, current model names, new-model onboarding, cost
+controls, merging, export, and verification. Keeping the operational procedure
+in one place prevents pasted commands here from drifting behind `config.MODELS`.
 
 ## Artifacts
 

annotations/us_full_run_20260612_policyengine_4_16_1_populace/us_case_reference_explanations.csv

@@ -786,7 +786,7 @@ us,scenario_086,child1_head_start_eligible,0.0,6,"PolicyEngine determined that t
 us,scenario_109,ssi,0.0,25,"PolicyEngine calculated total household SSI for 2026 as $0.00 because none of the five household members—the head (age 30), spouse (age 29), or the three children (ages 9, 6, and 0)—meet SSI's eligibility criteria. SSI is an individual benefit that requires recipients to be aged 65 or older, blind, or disabled; the engine's inputs indicate that all five household members have `is_ssi_aged_blind_disabled=False`, meaning none qualify on these grounds. Although the computation trace shows that all household members have `takes_up_ssi_if_eligible=True`, indicating they would claim the benefit if eligible, the absence of any qualifying condition results in $0.00 SSI for each person and therefore $0.00 for the household overall.",
 us,scenario_030,federal_income_tax_before_refundable_credits,0.0,8,"For this single adult in Texas with approximately $13,000 in household income, PolicyEngine calculated federal income tax before refundable credits as $0. The computation shows that income_tax_before_credits, the primary component, equals $0, indicating that the household's tax liability before any credits is eliminated entirely. This result reflects that the adjusted_gross_income of $13,000 falls below the standard deduction threshold for a single filer in 2026, which means no taxable income remains after the standard deduction is applied. The other components—net investment income tax, recapture of investment credit, unreported payroll tax, and qualified retirement penalty—all contribute $0 to the final calculation, as none of these additional tax items apply to this household's circumstances.",
 us,scenario_049,payroll_tax,15134.5,6,"PolicyEngine calculated a total payroll tax of $15,134.50 for this household by summing the employee-side payroll tax components. The employee social security tax contributed $11,439, calculated by applying the 6.2% employee rate to wages up to the 2026 social security wage base limit. The employee Medicare tax added $3,668.50, derived from the standard 1.45% rate on all wages earned by the higher-earning spouse. An additional Medicare tax of $27 was assessed on wages exceeding the $200,000 threshold for single filers (adjusted proportionally for the joint household structure). These three components—social security tax, standard Medicare tax, and additional Medicare tax—sum to the total spm_unit_payroll_tax of $15,134.50.",
-us,scenario_109,snap,8020.55419921875,495,"PolicyEngine calculated an annual SNAP benefit of $8,020.55 for this Florida household of 5 people with approximately $30,000 in household income by determining monthly eligibility and benefit amounts, then annualizing the result. The household qualifies for SNAP because it meets all eligibility tests: the gross income of $2,500 per month is 78% of the federal poverty guideline ($3,223.33), the net income of $1,733.15 (after applying a $266.85 standard deduction and $500 earned income deduction) is 54% of the poverty guideline, assets total only $900, and all household members meet work requirements and immigration status requirements. The monthly benefit is calculated as the maximum allotment of $1,209.52 for a 5-person household in the contiguous U.S. minus the expected contribution of $519.90 (30% of net income), yielding a monthly benefit of $689.62. Multiplying this monthly amount by 12 months produces the annual SNAP benefit of $8,020.55.",
+us,scenario_109,snap,8020.55419921875,495,"PolicyEngine calculates SNAP month by month and sums the twelve benefits. From January through September, the $261 standard deduction leaves $1,739 of net monthly income; subtracting the $521.70 expected contribution from the $1,183 maximum allotment produces about $661.30 per month. From October through December, updated fiscal-year parameters raise the standard deduction to $266.85 and the maximum allotment to $1,209.52, producing about $689.62 per month after the $519.90 expected contribution. The household passes the income, asset, and work-requirement tests throughout the year, and summing the unrounded monthly benefits yields $8,020.55.",
 us,scenario_086,child1_medicare_eligible,0.0,1,"PolicyEngine determined that child1_medicare_eligible equals False (represented as 0.0) for this Georgia household. Medicare eligibility is primarily determined by age, and the child in this household does not meet the age threshold of 65 years required for standard Medicare coverage. Additionally, the household does not qualify for Medicare based on disability or end-stage renal disease status, which are the other primary pathways to Medicare eligibility for individuals under 65. Therefore, PolicyEngine correctly classified the child as not Medicare-eligible for the 2026 tax year.",
 us,scenario_070,state_refundable_credits,0.0,46,"PolicyEngine calculated state_refundable_credits as $0 for this Illinois resident with ~$38,100 household income in 2026. The computation aggregates refundable tax credits across all 50 states and the District of Columbia, with il_refundable_credits specifically evaluated for this household since the IL state indicator is true. Since Illinois does not currently offer refundable tax credits that apply to this household's income level and filing status, il_refundable_credits returned $0. All other state-level refundable credit variables (from Alabama through Wyoming, plus New York City) also evaluated to $0, as the household does not reside in those jurisdictions. The sum of these zero values across all states and localities yields the final state_refundable_credits amount of $0.",
 us,scenario_049,self_employment_tax,0.0,1,"PolicyEngine calculated a self-employment tax of $0 for this New Hampshire joint household in 2026 because neither spouse had self-employment income. Self-employment tax is only assessed on net earnings from self-employment activities, and since this household's approximately $262,821 in total income was derived from other sources (such as wages or investment income rather than self-employment), no self-employment tax liability was triggered. The calculation correctly reflects that self-employment tax applies only to individuals with qualifying self-employment earnings, which this household did not report.",

app/src/components/Hero.tsx

@@ -2,7 +2,7 @@ import { DEFAULT_VERSION_ID } from "../lib/dataVersionsRuntime";
 import type { BenchData, CountryCode } from "../types";
 import SiteHeader, { type HeaderNavItem } from "./SiteHeader";
 
-const SNAPSHOT_DATE_LABEL = "Snapshot 2026-07-07";
+const SNAPSHOT_DATE_LABEL = "Snapshot 2026-07-09";
 
 export default function Hero({
   selectedView,

app/src/components/ProgramHeatmap.tsx

@@ -2,9 +2,9 @@ import { useMemo } from "react";
 import { getVariableLabel, type BenchData, type HeatmapEntry } from "../types";
 import {
   MODEL_LABELS,
-  MODEL_ORDER,
   getPerformanceSurfaceColor,
   getPerformanceTextColor,
+  orderModels,
 } from "../modelMeta";
 import { programIsActive, type ProgramOption } from "../lib/programFilters";
 import ProgramFilterDropdown from "./ProgramFilterDropdown";
@@ -73,8 +73,10 @@ export default function ProgramHeatmap({
     return { grid: lookup, variables };
   }, [activeProgramIds, data]);
 
-  const models = MODEL_ORDER.filter((m) =>
-    data.heatmap.some((h) => h.condition === "no_tools" && h.model === m),
+  const models = orderModels(
+    data.heatmap
+      .filter((entry) => entry.condition === "no_tools")
+      .map((entry) => entry.model),
   );
 
   const averageScores = useMemo(() => {
@@ -132,7 +134,7 @@ export default function ProgramHeatmap({
                   key={m}
                   className="text-center text-[10px] uppercase tracking-[0.14em] text-text-muted font-medium pb-3 px-2 w-28"
                 >
-                  {MODEL_LABELS[m]}
+                  {MODEL_LABELS[m] ?? m}
                 </th>
               ))}
               <th className="text-center text-[10px] uppercase tracking-[0.14em] text-text-muted font-medium pb-3 px-2 w-20">

app/src/components/ProviderMark.tsx

@@ -1,4 +1,14 @@
-import { Anthropic, DeepSeek, Google, OpenAI, XAI } from "@lobehub/icons";
+import {
+  Alibaba,
+  Anthropic,
+  DeepSeek,
+  Google,
+  Minimax,
+  Moonshot,
+  OpenAI,
+  XAI,
+  ZAI,
+} from "@lobehub/icons";
 import { PROVIDER_LABELS, type ProviderKey } from "../modelMeta";
 
 export default function ProviderMark({
@@ -32,11 +42,15 @@ export default function ProviderMark({
       aria-label={label}
       className={`inline-flex items-center justify-center text-text ${className}`}
     >
+      {provider === "alibaba" && <Alibaba.Color size={size} aria-hidden="true" />}
       {provider === "anthropic" && <Anthropic size={size} color="currentColor" aria-hidden="true" />}
       {provider === "deepseek" && <DeepSeek.Color size={size} aria-hidden="true" />}
       {provider === "google" && <Google.Color size={size} aria-hidden="true" />}
+      {provider === "minimax" && <Minimax.Color size={size} aria-hidden="true" />}
+      {provider === "moonshot" && <Moonshot size={size} color="currentColor" aria-hidden="true" />}
       {provider === "openai" && <OpenAI size={size} color="currentColor" aria-hidden="true" />}
       {provider === "xai" && <XAI size={size} color="currentColor" aria-hidden="true" />}
+      {provider === "zai" && <ZAI size={size} color="currentColor" aria-hidden="true" />}
     </span>
   );
 }

app/src/components/ScenarioExplorer.tsx

@@ -8,13 +8,12 @@ import {
 } from "../types";
 import { formatCurrency } from "../format";
 import {
-  FRONTIER_MODELS,
   MODEL_LABELS,
-  MODEL_ORDER,
   PROVIDER_LABELS,
+  frontierModelsFor,
   getProviderForModel,
   getPredictionTextColor,
-  isFrontierModel,
+  orderModels,
   type ProviderKey,
 } from "../modelMeta";
 import { binaryFlag } from "../lib/scoring";
@@ -216,19 +215,39 @@ export default function ScenarioExplorer({
     for (const varData of Object.values(filteredPredictions)) {
       for (const m of Object.keys(varData)) unique.add(m);
     }
-    return MODEL_ORDER.filter((m) => unique.has(m));
+    return orderModels(unique);
   }, [filteredPredictions]);
 
+  const frontierModels = useMemo(
+    () => frontierModelsFor(allModels),
+    [allModels],
+  );
+  const frontierModelSet = useMemo(
+    () => new Set(frontierModels),
+    [frontierModels],
+  );
+
+  const providerOptions = useMemo(() => {
+    const available = new Set(
+      allModels
+        .map((model) => getProviderForModel(model))
+        .filter((provider): provider is ProviderKey => provider !== null),
+    );
+    return (Object.keys(PROVIDER_LABELS) as ProviderKey[]).filter(
+      (provider) => available.has(provider) || providerFilter.has(provider),
+    );
+  }, [allModels, providerFilter]);
+
   const models = useMemo(() => {
     return allModels.filter((m) => {
-      if (frontierOnly && !isFrontierModel(m)) return false;
+      if (frontierOnly && !frontierModelSet.has(m)) return false;
       if (providerFilter.size > 0) {
         const provider = getProviderForModel(m);
         if (!provider || !providerFilter.has(provider)) return false;
       }
       return true;
     });
-  }, [allModels, frontierOnly, providerFilter]);
+  }, [allModels, frontierModelSet, frontierOnly, providerFilter]);
 
   // The detail modal opens on click. Selection is scoped to the current
   // scenario so switching households doesn't carry the old cell over.
@@ -299,7 +318,7 @@ export default function ScenarioExplorer({
       // explanation fetch would never trigger — start it now.
       setExplanationsEnabled(true);
       // The linked model may be hidden behind the frontier-only default.
-      if (!isFrontierModel(model)) setFrontierOnly(false);
+      if (!frontierModelSet.has(model)) setFrontierOnly(false);
     };
     const timer = window.setTimeout(apply, 0);
     return () => window.clearTimeout(timer);
@@ -538,7 +557,7 @@ export default function ScenarioExplorer({
                 ? "border-primary-strong bg-primary-strong text-white"
                 : "border-border bg-card text-text-secondary hover:text-text"
             }`}
-            title={`Show only one frontier flagship per provider (${FRONTIER_MODELS.map((m) => MODEL_LABELS[m] ?? m).join(", ")})`}
+            title={`Show only one frontier flagship per provider (${frontierModels.map((m) => MODEL_LABELS[m] ?? m).join(", ")})`}
           >
             Frontier only
           </button>
@@ -555,7 +574,7 @@ export default function ScenarioExplorer({
             aria-labelledby="scenarios-provider-label"
             className="inline-flex flex-wrap items-center gap-1"
           >
-            {(Object.keys(PROVIDER_LABELS) as ProviderKey[]).map((provider) => {
+            {providerOptions.map((provider) => {
               const isActive = providerFilter.has(provider);
               return (
                 <button
@@ -608,7 +627,7 @@ export default function ScenarioExplorer({
                       size={12}
                       className="flex-shrink-0"
                     />
-                    {MODEL_LABELS[m]?.split(" ").slice(-2).join(" ")}
+                    {(MODEL_LABELS[m] ?? m).split(" ").slice(-2).join(" ")}
                   </div>
                 </th>
               ))}

app/src/data.artifact.json

@@ -1,9 +1,9 @@
 {
   "version": 1,
   "repo": "PolicyEngine/policybench",
-  "tag": "dashboard-data-20260708",
+  "tag": "dashboard-data-20260709b",
   "asset": "dashboard-data.json",
-  "url": "https://github.com/PolicyEngine/policybench/releases/download/dashboard-data-20260708/dashboard-data.json",
-  "sha256": "d95e19a2f461f78367629bb75fdc0ad1771d0eab6d70550afbd2cf8dddabbf53",
-  "bytes": 56950706
+  "url": "https://github.com/PolicyEngine/policybench/releases/download/dashboard-data-20260709b/dashboard-data.json",
+  "sha256": "cdb99bfb3e5181803b2dd4229cfcf23e9bb31a4208d7130bcf7e73f939edc0e5",
+  "bytes": 64159293
 }

app/src/data.versions.json

@@ -5,7 +5,7 @@
     {
       "id": "1.1",
       "label": "1.1",
-      "description": "Corrected ground truth (policyengine-us 1.755.4); every model at unconfigured provider defaults; adds Claude Fable 5 and Sonnet 5 - 15 models",
+      "description": "Corrected ground truth (policyengine-us 1.755.4); every model at unconfigured provider defaults - 23 models",
       "snapshotLabel": null,
       "artifact": {
         "pointer": "live"