feat(vcr-ra): spill-cost ranking — Chaitin cost/degree spill choice (wiring step 2) (#285)

* feat(vcr-ra): spill-cost ranking — Chaitin cost/degree spill choice (#209/#242, wiring step 2)

VCR-RA-001 wiring step 2 (docs/design/vcr-ra-allocator-wiring.md). When no
node can be simplified (every degree >= k), the optimistic spill candidate is
now chosen by Chaitin's metric — minimise cost/degree, where cost is the
node's def+use occurrence count (each spill inserts a store per def and a
reload per use) and high degree is good to spill (more interference relieved).

  - color_graph_precolored_costed(g, k, precolored, costs): the costed
    variant. Integer cross-multiplied comparison (cost_a*deg_b vs
    cost_b*deg_a), ties to the smallest register — fully deterministic.
  - use_def_counts(instrs): per-register def+use occurrence counts via
    reg_effect — the cost numerator.
  - color_graph_precolored delegates with an empty cost map: every node
    defaults to cost 1, so minimising cost/degree degenerates to maximising
    degree — the historic degree-only candidate AND tie-break, exactly.
  - allocate_function now computes real counts and ranks spill candidates
    by them.

Pure and unwired: allocate_function is reachable only from the flag-gated
shadow pass (eprintln-only) and tests, so emitted bytes are unchanged by
construction — and verified: all four fixture ELFs are cmp-bit-identical to
main's, and the full differential passes (control_step 13/13 0x00210A55,
flight_seam inlined+flat 0x07FDF307, div_const 338/338).

Tests: costed_spill_picks_the_cheapest_node_not_the_smallest_reg (K4@k=3,
degree-symmetric, costs force R3 — exact spill set {R3}),
empty_costs_reproduce_the_degree_only_choice (same graph, no costs → {R0}),
use_def_counts_sums_defs_and_uses. 326/326 lib tests green, clippy clean.

Next (consequential, full differential gate): virtual-register selector
output (flag default-off) -> spill-code insertion -> wire-in + per-function
flip -> delete a hard-fail site.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(vcr-ra): settle the step-3 design — 3a post-pass range re-allocation, 3b exhaustion-only virtual temps (#242)

3a is bounded and wins the measured targets (spurious spills, const-CSE
residency) but cannot remove the hard-fail; 3b (virtual temp ids >= 9 on the
would-have-failed path only) is what meets the acceptance criterion. Both
consume the same no-regret machinery (rename_def, range_interference, generic
Chaitin core, apply_range_coloring).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(vcr): release-plan tags — SEL-002→v0.11.35, RA-001→v0.11.37, RA-002→v0.11.38 (#242)

Release plan lives in rivet (release-vX.Y tags; queryable via
rivet list --filter '(has-tag "release-v0.11.35")'). This rivet build has
no first-class release: field — friction filed separately.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: avrabe

artifacts/verified-codegen-roadmap.yaml

@@ -98,10 +98,16 @@ artifacts:
   #    cross-check / defense-in-depth). Verified to accept color_graph's own
   #    output on real codegen. Still unwired. Wiring plan:
   #    docs/design/vcr-ra-allocator-wiring.md.
+  #  - Spill-cost ranking (wiring step 2): color_graph_precolored_costed picks
+  #    the optimistic spill candidate by Chaitin's cost/degree (cost = def+use
+  #    occurrence count via use_def_counts) instead of degree-only; the uncosted
+  #    API delegates with all-costs-equal, reproducing the historic choice
+  #    exactly. allocate_function now ranks by real counts. Pure, unwired,
+  #    fixtures bit-identical (verified by cmp + full differential).
   #    Remaining for VCR-RA-001 (CONSEQUENTIAL, changes emitted bytes, full
-  #    differential gate): spill-cost ranking -> virtual-reg selector output
-  #    (flag default-off) -> spill-code insertion -> wire-in + per-function flip
-  #    where the differential proves no-regression -> delete a hard-fail site.
+  #    differential gate): virtual-reg selector output (flag default-off) ->
+  #    spill-code insertion -> wire-in + per-function flip where the
+  #    differential proves no-regression -> delete a hard-fail site.
   - id: VCR-RA-001
     type: sw-req
     title: SSA-based register allocator with spilling (remove the hard-fail)
@@ -116,7 +122,7 @@ artifacts:
       architecture (R9 globals / R11 mem-base / R12 IP-scratch) and the
       bounds-mode R10 invariant.
     status: proposed
-    tags: [codegen, register-allocation, ssa, regalloc2, track-a]
+    tags: [codegen, register-allocation, ssa, regalloc2, track-a, release-v0.11.37]
     links:
       - type: derives-from
         target: VCR-001
@@ -303,7 +309,7 @@ artifacts:
       (fewer moves per clamp, reuse a resident bound) emitted by the selector.
       Independent of the allocator (different code region); composes with it.
     status: proposed
-    tags: [codegen, selector, instruction-selection, clamp, track-b, perf]
+    tags: [codegen, selector, instruction-selection, clamp, track-b, perf, release-v0.11.35]
     links:
       - type: derives-from
         target: VCR-001
@@ -328,7 +334,7 @@ artifacts:
       9-vs-12 gap to native). Make R10 allocatable iff bounds-checking is off.
       Composes with the allocator (which reads the pool size from config).
     status: proposed
-    tags: [codegen, register-allocation, pool, reserved-registers, track-c, perf]
+    tags: [codegen, register-allocation, pool, reserved-registers, track-c, perf, release-v0.11.38]
     links:
       - type: derives-from
         target: VCR-001

crates/synth-synthesis/src/liveness.rs

@@ -937,6 +937,31 @@ pub fn color_graph_precolored(
     k: usize,
     precolored: &BTreeMap<Reg, usize>,
 ) -> ColorResult {
+    // No cost information: every node defaults to cost 1, so minimising
+    // cost/degree degenerates to maximising degree — exactly the historic
+    // degree-only heuristic (same candidate, same tie-break).
+    color_graph_precolored_costed(g, k, precolored, &BTreeMap::new())
+}
+
+/// `k`-colouring with precoloured pins **and spill-cost ranking** — VCR-RA-001
+/// wiring step 2 (`docs/design/vcr-ra-allocator-wiring.md`).
+///
+/// When no node can be simplified (every degree ≥ `k`), the optimistic spill
+/// candidate is chosen by **Chaitin's metric**: minimise `cost / degree`, where
+/// `cost` is the node's def+use occurrence count (each spill inserts a store
+/// per def and a reload per use, so cost scales with occurrences) and high
+/// degree is *good* to spill (removing the node relieves more interference).
+/// Compared cross-multiplied in integers (`cost_a·deg_b < cost_b·deg_a`) to
+/// avoid floats; ties break to the smallest register for determinism. A node
+/// missing from `costs` defaults to cost 1, so an empty map reproduces the
+/// degree-only heuristic exactly. Pure function: a decision, not a mutation.
+pub fn color_graph_precolored_costed(
+    g: &InterferenceGraph,
+    k: usize,
+    precolored: &BTreeMap<Reg, usize>,
+    costs: &BTreeMap<Reg, usize>,
+) -> ColorResult {
+    let cost_of = |r: &Reg| costs.get(r).copied().unwrap_or(1);
     // Working adjacency over the FREE (non-precoloured) nodes only; precoloured
     // nodes are fixed, so they never enter the simplify worklist — but they
     // remain in every free node's neighbourhood (via `g.neighbors`) so they
@@ -949,17 +974,23 @@ pub fn color_graph_precolored(
     let mut stack: Vec<Reg> = Vec::with_capacity(adj.len());
 
     // SIMPLIFY / SPILL: push every free node, preferring low-degree (< k) nodes;
-    // when none qualify, optimistically push the highest-degree node.
+    // when none qualify, optimistically push the cheapest-to-spill node.
     while !adj.is_empty() {
         let pick = adj
             .iter()
             .find(|(_, nbrs)| nbrs.len() < k)
             .map(|(r, _)| *r)
             .unwrap_or_else(|| {
-                // No simplifiable node → optimistic spill candidate: highest
-                // degree (ties broken by register order for determinism).
+                // No simplifiable node → optimistic spill candidate: minimise
+                // cost/degree (Chaitin), ties to the smallest register. The
+                // comparator never returns Equal for distinct registers, so the
+                // pick is deterministic regardless of iteration order.
                 adj.iter()
-                    .max_by(|a, b| a.1.len().cmp(&b.1.len()).then(b.0.cmp(a.0)))
+                    .min_by(|a, b| {
+                        (cost_of(a.0) * b.1.len())
+                            .cmp(&(cost_of(b.0) * a.1.len()))
+                            .then(a.0.cmp(b.0))
+                    })
                     .map(|(r, _)| *r)
                     .unwrap()
             });
@@ -1564,8 +1595,26 @@ pub fn apply_const_cse(instrs: &[ArmInstruction]) -> (Vec<ArmInstruction>, usize
     (out, count)
 }
 
+/// Per-register def+use occurrence counts over an instruction stream — the
+/// numerator of Chaitin's spill cost (each spill inserts a store per def and a
+/// reload per use, so spill cost scales with occurrences). Unmodeled ops are
+/// skipped; callers needing strict modelling already gate on
+/// [`interference_graph`] returning `Some`, which implies every op is modeled.
+pub fn use_def_counts(instrs: &[ArmInstruction]) -> BTreeMap<Reg, usize> {
+    let mut counts: BTreeMap<Reg, usize> = BTreeMap::new();
+    for ins in instrs {
+        if let Some(e) = reg_effect(&ins.op) {
+            for r in e.defs.iter().chain(e.uses.iter()) {
+                *counts.entry(*r).or_insert(0) += 1;
+            }
+        }
+    }
+    counts
+}
+
 /// Run the register-allocator pipeline on a function: interference graph →
-/// `k`-colouring with `precolored` reserved registers pinned → result. `k` is
+/// `k`-colouring with `precolored` reserved registers pinned and spill
+/// candidates ranked by Chaitin cost (def+use count / degree) → result. `k` is
 /// the allocatable-pool size (e.g. 9 for synth's R0–R8). Pure: it computes a
 /// decision and does not mutate the instruction stream — the call the codegen
 /// wiring (VCR-RA-001) makes, runnable today as a shadow pass.
@@ -1578,7 +1627,8 @@ pub fn allocate_function(
         return AllocationOutcome::Declined;
     };
     let remat_opportunities = analyze_function(instrs).redundant_consts.len();
-    match color_graph_precolored(&graph, k, precolored) {
+    let costs = use_def_counts(instrs);
+    match color_graph_precolored_costed(&graph, k, precolored, &costs) {
         ColorResult::Colored(coloring) => AllocationOutcome::Allocated {
             coloring,
             remat_opportunities,
@@ -2493,6 +2543,70 @@ mod tests {
         assert!(matches!(color_graph(&g, 0), ColorResult::Spilled(_)));
     }
 
+    #[test]
+    fn costed_spill_picks_the_cheapest_node_not_the_smallest_reg() {
+        // K4 at k=3: every node has degree 3, so degree alone cannot
+        // discriminate — the degree-only heuristic falls back to the smallest
+        // register (R0). With costs, R3 is by far the cheapest to spill
+        // (fewest def+use occurrences), so Chaitin's cost/degree metric must
+        // pick it instead. Spilling R3 leaves K3, which 3-colours, so the
+        // spill set is exactly {R3}.
+        let g = clique(&[Reg::R0, Reg::R1, Reg::R2, Reg::R3]);
+        let costs: BTreeMap<Reg, usize> =
+            [(Reg::R0, 10), (Reg::R1, 10), (Reg::R2, 10), (Reg::R3, 2)].into();
+        match color_graph_precolored_costed(&g, 3, &BTreeMap::new(), &costs) {
+            ColorResult::Spilled(s) => {
+                assert_eq!(s, BTreeSet::from([Reg::R3]), "cheapest node spills");
+            }
+            ColorResult::Colored(_) => panic!("K4 cannot be 3-coloured"),
+        }
+    }
+
+    #[test]
+    fn empty_costs_reproduce_the_degree_only_choice() {
+        // Same K4 at k=3 with NO costs: all nodes default to cost 1, the
+        // ratios tie, and the tie-break picks the smallest register — the
+        // historic degree-only behavior, byte-for-byte.
+        let g = clique(&[Reg::R0, Reg::R1, Reg::R2, Reg::R3]);
+        match color_graph(&g, 3) {
+            ColorResult::Spilled(s) => {
+                assert_eq!(
+                    s,
+                    BTreeSet::from([Reg::R0]),
+                    "degree-only tie → smallest reg"
+                );
+            }
+            ColorResult::Colored(_) => panic!("K4 cannot be 3-coloured"),
+        }
+    }
+
+    #[test]
+    fn use_def_counts_sums_defs_and_uses() {
+        // movw r0,#7        → r0: 1 def
+        // add  r1, r0, r0   → r1: 1 def, r0: +2 uses
+        // add  r2, r1, r0   → r2: 1 def, r1: +1 use, r0: +1 use
+        let seq = vec![
+            ins(ArmOp::Movw {
+                rd: Reg::R0,
+                imm16: 7,
+            }),
+            ins(ArmOp::Add {
+                rd: Reg::R1,
+                rn: Reg::R0,
+                op2: Operand2::Reg(Reg::R0),
+            }),
+            ins(ArmOp::Add {
+                rd: Reg::R2,
+                rn: Reg::R1,
+                op2: Operand2::Reg(Reg::R0),
+            }),
+        ];
+        let c = use_def_counts(&seq);
+        assert_eq!(c.get(&Reg::R0), Some(&4));
+        assert_eq!(c.get(&Reg::R1), Some(&2));
+        assert_eq!(c.get(&Reg::R2), Some(&1));
+    }
+
     #[test]
     fn precolored_node_keeps_its_colour_and_constrains_neighbours() {
         // Triangle R0—R1—R2; pin R0 to colour 2. With k=3, R1/R2 must avoid 2

docs/design/vcr-ra-allocator-wiring.md

@@ -97,3 +97,62 @@ Coalescing, live-range splitting, and rematerialisation are regalloc2-class
 refinements deferred until the basic spill-capable allocator is wired and proven.
 The first win is simply: **stop hard-failing**, so the cost-gates become
 revert-able.
+
+## Step 3 design decision (settled 2026-06-09, before any code)
+
+Step 3 splits into two pieces with different power and different risk. They are
+sequenced — 3a first, because it is bounded and delivers the measured wins; 3b
+second, because only it can meet the acceptance criterion.
+
+### 3a. Post-pass range re-allocation (bounded; lands first)
+
+Re-allocate the **greedy physical stream** after selection: split it into value
+ranges (`straight_line_value_ranges` — already landed), build interference over
+*ranges* instead of physical registers, colour with the costed Chaitin pass
+(step 2), and rewrite each range to its assigned register.
+
+- **What it wins (already measured):** removes the spurious spills (flat_flight
+  peak value-pressure = 9 = pool → 17 spills eliminable) and restores constant
+  residency, which is what lets `apply_const_cse` fire >0 times.
+- **What it cannot win:** the hard-fail sites. The selector fails *during
+  emission*, before any post-pass runs — so 3a does not remove the exhaustion
+  hard-fail and does not unlock the cost-gate reverts.
+- **Machinery needed (no-regret — 3b consumes the same pieces):**
+  `rename_def` (the def-side sibling of `rename_use`), `range_interference`
+  (ranges → adjacency over vreg ids; ranges interfere iff
+  `a.def < b.last_use && b.def < a.last_use`, plus co-defined ranges — the
+  dies-at-birth boundary case is deliberately non-interfering so a consumer can
+  reuse its operand's register, the in-place-select pattern), a Chaitin core
+  generic over node id (the step-2 colourer refactored from `Reg` to any
+  `Ord + Copy`), and `apply_range_coloring` (replay ranges, rename uses from the
+  open-range map then defs from the at-this-index map).
+- **Scope guard:** leaf, straight-line, fully-modeled segments only — the same
+  scope every analysis in `liveness.rs` already declines outside of. Flag-gated
+  default-off; bounds: bit-identical fixtures when off, measured spill/movw
+  delta when on.
+
+### 3b. Selector-side virtual temps (the hard-fail remover; lands second)
+
+Teach `select_with_stack` to emit **virtual temp ids past the physical pool**
+instead of hard-failing in `alloc_temp_safe`: when all of R0–R8 are pinned by
+live stack values, hand out a virtual id (representation: keep `next_temp: u8`
+but let indices ≥ 9 denote virtuals — no `Reg` enum change; the virtual ids
+exist only between selection and allocation, and 3a's allocator+rewrite maps
+them onto physical registers, spilling per step 4 where k-colouring fails).
+Encoding is unreachable for virtual ids by construction: the allocate+rewrite
+pass runs before the encoder and either assigns physical registers or spills.
+
+- This — and only this — removes the exhaustion hard-fail, which is the
+  VCR-RA-001 acceptance criterion ("a previously load-bearing greedy fix
+  becomes revertable").
+- Risk containment: virtual ids are emitted **only on the would-have-failed
+  path** at first (the existing 9-register fast path stays byte-identical for
+  every function that never exhausts), so all frozen fixtures are untouched by
+  construction until the flag flips wider.
+
+### Why not pure selector-side vregs from the start
+
+Rewriting the selector to emit vregs for *every* temp invalidates the byte-level
+behavior of every function at once — the opposite of the per-function,
+differential-gated migration this plan requires. 3a + exhaustion-only-3b keeps
+the frozen world the default at every merge.