feat: distributed FaaS coordination heuristics (FaaS-MADiG / MAPoD / MABR) + PLASMA + coordination rework

[miciav]

Summary

This branch adds a family of price-free distributed heuristics for DiFRALB/DeFRALB and reworks the shared coordination logic across all three for correctness, determinism, and consistency. The heuristics form a spectrum of coordination styles, all built as controlled ablations of the FaaS-MADeA auction:

• FaaS-MADiG — greedy diffusion (removes the price signal; buyer scans its whole one-hop neighbourhood greedily by score).
• FaaS-MAPoD — power-of-d-choices (removes full visibility; buyer probes only d sampled neighbours per step, serves best of sample).
• FaaS-MABR-S / -R / -O — Gauss-Seidel best response (the sequential counterpart to the simultaneous diffusion methods): fixed-order, randomized-order, and capped-reoptimization variants.

All keep the local planning stack and shared helpers, reuse the same seller-side clearing, and isolate exactly one mechanism each (pricing → visibility → sequential-vs-simultaneous coordination).

Validation: uv run pytest -q → 270 passed. FaaS-MABR e2e runs under real Gurobi (smoke + same-seed reproducibility for all three variants).

FaaS-MABR (Gauss-Seidel best response) — decentralized_bestresponse.py

• True best-response sweep: each node releases its current buyer row back to the shared residual-capacity ledger, recomputes its placement greedily by score, and commits the coordinate delta (new_row − previous_row). Later nodes observe earlier nodes' updates through the live ledger — the defining Gauss-Seidel property — and the loop converges to a fixed point.
• Fixed-point termination on allocation_changed (no node revised its row this sweep), the correct signal under release-and-recompute (raw placement volume never settles).
• Three variants: FaaS-MABR-S (fixed order), FaaS-MABR-R (seeded random order; variance via --n_experiments), FaaS-MABR-O (capped local re-optimization via LSP_capped/LSP_capped_fixedr and a per-node re-solve).
• Runtime amortization via compute_sweep_runtime (re-optimization time excluded before amortizing bookkeeping over active nodes); input validation for order/response.
• Additive wiring: CLI keys faas-br-s/-r/-o, method names FaaS-MABR-S/-R/-O (mkey LSPc), compare_results.py palette + default set, planar_comparison.json br_* blocks. Paper-ready LaTeX note under faas-bestresponse-note/ positioning it honestly as the textbook Gauss-Seidel relaxation (not claimed novel).

Cross-method coordination rework (diffusion / powerd / bestresponse)

Landed in lockstep across the three runners so they stay mutually consistent:

• Memory-aware seller eligibility: a node can host a replica only if rho[j] >= memory_requirement[f] (was rho[j] > 0).
• Deterministic seller clearing: explicit (score, index) tie-breaks replace unstable np.argsort; evaluate_assignments reworked — seller_pairs now includes current hosts (so saturated sellers can still be re-evaluated for incumbent replacement), leftover-aware replica start, and lowest-score-incumbent-first reassignment.
• LSPr_fixedr under --fix_r (social-welfare re-solve now fixes replicas consistently with the subproblem); best_centralized_cost initialized to -inf (fixes a latent bug where a negative centralized objective could never set the initial best).
• coordination_rho zeroes memory/replica expansion under --fix_r (no new replicas when replicas are pinned).
• force_memory_bids parity in the block-A memory-bid emission; vectorized rmp_omega/omega/fairness updates.

Shared run_faasmadea.start_additional_replicas: deterministic proportional allocation plus a leftover-memory packing pass (the old per-function floor division left memory unused).

Also on this branch

• FaaS-MADiG and FaaS-MAPoD (greedy diffusion + power-of-d), each with a design spec, plan, and citation-audited LaTeX note (faas-madig-note/, faas-mapod-note/ — 5/5 cited works verified against CrossRef, PDFs committed).
• run.py: method→(mkey, name) mapping flattened into a single METHOD_RESULT_MODELS dict.
• Hierarchical auction, uv migration, and extended test coverage.

PLASMA — Physarum routing + simulated bifurcation (plasma/)

A new fully decentralized method for the same FRALB/DiFRALB problem, built as a self-contained package (precedent: hierarchical_auction/) with zero new dependencies and no edits to existing modules beyond the additive run.py registration (--methods plasma, result label Plasma, LSPc-format outputs — compare_results.py and postprocessing work unchanged).

Two coupled local dynamics, synchronous round barrier only (comparable to every other method on the branch at matched round period):

• Layer A (fast): Physarum-type conductance routing on integer requests — per-request categorical sampling over gated conductances, window reinforcement D ← (1−μ)D + μ·φ·reward, hard capacity gates, spare-capacity gates from heartbeats, emergent failure handling via staleness + decay (no failure detector).
• Layer B (slow): replica allocation minimizing a strictly node-local Hamiltonian — served-demand field −α·min(r·u_max, demand+pull) (concave: idle replicas earn nothing, placement is throughput-aware) + capacity chance-constraint + churn — solved exactly per node by a gcd-scaled multi-choice-knapsack DP (sbm_method: exact, default; microseconds per node). The discrete Simulated Bifurcation solver (dSB, multi-restart) is retained as sbm_method: dsb for the paper's ablation. Hysteresis, randomized commit (p_commit) as the Jacobi-oscillation countermeasure.
• Routing efficiency: admission is reward-aware — when a fresh-spare neighbor's β exceeds local α, traffic is forwarded there first (with an anti-ping-pong guard and NACK→local-retry); otherwise it degenerates exactly into local-first (Physarum coordinates only the overflow, matching the reference model's x/ω split). Routing is window-vectorized (one multinomial per (node, function) — distributionally identical to per-request sampling, witnessed by the unchanged LP-convergence test). Replica provisioning targets arrivals (λ̂ + pull), not accepted traffic, closing the rejected-demand learning loop.
• M6/M7 tooling: plasma/eval/scenario.py (kill/revive scenarios vs dynamic oracle, stale-MILP, greedy — regret + adaptation-lag table) and plasma/eval/sweep.py (parameter-grid driver over solver_options.plasma).
• Protocol: heartbeats carry only spare/alpha/pull — strictly less disclosure than FaaS-MADeA's bid exchange (paper-facing privacy argument in faas-plasma-note/).
• Baselines & metrics: plasma/baselines/ (LP routing reference via scipy.linprog, centralized-MILP snapshot + stale-oracle with overflow shedding, greedy over the existing GreedyCoordinator, run_faasmadea adapter) and plasma/eval/regret.py (cumulative regret vs oracle, adaptation lag).

Acceptance criteria (spec §8) covered by tests: Physarum windowed fractions within ±5% of the LP optimum (measured 1.35%), dSB ≥95% ground-state rate vs brute force on 200 seeded trials, capacity gate, RAM repair, dead-neighbor decay + traffic redistribution, phase-locked-commit thrash/settle, message budget, heartbeat privacy whitelist, and a Gurobi-gated end-to-end gap-vs-MILP check on aligned timesteps.

Validation: uv run pytest -q → 574 passed (including the solver-gated gap test against real Gurobi). Milestone verdicts (all measured against the centralized LMM oracle, per-step Gurobi):

• M5 (≤10% stationary gap): not met, twice halved. Real 10-node instance: 22.7% steady (from 63.8% at first measurement). 20-node β-dominant benchmark: 32.2% — the reward-aware admission closed part of the policy gap (forwarding 12%→25%); the residual is over-subscription of high-β receivers, an open coordination problem (a fair-share spare/deg advertisement was tried and measurably rejected: it throttled forwarding back to pre-fix levels — negative result kept in history, commit ec90be7).
• M6 (failures + non-stationarity): kill/revive scenario on the real instance — PLASMA's gap degrades only ~2 pp during a 10-step node outage and recovers after revive (emergent failure handling works); stale-MILP shows the expected staleness sawtooth (10.1% mean), greedy 5.6%. Caveat stated in the driver docstring: both baselines read true instantaneous global loads — PLASMA is the only neighbor-only method in the table.
• M7 (rare functions): sampled vs unsplittable at 1–8 req/s: 36.6% vs 37.9% — no measurable benefit from unsplittable routing on this instance.

Simulation cost 0.30 s/step (was 45 s: exact knapsack-DP replaces dSB by default — dSB retained as ablation — plus gcd budget scaling and vectorized routing). The served-demand Hamiltonian field and reward-aware admission supersede the original spec's linear field and unconditional local routing — measured design iterations, to be documented in faas-plasma-note/. Design/plan docs under docs/plans/2026-07-0{4,5}-plasma-*.md.

Heads-up for reviewers / reproducibility

The determinism rework of define_assignments/evaluate_assignments and the leftover-packing in start_additional_replicas change the numeric outputs of the existing baselines (FaaS-MADeA, FaaS-MADiG, FaaS-MAPoD), not just the new method — the changes are more correct and deterministic, but any benchmark CSVs/figures produced before this rework are now stale and should be regenerated for an apples-to-apples three-way comparison.

🤖 Generated with Claude Code

[miciav]

Update from Codex: pushed commit e8b7189 (fix experiment orchestration regressions).

Summary:

• Fixed experiment postprocessing/resume handling, generate_only, and 1x1 plot handling.
• Restored top-level compatibility modules for generate_data, load_generator, and utilities.
• Hardened FaaS-MADeA helpers, log parsing, what-if analysis, load clipping, and result comparison.
• Added regression coverage for the fixed paths.

Validation:

• env MPLCONFIGDIR=/tmp/mpl .venv/bin/python -m pytest -q -> 206 passed, 1 warning.
• env MPLCONFIGDIR=/tmp/mpl .venv/bin/ruff check . -> All checks passed.
• GitNexus staged detect_changes run before commit; reported critical scope due shared runner/load/log paths.