Add MiniMax-M2.7-NVFP4 (N=5, TP=2): temp serving-trap + exhaustive-completer findings

- harness.py: add --top-p/--top-k plumbing + retain reasoning_content across turns
  (interleaved-thinking models); run_microbench.sh: BENCH_TEMP/TOP_P/TOP_K env overrides
  (default temp=0.3 unchanged for other models).
- Finding 1: MiniMax-NVFP4 runs away (repetition loop -> model_exceeded_max_tokens) at the
  bench default temp=0.3 (74% of coding cells; testwrite 10/10); at the model-card
  temp=1.0/top_p=0.95/top_k=40 the SAME cells on the SAME 131072 cap are 0/10 runaway,
  58/60 done_signal. Clean A/B => sampling, not capability, not cap.
- Finding 2: exhaustive-completer temperament. Aggregate 7/12 (35/60). p2 analysis 20/20;
  scope-constrained coding 0/5 (testwrite+refactor edit files told to leave alone);
  p3_market ctx-exhaustion. Complementary to 397B's surgical restraint.
- Power: TP=2 heaviest simultaneous draw (median ~896W active, peak 1089W=91% cap, both
  GPUs >400W 64% of samples) vs 397B pipeline 670W/985W never >1000W.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: User Name

hardware-tests/qwen3.5-397b-vs-step3.7-flash-2026-05-29/README.md

@@ -21,6 +21,9 @@ Step-3.7-Flash-NVFP4 entry on the same box.
 
 ## Files
 - [findings.md](findings.md) — N=10 scorecard + headline findings + power + cross-model qualitative.
+- [findings-minimax-m2.7.md](findings-minimax-m2.7.md) — **MiniMax-M2.7-NVFP4 (N=5, vLLM TP=2):** the
+  temp=0.3→1.0 serving-trap (0 runaways at spec vs 74% at default), the "exhaustive completer" temperament
+  (p2 analysis 20/20, scope-constrained coding 0/5, p3_market ctx-exhaustion), and the heaviest TP=2 power draw.
 - [findings-n10.md](findings-n10.md) — auto-generated replicate-stability table (flags small-N flips) + finish-reason audit.
 - [power-analysis.md](power-analysis.md) — dual-GPU power percentiles, pipeline asymmetry, %-of-cap.
 - [QUALITATIVE.md](QUALITATIVE.md) — behavioral analysis beyond pass/fail (token economy, packaging,

hardware-tests/qwen3.5-397b-vs-step3.7-flash-2026-05-29/findings-minimax-m2.7.md

@@ -0,0 +1,133 @@
+# MiniMax-M2.7-NVFP4 on 2× RTX PRO 6000 Blackwell — microbench N=5 (+ the temp serving-trap)
+
+MiniMax-M2.7 (230B-A10B MoE), served as **NVFP4 on vLLM tensor-parallel (TP=2)**, run through the MMBT
+12-family agentic microbench. Added to the [397B vs Step-3.7-Flash entry](findings.md) as a fifth model.
+
+**N=5** per cell (60 cells). Comparators in the main entry are N=10 (397B) / N=1 (Step, 27B-Q4, Coder-Q4) —
+so MiniMax is **N=5, an asymmetry to read with the same caution this entry already documents for small N.**
+
+## TL;DR — two findings, the first is the bigger one
+
+1. **A serving trap, not a capability gap (the headline).** Run at the bench's cross-model default
+   **`temperature=0.3`**, MiniMax-NVFP4 looked *broken* on coding: it ran the agent loop correctly for
+   ~29 iterations, then on the **final text turn entered a repetition loop that generated tens of
+   thousands of tokens to the `max_tokens` cap** (`finish_reason: model_exceeded_max_tokens`). Across the
+   two coding families the temp=0.3 run reached before it was stopped: **14/19 cells ran away (74%)** —
+   `p1_bugfix` 4/9, `p1_testwrite` **10/10**. Re-run at MiniMax's **model-card sampling
+   (`temperature=1.0, top_p=0.95, top_k=40`)**, the *same cells on the same 131072 cap* produce **0/10
+   runaways** — clean `done_signal`. Across all **60** N=5 cells: **58 `done_signal`, 0 runaways**, 2
+   context-exhaustion. The clean A/B on the identical cell/cap proves the runaway was **sampling**, not
+   the model and not a stingy token cap. *Greedy-ish decode on a reasoning model is a documented
+   repetition-loop trap; MiniMax's card mandates temp=1.0.*
+
+2. **MiniMax is an "exhaustive completer" — a genuinely double-edged temperament.** Its instinct is to do
+   the maximum: fix every bug it sees, write a full test suite + audit/ADR/CHANGELOG docs, research
+   exhaustively. This **dominates open analysis** and **sinks scope-constrained edits** (see scorecard).
+   It is the mirror image of 397B's surgical restraint.
+
+## The serving trap — before/after on the same cells
+
+| family | temp=0.3 (broken) | temp=1.0 (spec) |
+|---|---|---|
+| p1_bugfix | 5 `done_signal` / **4 runaway** (44%) | **5 `done_signal` / 0 runaway** |
+| p1_testwrite | 0 / **10 runaway (100%)** | **5 `done_signal` / 0 runaway** |
+| all 60 cells (spec) | — | **58 `done_signal`, 0 runaway, 2 ctx-exhaustion** |
+
+A temp=0.3-runaway cell generated **71k–106k tokens in a single final turn** (100% of the per-turn cap).
+That is degenerate repetition, not legitimate output — and the identically-capped 397B (also 131072,
+also temp 0.3-equivalent path) had **0 runaways in 240 cells**, so the cap is not the cause.
+
+## Scorecard (N=5; PASS = grader verdict, majority ≥3/5 marks the family ✓)
+
+| family | PASS/5 | avg iters | dominant fail-reason |
+|---|---|---|---|
+| p2_extract | **5/5** ✓ | 11 | — |
+| p2_ci | **5/5** ✓ | 41 | — |
+| p2_hallucination | **5/5** ✓ | 15 | — |
+| p2_triage | **5/5** ✓ | 13 | — |
+| p3_business | **5/5** ✓ | 18 | — |
+| p3_doc | **4/5** ✓ | 19 | 1× word-limit |
+| p3_pm | **3/5** ✓ | 14 | — |
+| p1_bugfix | 2/5 | 131 | `ruff_no_regression` ×3 (lint nits in its own new tests) |
+| p3_writing | 1/5 | 30 | length/quality |
+| p1_testwrite | 0/5 | 86 | `logalyzer_unchanged` ×5 (**scope violation**) |
+| p1_refactor | 0/5 | 75 | `tests_unchanged` ×5, `non_output_files_unchanged` ×2 (**scope violation**) |
+| p3_market | 0/5 | 51 | 2× **ctx-exhaustion (HTTP 400)** + 3 fail |
+
+**Aggregate: 7/12 families pass majority; 35/60 cells (58%).** This lands in the same **~7–8/12 band** as
+397B (8/12 no-think), Step-3.7-Flash, 27B-Q4 and Coder-Q4 — MiniMax **aggregate-ties the field, but the
+per-family *shape* is distinctive** (and that shape, not the tie, is the finding).
+
+## Qualitative — the "exhaustive completer" pattern (every claim cited to graded cells)
+
+- **Open analysis = pure upside. p2 is a perfect 20/20** (extract/ci/hallucination/triage all 5/5), fast
+  (~11–41 iters). When there is no scope or length constraint, thoroughness only helps. Best-in-class here.
+- **Scope-constrained coding = real liability, not a grader artifact. `p1_testwrite` 0/5 and `p1_refactor`
+  0/5**, failing on **substantive** criteria: the test-writing task says *"add tests, leave the production
+  code unchanged"* → MiniMax rewrites the production code anyway (`logalyzer_unchanged` fails 5/5); the
+  refactor task says *"only touch the output package"* → it modifies tests and out-of-scope files
+  (`tests_unchanged` 5/5, `non_output_files_unchanged` 2/5). It **cannot resist improving everything**,
+  which is exactly what these guardrails forbid. This is the **opposite of 397B**, whose surgical restraint
+  *passes* `p1_refactor`.
+- **`p1_bugfix` is the in-between case (2/5):** it fixes the planted bugs correctly (the O(n²) `load()`
+  measured 11.9s→0.57–0.63s by the grader; the `collections.Iterable` import removed; 69–82 tests pass)
+  but trips `ruff_no_regression` (2→3, 2→5) on **unused-import nits in its own newly-written tests** —
+  a lint technicality. *Here* the binary score understates it; on testwrite/refactor it does not.
+- **Open-ended sinks exhaust context. `p3_market` 0/5**, with **2 cells hitting the 131072 ceiling
+  outright (HTTP 400)** — it keeps issuing research tool-calls until the conversation won't fit. The p3
+  analog of over-delivery.
+- **Structured synthesis is fine** — `p3_business` 5/5, `p3_doc` 4/5, `p3_pm` 3/5. Where the deliverable
+  is well-bounded, the thoroughness lands.
+- **Failure texture is its own category:** not 397B's quiet *stall* (omission), not Coder/Flash's
+  *runaway* (over-generation cutoff) — MiniMax's misses are **self-inflicted: scope violations and context
+  exhaustion from doing too much.** And it is **expensive** — 75–131 iters/cell on p1 (the high end of the
+  field) vs ~11–18 on the analysis cells it aces.
+
+**One-line verdict:** complementary to 397B. MiniMax aces analysis where 397B is average; 397B respects
+guardrails where MiniMax bulldozes them. A task-class strengths finding, not a scaling-law tie.
+
+## GPU power — TP=2 is the heaviest simultaneous draw of the bench
+
+| | serving | combined draw | both GPUs hot at once | crosses 1000W |
+|---|---|---|---|---|
+| **MiniMax-M2.7** | vLLM **TP=2** (NVFP4) | active-decode median **~896W**, peak **1089W (91% of 1200 cap)** | **64% of samples** (>400W each) | **yes** |
+| 397B | llama.cpp **pipeline** | median 670W, max 985W | rarely | **never** (0 / 3868) |
+
+The cause is parallelism mode, not size: **tensor-parallel makes both GPUs compute the same layers
+simultaneously** (so they fire together and push toward the PSU ceiling), whereas **pipeline-parallel
+alternates** them (GPU0 leads, combined draw stays well under the cap). MiniMax is balanced per-GPU
+(320W/363W mean) — TP splits evenly — vs 397B's GPU0-leads pipeline asymmetry.
+
+## Caveats (read these with the numbers)
+
+- **Sampling deviation:** MiniMax ran at its **card-specified `temp=1.0/top_p=0.95/top_k=40`**, *not* the
+  bench's cross-model `temp=0.3`. This is a deliberate, documented per-model deviation — at temp=0.3 the
+  model is off-spec and degenerates (finding #1), so a temp=0.3 score would be meaningless. The receipt
+  schema records `temperature` but **not** `top_p`/`top_k`; the full profile is recorded here and in the
+  launch command.
+- **N=5 vs comparators' N=10/N=1** — directional. The per-family rates above (esp. the 0/5 and 5/5 ones)
+  are stable enough to characterize, but treat the aggregate as ±1 family.
+- **131072 context** (vs Step/27B/Coder at 262144) — a real cap asymmetry, footnoted; it did not cause the
+  runaways (those were sampling) but it is where `p3_market` exhausted context. MiniMax's native max is ~196k.
+- **NVFP4 on Blackwell SM120** is a maturity-rough surface (vLLM has open repetition/kernel issues for
+  MiniMax-NVFP4); we did **not** use `--enable-expert-parallel` (the named trigger in vLLM #31856).
+- This build emits **no separate `<think>`/`reasoning_content`** despite MiniMax-M2 being an
+  interleaved-thinking model — so the interleaved-thinking history requirement was moot here (the harness
+  retains `reasoning_content` defensively regardless).
+
+## Reproduce
+```bash
+# Serve (vLLM TP=2, NVFP4). minimax_m2 parsers; no expert-parallel.
+docker run -d --name vllm-minimax --gpus all --shm-size 16g -e NCCL_P2P_DISABLE=1 \
+  -v $HOME/models:/models:ro -p 127.0.0.1:8001:8000 vllm/vllm-openai:latest \
+  --model /models/nvidia-MiniMax-M2.7-NVFP4 --served-model-name minimax-m2.7 \
+  --tensor-parallel-size 2 --gpu-memory-utilization 0.92 --quantization modelopt \
+  --trust-remote-code --max-model-len 131072 --disable-custom-all-reduce \
+  --tool-call-parser minimax_m2 --reasoning-parser minimax_m2 --enable-auto-tool-choice \
+  --host 0.0.0.0 --port 8000
+
+# Run at the model card's sampling (BENCH_* env overrides the default temp=0.3):
+BENCH_TEMP=1.0 BENCH_TOP_P=0.95 BENCH_TOP_K=40 \
+  bash tooling/scripts/run_microbench.sh minimax-m2.7 8001 minimax-m2.7-spec 5 "" "" 131072
+bash tooling/scripts/grade_microbench.sh minimax-m2.7-spec
+```

hardware-tests/qwen3.5-397b-vs-step3.7-flash-2026-05-29/findings.md

@@ -153,9 +153,26 @@ analyzed with `tooling/bench_power.py`. 3,868 paired samples.
   the cards and fire in sequence, so the two rarely peak simultaneously. Combined with the CPU-bound tool
   phases, this is *why* the rig never thermally stresses on this workload.
 - **Why it matters:** the "both GPUs at 600W = 1200W" worst case simply does not occur for single-stream
-  agentic inference under pipeline parallelism. (The MiniMax-M2.7 run uses vLLM **tensor**-parallel, which
-  fires both GPUs per token — the expected contrast is a higher, more simultaneous combined draw; that
-  comparison is added when the MiniMax run lands.)
+  agentic inference under pipeline parallelism. (Contrast confirmed by the MiniMax-M2.7 run, which uses
+  vLLM **tensor**-parallel: both GPUs fire per token, combined active-decode draw **median ~896W, peak
+  1089W (91% of cap), both GPUs >400W in 64% of samples, crossing 1000W** — the heaviest simultaneous
+  draw of the bench. TP-vs-pipeline topology, not model size, drives it. See
+  [findings-minimax-m2.7.md](findings-minimax-m2.7.md).)
+
+## MiniMax-M2.7-NVFP4 (added — N=5, vLLM TP=2) — see [findings-minimax-m2.7.md](findings-minimax-m2.7.md)
+
+Two findings, full detail in the linked doc:
+1. **A temperature serving-trap, not a capability gap.** At the bench's default `temp=0.3`, MiniMax-NVFP4
+   runs the agent loop correctly then degenerates into a repetition loop on its final turn
+   (`model_exceeded_max_tokens`): **14/19 coding cells ran away (testwrite 10/10)**. At the model card's
+   `temp=1.0/top_p=0.95/top_k=40`, the *same cells on the same 131072 cap* are **0/10 runaway**; all 60
+   N=5 cells: **58 `done_signal`, 0 runaway**. Clean A/B → sampling, not the model, not the cap.
+2. **An "exhaustive completer" temperament.** Aggregate **7/12 (35/60 cells)** — ties the ~7–8/12 band —
+   but the shape is distinctive and **complementary to 397B**: **p2 analysis a perfect 20/20** (thoroughness
+   pure upside), **scope-constrained coding 0/5 on both testwrite & refactor** (it edits files it was told
+   to leave alone — real over-reach, opposite of 397B's surgical restraint), `p3_market` exhausts context
+   (2× HTTP 400). *Deviation footnote: MiniMax alone runs temp=1.0 (off the temp=0.3 cohort) — justified
+   because temp=0.3 is off-spec for it; and N=5 vs the cohort's N=10/N=1.*
 
 ## Cross-model qualitative (vs 27B / Coder-Next at Q4)
 

tooling/harness.py

@@ -337,7 +337,7 @@ def execute_tool(name, args, log_dir, require_files=None, require_git_tag=False)
 
 def agent_loop(api_url, model, system_prompt, task, log_dir, max_iters=10000,
                max_completion_total=10**12, max_model_len=262144,
-               stuck_threshold=30, temperature=0.0,
+               stuck_threshold=30, temperature=0.0, top_p=None, top_k=None,
                require_files=None, require_git_tag=False, reasoning_effort=None,
                enable_thinking=None):
     """Run the agent until done() or limits hit. Returns final state dict."""
@@ -377,6 +377,13 @@ def agent_loop(api_url, model, system_prompt, task, log_dir, max_iters=10000,
             "max_tokens": max_tokens_safe,
             "stream": False,
         }
+        # Optional nucleus / top-k sampling. Some models specify a required operating point
+        # (e.g. MiniMax-M2 card: temperature=1.0, top_p=0.95, top_k=40) and degenerate into
+        # repetition loops under greedy decode. Sent only when set; seed=42 keeps it reproducible.
+        if top_p is not None:
+            payload["top_p"] = top_p
+        if top_k is not None:
+            payload["top_k"] = top_k  # vLLM OpenAI server accepts top_k as a top-level sampling param
         # Top-level chat_template_kwargs are passed through to the template render
         # (vLLM and llama.cpp --jinja both honor them). Each key is a no-op for
         # models whose template doesn't reference it.
@@ -445,8 +452,17 @@ def agent_loop(api_url, model, system_prompt, task, log_dir, max_iters=10000,
             finish_reason = f"model_exceeded_max_tokens_{max_tokens_safe}"
             break
 
-        # Append assistant message to history (preserve tool_calls so the model sees its own tool intents)
+        # Append assistant message to history (preserve tool_calls so the model sees its own tool intents).
+        # Interleaved-thinking models (e.g. MiniMax-M2, served with --reasoning-parser minimax_m2) split
+        # the <think> block out into a separate reasoning_content field. The MiniMax card REQUIRES
+        # retaining it across turns: "Do not remove the <think>...</think> part, otherwise the model's
+        # performance will be negatively affected" (their numbers: -36% tau2, -40% BrowseComp when prior
+        # thinking is stripped — state drift + weakened self-correction on long-horizon toolchains, which
+        # is exactly a clean agentic run that derails on a final turn). Pass reasoning_content back so the
+        # template re-renders the prior thinking. No-op for models that don't emit the field.
         assistant_msg = {"role": "assistant", "content": msg.get("content") or ""}
+        if msg.get("reasoning_content"):
+            assistant_msg["reasoning_content"] = msg["reasoning_content"]
         if msg.get("tool_calls"):
             assistant_msg["tool_calls"] = msg["tool_calls"]
         messages.append(assistant_msg)
@@ -561,6 +577,12 @@ def main():
                          "At temp=0 with seed=42, models can fall into fixed-point loops on long-horizon "
                          "tasks (same context → same response → same tool result → same response). "
                          "0.3-0.5 is typical for agentic work and breaks these traps without much off-task drift.")
+    ap.add_argument("--top-p", type=float, default=None,
+                    help="Nucleus sampling top_p, sent only when set. Pair with --temperature/--top-k to "
+                         "match a model's specified operating point (e.g. MiniMax-M2: temp=1.0 top_p=0.95 "
+                         "top_k=40). Leave unset to use the server default.")
+    ap.add_argument("--top-k", type=int, default=None,
+                    help="Top-k sampling, sent only when set (vLLM extension). See --top-p.")
     ap.add_argument("--stuck-threshold", type=int, default=30,
                     help="Iterations of unchanged workspace state hash before the harness aborts the run. "
                          "Default 30 was tuned on the memo/board/code tasks (whole job fits in ~100 iters, "
@@ -705,6 +727,7 @@ def main():
 
     summary = agent_loop(api_url, args.model, system_prompt, task, log_dir,
                          max_iters=args.max_iters, temperature=args.temperature,
+                         top_p=args.top_p, top_k=args.top_k,
                          stuck_threshold=args.stuck_threshold,
                          require_files=require_files,
                          require_git_tag=bool(args.require_git_tag),

tooling/scripts/run_microbench.sh

@@ -65,6 +65,14 @@ MAXLEN="${7:-}"             # optional: served context window (e.g. 131072 for t
 MAXLEN_FLAG=""
 [ -n "$MAXLEN" ] && MAXLEN_FLAG="--max-model-len $MAXLEN"
 
+# Sampling overrides via env (default keeps the cross-model temp=0.3 protocol). Some models
+# specify a required operating point and loop under low temp — e.g. MiniMax-M2 card mandates
+# temperature=1.0, top_p=0.95, top_k=40. Set BENCH_TEMP / BENCH_TOP_P / BENCH_TOP_K to deviate;
+# the deviation is recorded per-run in receipt.json (temperature) and must be footnoted in findings.
+TEMP="${BENCH_TEMP:-0.3}"
+TOPP_FLAG=""; [ -n "${BENCH_TOP_P:-}" ] && TOPP_FLAG="--top-p $BENCH_TOP_P"
+TOPK_FLAG=""; [ -n "${BENCH_TOP_K:-}" ] && TOPK_FLAG="--top-k $BENCH_TOP_K"
+
 # Guard: run names + the idempotent skip check are keyed by LABEL only. If an
 # effort is set but not encoded in the label, a later effort would reuse the same
 # run names and be skipped as "already complete". Fail fast instead.
@@ -157,7 +165,9 @@ for entry in "${TASKS[@]}"; do
       "$TOOLING/tasks/$task_file" \
       --model "$MODEL" \
       --port "$PORT" \
-      --temperature 0.3 \
+      --temperature "$TEMP" \
+      $TOPP_FLAG \
+      $TOPK_FLAG \
       --stuck-threshold 500 \
       $REASONING_FLAG \
       $THINKING_FLAG \