Problem
This is a specific instance of F1 — a campaign-author found mid-run that they'd left a critical knob unlocked, the design agent picked a value that nullified the experiment, and ad-hoc cataloguing of locked parameters in the campaign description's narrative didn't surface it in time.
In paper-memorytime-mirage iter-1, the campaign's locked-parameter table covered model, concurrency, duration, warmup — but NOT total_kv_blocks. The campaign delegated K to "the BLIS hardware_config", giving the agent latitude. iter-1's agent set total_kv_blocks: 1000000. With K=1M blocks × 16 tokens/block = 16M tokens, the workload's largest request (B's 4720-token prefill) used 0.03% of capacity. There was zero cache contention, so the workload's adversarial-variance design produced no memory-time separation. iter-1 reported ρ_memorytime = 1.06 — null result, not because the theory was wrong, but because the agent's choice of K removed the contention the experiment was designed to test.
Why this issue exists separately from F1
The headline problem (no campaign-side anchor for K) is solved by F1's locked_parameters schema field. This issue is the adoption / migration / examples angle:
- Once F1 ships, every existing campaign yaml needs a one-time review: which currently-unlocked knobs should be locked?
- Patterns to look for: any parameter that affects experimental physics (e.g., K, MaxModelLen, MaxOutputLen, max_num_seqs, gpu_memory_utilization, beta_seconds for bucket schedulers) — not just the obvious ones (model, concurrency, duration, warmup).
- The
nous create-campaign scaffold should pre-seed locked_parameters with a recommended template that covers common categories.
Desired behavior
- Once F1 lands, update
nous create-campaign to scaffold a locked_parameters block with the common categories pre-listed (commented out, with a note: "uncomment and set values for parameters whose deviation would invalidate your experiment").
- Add a docs page or example campaign showing the discipline: when designing a campaign, enumerate every parameter that could affect the experimental physics, decide for each whether deviation is acceptable, and lock the non-acceptable ones.
Suggested implementation sketch
- Modify the
nous create-campaign template to include a locked_parameters block with common KV / batching / latency parameters as commented placeholders.
- Add a
docs/campaign-authoring-guide.md section: "Parameters likely worth locking" (model, K-class, concurrency, duration, warmup, MaxModelLen, MaxOutputLen, max_num_seqs, gpu_memory_utilization, scheduler-tunables like beta/omega/H).
- Add F16's BLIS-style parameter inventory checklist as a campaign-authoring template.
Acceptance criteria
Severity
HIGH — caused iter-1 of paper-memorytime-mirage to produce a null result for the headline mirage. Generalizable to any campaign that delegates a physics-affecting parameter without locking it.
Source
friction-report.md F13, paper-memorytime-mirage campaign (2026-05). Depends on F1 landing.
Part of friction-report tracking issue #245.
Problem
This is a specific instance of F1 — a campaign-author found mid-run that they'd left a critical knob unlocked, the design agent picked a value that nullified the experiment, and ad-hoc cataloguing of locked parameters in the campaign description's narrative didn't surface it in time.
In paper-memorytime-mirage iter-1, the campaign's locked-parameter table covered
model,concurrency,duration,warmup— but NOTtotal_kv_blocks. The campaign delegated K to "the BLIS hardware_config", giving the agent latitude. iter-1's agent settotal_kv_blocks: 1000000. With K=1M blocks × 16 tokens/block = 16M tokens, the workload's largest request (B's 4720-token prefill) used 0.03% of capacity. There was zero cache contention, so the workload's adversarial-variance design produced no memory-time separation. iter-1 reportedρ_memorytime = 1.06— null result, not because the theory was wrong, but because the agent's choice of K removed the contention the experiment was designed to test.Why this issue exists separately from F1
The headline problem (no campaign-side anchor for K) is solved by F1's
locked_parametersschema field. This issue is the adoption / migration / examples angle:nous create-campaignscaffold should pre-seedlocked_parameterswith a recommended template that covers common categories.Desired behavior
nous create-campaignto scaffold alocked_parametersblock with the common categories pre-listed (commented out, with a note: "uncomment and set values for parameters whose deviation would invalidate your experiment").Suggested implementation sketch
nous create-campaigntemplate to include alocked_parametersblock with common KV / batching / latency parameters as commented placeholders.docs/campaign-authoring-guide.mdsection: "Parameters likely worth locking" (model, K-class, concurrency, duration, warmup, MaxModelLen, MaxOutputLen, max_num_seqs, gpu_memory_utilization, scheduler-tunables like beta/omega/H).Acceptance criteria
nous create-campaignscaffold includes alocked_parametersblock with categorized common parameters.Severity
HIGH — caused iter-1 of paper-memorytime-mirage to produce a null result for the headline mirage. Generalizable to any campaign that delegates a physics-affecting parameter without locking it.
Source
friction-report.mdF13, paper-memorytime-mirage campaign (2026-05). Depends on F1 landing.Part of friction-report tracking issue #245.