Skip to content

[F13] Add total_kv_blocks and other K-class params to lockable parameter inventory #258

@sriumcp

Description

@sriumcp

Problem

This is a specific instance of F1 — a campaign-author found mid-run that they'd left a critical knob unlocked, the design agent picked a value that nullified the experiment, and ad-hoc cataloguing of locked parameters in the campaign description's narrative didn't surface it in time.

In paper-memorytime-mirage iter-1, the campaign's locked-parameter table covered model, concurrency, duration, warmup — but NOT total_kv_blocks. The campaign delegated K to "the BLIS hardware_config", giving the agent latitude. iter-1's agent set total_kv_blocks: 1000000. With K=1M blocks × 16 tokens/block = 16M tokens, the workload's largest request (B's 4720-token prefill) used 0.03% of capacity. There was zero cache contention, so the workload's adversarial-variance design produced no memory-time separation. iter-1 reported ρ_memorytime = 1.06 — null result, not because the theory was wrong, but because the agent's choice of K removed the contention the experiment was designed to test.

Why this issue exists separately from F1

The headline problem (no campaign-side anchor for K) is solved by F1's locked_parameters schema field. This issue is the adoption / migration / examples angle:

  1. Once F1 ships, every existing campaign yaml needs a one-time review: which currently-unlocked knobs should be locked?
  2. Patterns to look for: any parameter that affects experimental physics (e.g., K, MaxModelLen, MaxOutputLen, max_num_seqs, gpu_memory_utilization, beta_seconds for bucket schedulers) — not just the obvious ones (model, concurrency, duration, warmup).
  3. The nous create-campaign scaffold should pre-seed locked_parameters with a recommended template that covers common categories.

Desired behavior

  1. Once F1 lands, update nous create-campaign to scaffold a locked_parameters block with the common categories pre-listed (commented out, with a note: "uncomment and set values for parameters whose deviation would invalidate your experiment").
  2. Add a docs page or example campaign showing the discipline: when designing a campaign, enumerate every parameter that could affect the experimental physics, decide for each whether deviation is acceptable, and lock the non-acceptable ones.

Suggested implementation sketch

  1. Modify the nous create-campaign template to include a locked_parameters block with common KV / batching / latency parameters as commented placeholders.
  2. Add a docs/campaign-authoring-guide.md section: "Parameters likely worth locking" (model, K-class, concurrency, duration, warmup, MaxModelLen, MaxOutputLen, max_num_seqs, gpu_memory_utilization, scheduler-tunables like beta/omega/H).
  3. Add F16's BLIS-style parameter inventory checklist as a campaign-authoring template.

Acceptance criteria

  • nous create-campaign scaffold includes a locked_parameters block with categorized common parameters.
  • Campaign-authoring docs include a "what to lock" section.
  • Friction report F13 row in the tracking issue checks off.

Severity

HIGH — caused iter-1 of paper-memorytime-mirage to produce a null result for the headline mirage. Generalizable to any campaign that delegates a physics-affecting parameter without locking it.

Source

friction-report.md F13, paper-memorytime-mirage campaign (2026-05). Depends on F1 landing.


Part of friction-report tracking issue #245.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or requestfriction-reportFrom external campaign-author friction reports

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions