Skip to content
2 changes: 1 addition & 1 deletion docs/benchmark-modes/dag.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ SPAWN targets may be referenced from multiple parents — the child conversation

### Join semantics

DAG-style conversations can declare that a turn dispatches only after children from a prior SPAWN branch complete. Gating is declared via a `TurnPrerequisite(kind=SPAWN_JOIN, branch_id=...)` on the *consuming* turn rather than on the spawning branch. The runtime builds a `(conversation_id, branch_id) -> gated_turn_index` index at phase init; when `BranchOrchestrator.intercept()` sees a spawning turn complete, it resolves the gate from the index and suspends the parent until every outstanding child drains. `CreditIssuer.dispatch_join_turn` then issues the parent's gated turn — reusing the parent's already-held session slot (the gated turn has `turn_index > 0`, so session-slot acquisition is naturally skipped).
DAG-style conversations can declare that a turn dispatches only after children from a prior SPAWN branch complete. Gating is declared via a `TurnPrerequisite(kind=SPAWN_JOIN, branch_id=...)` on the *consuming* turn rather than on the spawning branch. The runtime builds a `(conversation_id, branch_id) -> gated_turn_index` index at phase init; when `BranchOrchestrator.intercept()` sees a spawning turn complete, it resolves the gate and suspends the parent. The gated turn dispatches at the later of its normal recorded replay deadline and the completion of every outstanding child, so a fast child cannot erase the parent's authored think time and a slow child does not add that delay twice. `CreditIssuer.dispatch_join_turn` reuses the parent's already-held session slot (the gated turn has `turn_index > 0`, so session-slot acquisition is naturally skipped).

For v1, the orchestrator honors these gate shapes:

Expand Down
18 changes: 11 additions & 7 deletions docs/benchmark-modes/semianalysis-agentx-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,9 @@ transparency.
is rejected), and a fresh `--random-seed` is filled in — pin one yourself for reproducible run-to-run
comparisons ([§7](#7-reading-the-results-metrics-validity-and-submission-requirements)). Timing mode is
locked to agentic-replay.
- **Forbidden — do not pass:** `--ignore-trace-delays`, `--trace-idle-gap-cap-seconds`,
- **Optional timing control:** `--trace-idle-gap-cap-seconds` limits observed
whole-tree idle time independently for each root and all of its descendants.
- **Forbidden — do not pass:** `--ignore-trace-delays`,
`--inter-turn-delay-cap-seconds`, `--synthesis-max-isl` (input truncation), and the
rate/schedule flags `--request-rate` / `--arrival-pattern` / `--user-centric-rate` / `--fixed-schedule`
/ `--adaptive-scale`.
Expand Down Expand Up @@ -282,10 +284,11 @@ the benchmark waits the recorded "think time"/gap before sending the next turn.
request-start to request-start. This is always the case for weka trace replay; there is no flag to
change it. Replay dispatches each turn only after the previous one completes,
so start-to-start deltas would double-count the server's own response time and make every session
drift later turn by turn. Individual trace gaps are not capped. If the entire replay has no active
or ready request, AIPerf uniformly shifts every pending request timer so the next request arrives
within 10 seconds. This avoids benchmarking dead air without changing the recorded spacing inside
one trace while other sessions keep the system busy.
drift later turn by turn. Individual trace gaps are not capped by default; pass
`--trace-idle-gap-cap-seconds S` to advance a trajectory's pending timers when its root and
all descendants have had no request in flight for `S` seconds. This runtime guard does not
rewrite dataset timestamps or bypass spawn/join dependencies. If the entire replay has no active or ready request, AIPerf uniformly
shifts every pending request timer so the next request arrives within 10 seconds.

### Q: What does a single session look like on the wire?
A sequence of chat-completions requests that grow turn over turn (the prompt prefix accumulates),
Expand Down Expand Up @@ -793,7 +796,7 @@ the context-overflow bound is evaluated during the run.
| SemiAnalysis corpus | A pinned `*_weka_*` `--public-dataset` alias, or `weka_hf` pinned to `semianalysisai/cc-traces-weka-062126` | A non-pinned dataset, local `weka_trace`, or another `--hf-weka-dataset` under the scenario; omitting a dataset entirely (CLI synthetic default) | Explicit wrong / unpinned loader: refuses to start (or `submission_valid: false` via `--unsafe-override`). Missing/synthetic: always refuses — `--unsafe-override` cannot bypass |
| `ignore_eos` | Injected `ignore_eos=true` | Explicit `ignore_eos=false` | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
| Streaming | `--streaming` auto-enabled | Explicit `--no-streaming` | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
| Honored trace delays | Recorded think-time gaps are preserved; only globally idle replay time is capped at 10s | `--ignore-trace-delays`, `--trace-idle-gap-cap-seconds`, or `--inter-turn-delay-cap-seconds` | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
| Honored trace delays | Recorded think-time gaps are preserved; `--trace-idle-gap-cap-seconds` may bound observed whole-tree runtime idle without rewriting them | `--ignore-trace-delays` or `--inter-turn-delay-cap-seconds` | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
| No input truncation | Prompts built to the full recorded token counts | Truncating/capping input below the recorded length | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
| Minimum duration | `--benchmark-duration` ≥ 900s (default 1800s) | A duration below 900s | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
| First-turn-prefix cache-busting | Uniqueness marker placed as a first-turn prefix | Disabling cache-busting or relocating the marker | Refuses to start (or `submission_valid: false` via `--unsafe-override`) |
Expand Down Expand Up @@ -1152,7 +1155,8 @@ The ones a serving engineer is most likely to use:
| `--max-context-length` | Drops whole traces whose peak prompt+output exceeds your server's window, so you don't get guaranteed mid-run overflows. Blunter than a `_256k` corpus (removes entire traces, not just over-limit turns). If it would drop everything, the run errors instead of silently emptying the dataset. |
| `--trajectory-start-min-ratio` / `--trajectory-start-max-ratio` | The window within each session where t\* is sampled — i.e. how deep into sessions the measured traffic sits. Defaults to the **full session (0.0–1.0)** under the `inferencex-agentx-mvp` scenario; 0.25–0.75 is the generic CLI default when not scenario-locked. |
| `--system-idle-gap-cap-seconds` | Caps only globally idle replay time (scenario default 10s); all pending timers shift uniformly, preserving order and relative spacing. |
| `--trace-idle-gap-cap-seconds` / `--inter-turn-delay-cap-seconds` | Per-trace/per-turn timing compression; forbidden by the scenario so recorded think times and cache-TTL intervals remain faithful. |
| `--trace-idle-gap-cap-seconds` | Optional observed whole-tree runtime idle cap; unset by default and allowed by the scenario. |
| `--inter-turn-delay-cap-seconds` | Per-turn timing compression; forbidden by the scenario. |
| `--cache-bust` | Where the per-session uniqueness marker goes; the scenario locks `first_turn_prefix`. |

### Validity thresholds (environment variables)
Expand Down
8 changes: 4 additions & 4 deletions docs/cli-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -1087,12 +1087,12 @@ AGENTIC_REPLAY only: upper bound (inclusive) on the random start position within

#### `--burst-phase-starts`

AGENTIC_REPLAY only: collapse the WARMUP-start and PROFILING-start dispatches into synchronized bursts instead of spreading them by each request's recorded offset from t*. By default (False) the phase starts are SPREAD: WARMUP requests are aligned globally so every trajectory reaches its t* at the same instant (the warmup end), and each lane's first PROFILING request waits out its recorded gap after t* -- reproducing the recorded arrival pattern at both phase boundaries. The rest of the replay (inter-turn delays) is timing-faithful regardless of this flag; it governs ONLY the burst-vs-spread of the two phase starts. Pass --burst-phase-starts to fire each phase's first requests together (faster concurrency ramp, synchronized start), e.g. for a throughput-oriented run rather than a faithful arrival replay.
AGENTIC_REPLAY only: collapse the WARMUP-start and PROFILING-start dispatches into synchronized bursts instead of preserving recorded spacing. By default (False), WARMUP requests are aligned globally so every trajectory reaches t* together. PROFILING subtracts one phase-wide minimum from every first request offset: the earliest request starts immediately and all other trajectories retain their recorded spacing. Pass --burst-phase-starts to subtract a separate minimum per lane, making every lane start immediately. Subsequent inter-turn delays are timing-faithful in either mode.
<br/>_Flag (no value required)_

#### `--trace-idle-gap-cap-seconds` `<float>`

Hard ceiling (seconds) for idle gaps within each individual trace. For Weka trace replay, AIPerf looks at all parent and subagent request submission timestamps within one root trace, compresses long gaps between consecutive request submissions, and derives turn delays from the compressed per-trace timeline. Original request api_time values are not used to decide these idle gaps. When set for Weka, this takes precedence over `--inter-turn-delay-cap-seconds` so individual parent/subagent-line delays are not separately capped. Defaults to None (no per-trace idle-gap compression).
Hard ceiling (seconds) for observed runtime idle time within each AgentX trajectory tree. The idle clock covers initial future work and restarts when the final in-flight request across the root and all descendant streams completes. If no request from that tree reaches the wire before the cap, AIPerf uniformly advances only that tree's pending replay timers. Dataset timestamps are unchanged, and request order, spawn/join dependencies, and replay barriers remain authoritative. Defaults to None (no per-trace runtime idle cap).
<br/>_Constraints: ≥ 0.0_

#### `--system-idle-gap-cap-seconds` `<float>`
Expand Down Expand Up @@ -2622,12 +2622,12 @@ AGENTIC_REPLAY only: upper bound (inclusive) on the random start position within

#### `--burst-phase-starts`

AGENTIC_REPLAY only: collapse the WARMUP-start and PROFILING-start dispatches into synchronized bursts instead of spreading them by each request's recorded offset from t*. By default (False) the phase starts are SPREAD: WARMUP requests are aligned globally so every trajectory reaches its t* at the same instant (the warmup end), and each lane's first PROFILING request waits out its recorded gap after t* -- reproducing the recorded arrival pattern at both phase boundaries. The rest of the replay (inter-turn delays) is timing-faithful regardless of this flag; it governs ONLY the burst-vs-spread of the two phase starts. Pass --burst-phase-starts to fire each phase's first requests together (faster concurrency ramp, synchronized start), e.g. for a throughput-oriented run rather than a faithful arrival replay.
AGENTIC_REPLAY only: collapse the WARMUP-start and PROFILING-start dispatches into synchronized bursts instead of preserving recorded spacing. By default (False), WARMUP requests are aligned globally so every trajectory reaches t* together. PROFILING subtracts one phase-wide minimum from every first request offset: the earliest request starts immediately and all other trajectories retain their recorded spacing. Pass --burst-phase-starts to subtract a separate minimum per lane, making every lane start immediately. Subsequent inter-turn delays are timing-faithful in either mode.
<br/>_Flag (no value required)_

#### `--trace-idle-gap-cap-seconds` `<float>`

Hard ceiling (seconds) for idle gaps within each individual trace. For Weka trace replay, AIPerf looks at all parent and subagent request submission timestamps within one root trace, compresses long gaps between consecutive request submissions, and derives turn delays from the compressed per-trace timeline. Original request api_time values are not used to decide these idle gaps. When set for Weka, this takes precedence over `--inter-turn-delay-cap-seconds` so individual parent/subagent-line delays are not separately capped. Defaults to None (no per-trace idle-gap compression).
Hard ceiling (seconds) for observed runtime idle time within each AgentX trajectory tree. The idle clock covers initial future work and restarts when the final in-flight request across the root and all descendant streams completes. If no request from that tree reaches the wire before the cap, AIPerf uniformly advances only that tree's pending replay timers. Dataset timestamps are unchanged, and request order, spawn/join dependencies, and replay barriers remain authoritative. Defaults to None (no per-trace runtime idle cap).
<br/>_Constraints: ≥ 0.0_

#### `--system-idle-gap-cap-seconds` `<float>`
Expand Down
Loading
Loading