Skip to content

Commit 6d7eac5

Browse files
committed
docs: clarify trace replay timestamp semantics
Signed-off-by: Vincent Gimenes <vincent.gimenes@gmail.com>
1 parent 09dcb32 commit 6d7eac5

2 files changed

Lines changed: 18 additions & 14 deletions

File tree

docs/getting-started/benchmark.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -190,29 +190,31 @@ You can customize synthetic data generation with additional parameters such as s
190190

191191
### Trace Replay Benchmarking (beta)
192192

193-
For realistic load testing, replay trace events using each row's timestamp and token lengths. Trace files must be JSONL and are loaded with the `trace_synthetic` data type. By default, each row uses `timestamp`, `input_length`, and `output_length` fields:
193+
For realistic load testing, replay trace events using each row's timestamp and token lengths. Trace files must be JSONL and are loaded with the `trace_synthetic` data type. By default, each row uses `timestamp`, `input_length`, and `output_length` fields. Timestamps may be absolute or monotonic values; GuideLLM sorts them and converts them to offsets from the first event before scheduling:
194194

195195
```json
196-
{"timestamp": 0, "input_length": 256, "output_length": 128}
197-
{"timestamp": 0.5, "input_length": 512, "output_length": 64}
196+
{"timestamp": 1234500.0, "input_length": 256, "output_length": 128}
197+
{"timestamp": 1234500.5, "input_length": 512, "output_length": 64}
198198
```
199199

200-
Run with the `replay` profile. This example also maps custom trace column names:
200+
In this example, the second request is scheduled 0.5 seconds after the first request.
201+
202+
Run with the `replay` profile:
201203

202204
```bash
203205
guidellm benchmark \
204206
--target "http://localhost:8000" \
205207
--data "path/to/trace.jsonl" \
206-
--data-args '{"type_": "trace_synthetic", "timestamp_column": "ts", "prompt_tokens_column": "input_tokens", "output_tokens_column": "output_tokens"}' \
208+
--data-args '{"type_": "trace_synthetic"}' \
207209
--profile replay \
208210
--rate 1.0
209211
```
210212

211-
The `--rate` parameter acts as a time scale, not requests per second: `1.0` for original speed, `2.0` to multiply timestamps by 2 and run twice as long, `0.5` to multiply timestamps by 0.5 and run twice as fast.
213+
The `--rate` parameter acts as a time scale for the intervals between trace events, not requests per second: `1.0` preserves the original timing, `2.0` doubles the intervals and runs twice as long, and `0.5` halves the intervals and runs twice as fast.
212214

213215
GuideLLM orders trace rows by timestamp before scheduling and payload generation, so each scheduled event uses the token lengths from the same sorted row. Use `--data-samples` to limit how many trace rows are loaded and replayed. `--max-requests` remains a runtime completion constraint; it does not truncate the trace dataset.
214216

215-
If your trace uses the default column names shown above, omit `timestamp_column`, `prompt_tokens_column`, and `output_tokens_column` from `--data-args`.
217+
If your trace uses different column names, map them with `timestamp_column`, `prompt_tokens_column`, and `output_tokens_column` in `--data-args`.
216218

217219
### Working with Real Data
218220

docs/guides/datasets.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -141,27 +141,29 @@ GuideLLM supports various file formats for datasets, including text, CSV, JSON,
141141
{"prompt": "What is your name?", "output_tokens_count": 3, "additional_column": "baz", "additional_column2": "qux"}
142142
```
143143

144-
- **Trace files (`.jsonl` with `trace_synthetic` type)**: Specialized JSONL files for replay benchmarking with `timestamp`, `input_length`, and `output_length` fields. Used with `--profile replay` to replay trace events using each row's timestamp and token lengths. See [Trace Replay Benchmarking](../getting-started/benchmark.md#trace-replay-benchmarking).
144+
- **Trace files (`.jsonl` with `trace_synthetic` type)**: Specialized JSONL files for replay benchmarking with `timestamp`, `input_length`, and `output_length` fields. Used with `--profile replay` to replay trace events using each row's timestamp and token lengths. Timestamps may be absolute or monotonic values; GuideLLM sorts them and converts them to offsets from the first event before scheduling. See [Trace Replay Benchmarking](../getting-started/benchmark.md#trace-replay-benchmarking).
145145

146146
```json
147-
{"timestamp": 0, "input_length": 256, "output_length": 128}
148-
{"timestamp": 0.5, "input_length": 512, "output_length": 64}
147+
{"timestamp": 1234500.0, "input_length": 256, "output_length": 128}
148+
{"timestamp": 1234500.5, "input_length": 512, "output_length": 64}
149149
```
150150

151-
Trace rows are ordered by timestamp before GuideLLM schedules requests and generates synthetic payloads. This keeps each scheduled event aligned with the prompt and output token lengths from the same row.
151+
In this example, the second request is scheduled 0.5 seconds after the first request. Trace rows are ordered by timestamp before GuideLLM schedules requests and generates synthetic payloads. This keeps each scheduled event aligned with the prompt and output token lengths from the same row.
152152

153-
Use `--data-args '{"type_": "trace_synthetic"}'` to enable trace loading. If your trace uses different column names, configure them with `timestamp_column`, `prompt_tokens_column`, and `output_tokens_column`:
153+
Use `--data-args '{"type_": "trace_synthetic"}'` to enable trace loading:
154154

155155
```bash
156156
guidellm benchmark \
157157
--target "http://localhost:8000" \
158158
--profile replay \
159159
--rate 1.0 \
160160
--data "path/to/trace.jsonl" \
161-
--data-args '{"type_": "trace_synthetic", "timestamp_column": "ts", "prompt_tokens_column": "input_tokens", "output_tokens_column": "output_tokens"}'
161+
--data-args '{"type_": "trace_synthetic"}'
162162
```
163163

164-
For replay, `--rate` is a time scale rather than requests per second. Use `--data-samples` to limit how many trace rows are loaded and replayed. Use `--max-requests` only as a runtime completion constraint; it does not limit the trace rows loaded from the file.
164+
If your trace uses different column names, configure them with `timestamp_column`, `prompt_tokens_column`, and `output_tokens_column`.
165+
166+
For replay, `--rate` is a time scale for the intervals between trace events rather than requests per second. Use `--data-samples` to limit how many trace rows are loaded and replayed. Use `--max-requests` only as a runtime completion constraint; it does not limit the trace rows loaded from the file.
165167

166168
- **JSON files (`.json`)**: Where the entire dataset is represented as a JSON array of objects nested under a specific key. To surface the correct key to use, a `--data-column-mapper` argument must be passed in of `"field": "NAME"` for where the array exists. The objects should include `prompt` or other common names for the prompt which will be used as the prompt column. Additional fields can be included based on the previously mentioned aliases for the `--data-column-mapper` argument.
167169

0 commit comments

Comments
 (0)