Skip to content

Commit 7f893fb

Browse files
committed
Add trace replay documentation
Signed-off-by: Vincent Gimenes <vincent.gimenes@gmail.com>
1 parent 4a44831 commit 7f893fb

2 files changed

Lines changed: 27 additions & 0 deletions

File tree

docs/getting-started/benchmark.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ GuideLLM supports several benchmark profiles and strategies:
6161
- `constant`: Sends requests at a fixed rate per second
6262
- `poisson`: Sends requests following a Poisson distribution
6363
- `sweep`: Automatically determines optimal performance points (default)
64+
- `replay`: Replays requests from a trace file to reproduce real-world traffic patterns
6465

6566
### Data Options
6667

@@ -76,6 +77,27 @@ For a complete list of options, run:
7677
guidellm benchmark run --help
7778
```
7879

80+
## Trace Replay Benchmarking
81+
82+
For realistic load testing, replay traffic patterns from trace files. Trace files must be JSONL with `timestamp`, `input_length`, and `output_length` fields:
83+
84+
```json
85+
{"timestamp": 0, "input_length": 256, "output_length": 128}
86+
{"timestamp": 0.5, "input_length": 512, "output_length": 64}
87+
```
88+
89+
Run with the `replay` profile:
90+
91+
```bash
92+
guidellm benchmark \
93+
--target "http://localhost:8000" \
94+
--data "path/to/trace.jsonl|trace_synthetic" \
95+
--profile replay \
96+
--rate 1.0
97+
```
98+
99+
The `rate` parameter acts as a time scale: `1.0` for original speed, `2.0` for 2x faster, `0.5` for half speed.
100+
79101
## Working with Real Data
80102

81103
While synthetic data is convenient for quick tests, you can benchmark with real-world data:

docs/guides/datasets.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,11 @@ GuideLLM supports various file formats for datasets, including text, CSV, JSON,
131131
{"prompt": "Hello, how are you?", "output_tokens_count": 5, "additional_column": "foo", "additional_column2": "bar"}
132132
{"prompt": "What is your name?", "output_tokens_count": 3, "additional_column": "baz", "additional_column2": "qux"}
133133
```
134+
- **Trace files (`.jsonl` with `trace_synthetic` type)**: Specialized JSONL files for replay benchmarking with `timestamp`, `input_length`, and `output_length` fields. Used with `--profile replay` to reproduce production traffic patterns. See [Trace Replay Benchmarking](../getting-started/benchmark.md#trace-replay-benchmarking).
135+
```json
136+
{"timestamp": 0, "input_length": 256, "output_length": 128}
137+
{"timestamp": 0.5, "input_length": 512, "output_length": 64}
138+
```
134139
- **JSON files (`.json`)**: Where the entire dataset is represented as a JSON array of objects nested under a specific key. To surface the correct key to use, a `--data-column-mapper` argument must be passed in of `"field": "NAME"` for where the array exists. The objects should include `prompt` or other common names for the prompt which will be used as the prompt column. Additional fields can be included based on the previously mentioned aliases for the `--data-column-mapper` argument.
135140
```json
136141
{

0 commit comments

Comments
 (0)