Skip to content

Commit 871d39a

Browse files
ushaketcursoragent
andcommitted
Address review feedback: fix docs, comments, and docstring
- Simplify _should_stop_escalating param docstring (not a precondition) - Consolidate sweep.py early-exit comments into one clear block - Revert README multi-rate sentence, move docs to getting-started/benchmark.md - Add early-exit behavior notes to Concurrent, Constant, Poisson, Sweep sections Signed-off-by: Uri Shaket <ushaket@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Uri Shaket <ushaket@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 634455d commit 871d39a

4 files changed

Lines changed: 13 additions & 9 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ guidellm run \
180180
**Key parameters:**
181181

182182
- `--profile kind=<type>`: Defines the traffic pattern — `synchronous`, `concurrent`, `throughput`, `constant`, `poisson`, or `sweep`
183-
- `--profile kind=constant,rate=10`: For `constant`/`poisson`, set requests per second in the profile config; for `concurrent`, use `streams=`; for `throughput`, use `max_concurrency=`. For `constant`, `poisson`, and `concurrent`, multiple rates can be specified as a JSON list (e.g., `rate=[1,5,10]`). Rates are sorted ascending, and if a failure constraint (over-saturation, errors) triggers at a given rate, remaining higher rates are skipped.
183+
- `--profile kind=constant,rate=10`: For `constant`/`poisson`, set requests per second in the profile config; for `concurrent`, use `streams=`; for `throughput`, use `max_concurrency=`
184184
- `--constraint kind=max_duration,seconds=<seconds>` or `--constraint kind=max_requests,count=<count>`: Limit each strategy by time or request count
185185

186186
### Dataset Sources

docs/getting-started/benchmark.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,8 @@ guidellm run --profile kind=concurrent,streams=10
141141
| `rampup_duration` | Seconds to spread initial requests | `--profile kind=concurrent,streams=10,rampup_duration=10` |
142142
| `max_concurrency` | Maximum concurrent requests to schedule | `--profile kind=concurrent,streams=10,max_concurrency=10` |
143143

144+
When multiple stream counts are specified, they are sorted in ascending order. If a failure constraint (such as over-saturation or excessive errors) triggers at a given stream count, all remaining higher stream counts are skipped.
145+
144146
#### Constant Profile
145147

146148
Sends asynchronous requests at a fixed rate per second.
@@ -157,6 +159,8 @@ guidellm run --profile '{"kind":"constant","rate":[16,32]}'
157159
| `rampup_duration` | Seconds to linearly ramp from 0 to target rate | `--profile kind=constant,rate=10,rampup_duration=10` |
158160
| `max_concurrency` | Maximum concurrent requests to schedule | `--profile kind=constant,rate=10,max_concurrency=32` |
159161

162+
When multiple rates are specified, they are sorted in ascending order. If a failure constraint (such as over-saturation or excessive errors) triggers at a given rate, all remaining higher rates are skipped.
163+
160164
#### Poisson Profile
161165

162166
Sends asynchronous requests at varying rates using a Poisson distribution around the specified target rate(s). This probabilistic pattern is useful for simulating more realistic real-world traffic patterns.
@@ -170,6 +174,8 @@ guidellm run --profile kind=poisson,rate=16 --seed kind=static,value=42
170174
| `rate` | Target rate(s) in requests per second | `--profile kind=poisson,rate=10` or `--profile '{"kind":"poisson","rate":[10,20]}'` |
171175
| `max_concurrency` | Maximum concurrent requests to schedule | `--profile kind=poisson,rate=10,max_concurrency=32` |
172176

177+
When multiple rates are specified, they are sorted in ascending order. If a failure constraint (such as over-saturation or excessive errors) triggers at a given rate, all remaining higher rates are skipped.
178+
173179
Use `--seed kind=static,value=42` for reproducible Poisson scheduling.
174180

175181
#### Sweep Profile
@@ -178,7 +184,7 @@ The sweep profile applies a sequence of benchmark strategies to find the optimal
178184

179185
1. It runs a `synchronous` strategy to measure the baseline rate,
180186
2. then runs a `throughput` strategy to determine peak throughput,
181-
3. and finally runs a series of asynchronous strategies with rates interpolated between the baseline and maximum throughput. (The number of interpolated strategies is `sweep_size` minus 2.) The asynchronous strategy type is determined by the `strategy_type` profile parameter. The default strategy type is `constant`.
187+
3. and finally runs a series of asynchronous strategies with rates interpolated between the baseline and maximum throughput. (The number of interpolated strategies is `sweep_size` minus 2.) The asynchronous strategy type is determined by the `strategy_type` profile parameter. The default strategy type is `constant`. During the async phase, if a failure constraint triggers at a given rate, all remaining higher rates are skipped.
182188

183189
For example, to run a sweep with 10 strategies, 10 seconds of rampup, and a strategy type of `poisson`:
184190

src/guidellm/benchmark/profiles/profile.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ def _should_stop_escalating(prev_benchmark: Benchmark) -> bool:
127127
Constraints that use "stop_local" (max duration, max requests) are normal
128128
completions and do not trigger escalation stops.
129129
130-
:param prev_benchmark: Benchmark instance with a scheduler_state attribute
130+
:param prev_benchmark: Benchmark instance
131131
:return: True if a failure constraint was triggered, False otherwise
132132
"""
133133
scheduler_state = getattr(prev_benchmark, "scheduler_state", None)

src/guidellm/benchmark/profiles/sweep.py

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -131,13 +131,11 @@ def next_strategy(
131131
self.args.sweep_size - 1,
132132
)
133133
)[1:] # don't rerun synchronous
134-
# After throughput, fall through to async rate logic below.
135-
# Don't check escalation since throughput is designed to push
136-
# beyond sustainable load (over-saturation is expected).
137134

138-
# Stop escalation if a failure constraint was triggered.
139-
# The throughput guard above skips this via the != "throughput" check.
140-
# Synchronous never reaches here (returns ThroughputStrategy above).
135+
# Stop escalation if a failure constraint was triggered during the
136+
# async phase. Throughput is excluded (type_ != "throughput") because
137+
# it intentionally pushes beyond sustainable load. Synchronous never
138+
# reaches here — it returns a ThroughputStrategy above.
141139
if (
142140
prev_strategy.type_ != "throughput"
143141
and self._should_stop_escalating(prev_benchmark) # type: ignore[arg-type]

0 commit comments

Comments
 (0)