Description
On macOS, benchmark_runner may hang indefinitely during benchmark run when using default runtime settings.
The process reaches the benchmark phase but does not make progress for several minutes (or longer), and no explicit error is raised.
Environment
- OS: macOS (Apple Silicon)
- Python: 3.11
- benchmark-runner: v0.0.2
Backend: OpenAI-compatible HTTP endpoint
Data: synthetic (prompt_tokens=...,output_tokens=...)
Reproduction
python -m benchmark_runner.main benchmark run \
--target http://<server>:<port> \
--sample-requests 0 \
--output-dir .cache \
--outputs full.json \
--processor <local_tokenizer_path> \
--data prompt_tokens=8192,output_tokens=100 \
--profile constant \
--rate 1 \
--max-requests 10
Expected behavior
Benchmark starts and either: progresses normally, or fails fast with a clear error.
Actual behavior
The command appears stuck/hung (often at the benchmark stage) without completion or explicit failure.
Description
On macOS, benchmark_runner may hang indefinitely during benchmark run when using default runtime settings.
The process reaches the benchmark phase but does not make progress for several minutes (or longer), and no explicit error is raised.
Environment
Backend: OpenAI-compatible HTTP endpoint
Data: synthetic (prompt_tokens=...,output_tokens=...)
Reproduction
Expected behavior
Benchmark starts and either: progresses normally, or fails fast with a clear error.
Actual behavior
The command appears stuck/hung (often at the benchmark stage) without completion or explicit failure.