Skip to content

macOS: benchmark run can hang indefinitely with default multiprocessing/data loader settings #1

@aiwantaozi

Description

@aiwantaozi

Description

On macOS, benchmark_runner may hang indefinitely during benchmark run when using default runtime settings.
The process reaches the benchmark phase but does not make progress for several minutes (or longer), and no explicit error is raised.

Environment

  • OS: macOS (Apple Silicon)
  • Python: 3.11
  • benchmark-runner: v0.0.2
    Backend: OpenAI-compatible HTTP endpoint
    Data: synthetic (prompt_tokens=...,output_tokens=...)

Reproduction

python -m benchmark_runner.main benchmark run \
  --target http://<server>:<port> \
  --sample-requests 0 \
  --output-dir .cache \
  --outputs full.json \
  --processor <local_tokenizer_path> \
  --data prompt_tokens=8192,output_tokens=100 \
  --profile constant \
  --rate 1 \
  --max-requests 10

Expected behavior

Benchmark starts and either: progresses normally, or fails fast with a clear error.

Actual behavior

The command appears stuck/hung (often at the benchmark stage) without completion or explicit failure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions