Skip to content

Consolidate --workers flag with auto mode, resilient barrier DNS, and docs#53

Merged
tlrmchlsmth merged 7 commits into
mainfrom
consolidate-workers-flag
May 19, 2026
Merged

Consolidate --workers flag with auto mode, resilient barrier DNS, and docs#53
tlrmchlsmth merged 7 commits into
mainfrom
consolidate-workers-flag

Conversation

@tlrmchlsmth
Copy link
Copy Markdown
Member

@tlrmchlsmth tlrmchlsmth commented May 12, 2026

Summary

  • Replaces --sync '{"workers":N}' and --num-workers N with a single --workers flag
  • --workers auto computes worker count as ceil(max_concurrency / 1024) so each worker handles at most 1024 concurrent streams
  • When --workers > 1: barrier sync is enabled automatically, concurrency and rate are divided across workers so config files always express total desired load
  • Remainder from integer division is distributed to lower-indexed workers (e.g. concurrency=10, workers=3 → 4, 3, 3)
  • Barrier DNS NXDOMAIN is now retried until timeout instead of failing fast after 3 attempts (Kubernetes headless service DNS takes time to propagate)
  • Updated README and CLAUDE.md to document --workers flag, auto mode, and load division

Breaking changes

  • --sync flag removed — barrier sync is now implicit when --workers > 1
  • --num-workers renamed to --workers
  • --workers is now a string flag (accepts integer or "auto", default "1")
  • Concurrency/rate values in configs now mean total, not per-worker

Test plan

  • go build ./cmd/nyann-bench/
  • go test ./... -count=1 — all pass
  • TestResolveWorkers — covers auto resolution, edge cases, invalid inputs
  • TestMaxConcurrency — verifies max concurrency across stages
  • just smoke-test-multi — two workers coordinate with --workers 2
  • Single-worker behavior unchanged (default --workers 1)
  • --workers auto resolves correctly with a multi-stage config

🤖 Generated with Claude Code

tlrmchlsmth and others added 3 commits May 12, 2026 10:02
…ad division

Replace two overlapping flags (--sync JSON and --num-workers) with a single
--workers N flag. When N > 1, barrier sync is enabled automatically and
concurrency/rate are divided across workers so config files always express
total desired load.

Breaking changes:
- --sync flag removed (barrier sync now implicit when --workers > 1)
- --num-workers renamed to --workers
- concurrency/rate in configs now mean total, not per-worker

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Resolve conflict in InsertImplicitBarrier comment (keep new comment
from main that reflects the moved barrier position).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
In Kubernetes, headless service DNS records take a few seconds to
propagate after pod creation. NXDOMAIN is expected early on and
should be retried with backoff (like connection errors), not treated
as a permanent failure after 3 attempts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Copy link
Copy Markdown
Member Author

@tlrmchlsmth tlrmchlsmth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs documentation on how to use it

tlrmchlsmth and others added 3 commits May 18, 2026 22:30
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Update README and CLAUDE.md to replace old --sync '{"workers":N}'
references with the new --workers flag, and document automatic load
division behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
--workers auto sets workers = ceil(max_concurrency / 1024) so each
worker handles at most 1024 concurrent streams. Config is parsed
before the kube branch so auto-resolution works for both local and
Kubernetes deploys.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
@tlrmchlsmth tlrmchlsmth changed the title Consolidate --num-workers and --sync into --workers with auto load division Consolidate --workers flag with auto mode, resilient barrier DNS, and docs May 19, 2026
- Rename `max` variable in MaxConcurrency to avoid shadowing Go builtin
- Add rationale for the 1024 streams-per-worker threshold
- Document that --workers auto sizes for peak stage, which may leave
  workers idle during low-concurrency staircase stages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
@tlrmchlsmth tlrmchlsmth enabled auto-merge (squash) May 19, 2026 02:54
@tlrmchlsmth tlrmchlsmth merged commit ddae7b7 into main May 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant