fix(client): pace connection establishment; auto warmup claims 25% of the port budget

[viraatc]

Problem

Since #386, the full-auto config (max_connections=-1, warmup_connections=-1) resolves the pool ceiling to the entire ephemeral-port budget (range × distinct_endpoints, ~28k on default Linux). Auto warmup derives from that ceiling (connections_per_worker // 2), and ConnectionPool.warmup() fires the whole count as one unbounded asyncio.gather — so a default run opens ~half the port range as one simultaneous connect burst at t=0 (~14k SYNs to a single endpoint). The max_throughput t=0 wave can grow the pool the same unbounded way through acquire().

That burst is what regressed real runs: it can overflow the server accept queue (SYN drops → seconds of RTO retransmits) and transiently exhaust client ephemeral ports (EADDRNOTAVAIL), and a failed connect currently surfaces as a failed query.

An alternative mitigation was proposed in #407 (halve the auto max_connections). That helps empirically, but it works by indirectly halving the warmup burst: the ceiling only counts live pool members (established + creating), so it cannot bound TIME_WAIT/port pressure in principle — and it permanently halves the client's steady-state connection roofline (silently: excess demand just queues in the pool) to fix a startup-time event. It also loses its protection the moment someone sets an explicit max_connections, since warmup scales off it.

Fix

Keep the full auto ceiling; fix the burst directly:

1. Pace connection establishment — ConnectionPool bounds in-flight connect() calls with a connect limiter (MAX_CONCURRENT_CONNECTS = 128 per pool). This covers both warmup and runtime pool growth, and never touches the pooled-connection reuse path. Connects complete in ~1 RTT, so establishment now proceeds in fast waves instead of one flood — total warmup time is barely affected. _creating is reserved before waiting on the limiter so queued acquires can't overshoot the max_connections ceiling.
2. Bound the auto warm set — when both max_connections and warmup_connections are -1, warmup resolves to 25% of the port budget (AUTO_WARMUP_BUDGET_FRACTION) instead of half the pool (= half the port range). The ceiling stays at the full budget; connections beyond the warm set are opened on demand at runtime (paying the connect cost lazily, now paced). Warming far beyond steady-state concurrency only wastes ports — the LIFO idle stack means excess warm connections sit established at the bottom holding ephemeral ports until shutdown — and inflates the startup burst.
3. Cleanup — removes the dead get_used_port_count() / get_ephemeral_port_limit() helpers (unused since fix: dont discount open tcp-ports against max-connections on init #386); fixes docs/ENDPOINT_CLIENT.md's auto-configuration section, which still described the pre-fix: dont discount open tcp-ports against max-connections on init #386 live-socket subtraction and a wrong 90% figure for min_required_connections (code: 12.5%); pins the regenerate-templates pre-commit hook to the project env via uv run --no-sync (a bare python entry resolves whatever is first on PATH and can regenerate templates against a sibling checkout's schema — which briefly happened on this very branch and was reverted in 09fa7b95).

Unchanged behavior:

• Explicit max_connections and explicit warmup_connections values are untouched (over-budget guard unchanged).
• With an explicit max_connections, auto warmup remains 50% of the pool (worker-side).
• No change to the request hot path; the limiter is only on the pool-growth path.
• YAML config templates are unchanged (verified byte-for-byte against main and round-tripped through BenchmarkConfig.from_yaml_file).

Notes

• On a busy/shared host, actual port contention still surfaces at connect() time as an OSError with no automatic retry — a bounded EADDRNOTAVAIL retry in _create_connection is a natural follow-up to turn transient port pressure into backpressure instead of a failed query (MLPerf run-validity relevance).
• Host-wide simultaneous connects are bounded at 128 × num_workers (e.g. ~2k with 16 workers); the constant is deliberately conservative and easy to revisit with measured errno counts.

Tests

• TestAutoWarmupResolution (new): full-auto resolves warmup to 25% of the budget, scales with distinct endpoints, explicit/disabled warmup preserved, -1 sentinel survives when max_connections is explicit.
• test_connect_pacing_bounds_inflight_creates (new): against the real echo server, peak in-flight connect() never exceeds the limiter during warmup.
• Updated TestEndpointBudgetScaling guard test (dead-helper patch removed).
• tests/unit/endpoint_client/: 79 passed. Template round-trip + template tests: 11 passed. Full unit suite: no new failures vs main (an identical pre-existing set of xdist-parallel failures in metrics_aggregator tests reproduces on clean main).
• Hardened hook verified against the failure mode: with a PATH whose python imports a sibling checkout, pre-commit run regenerate-templates --all-files now produces zero template diff.
• pre-commit run --all-files green.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[gemini-code-assist]

Code Review

This pull request refactors the connection pool and client configuration to improve connection warmup and pacing, and reorganizes timeout settings in the configuration templates. Specifically, it introduces a connection pacing mechanism (MAX_CONCURRENT_CONNECTS) using an asyncio.Semaphore to prevent SYN floods and ephemeral port exhaustion during bursts. It also changes how max_connections=-1 and warmup_connections=-1 are resolved, using a static fraction of the ephemeral port budget (AUTO_WARMUP_BUDGET_FRACTION) instead of dynamically subtracting live socket occupancy, which was racy. Unused port-counting utility functions were removed, and corresponding unit tests and documentation were updated. Finally, global timeout and drain configurations in the YAML templates were consolidated under a new timeouts section. I have no feedback to provide as there are no review comments.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.