Skip to content

Commit f1277b2

Browse files
committed
feat(cli): add time-barrier wave staggering to e2e batch fan-out
The e2e CodeBuild batch fans out a large number of shards that all depend only on `upb` (Linux) or `build_windows`/`upb` (Windows). Dispatching that many builds as a single instantaneous burst is unreliable, so this change spreads dispatch over time. This change spreads dispatch over time using TIME barriers rather than completion chaining. Shards are grouped into waves of `E2E_WAVE_SIZE` (per OS). Wave 1 keeps its original dependency on `upb`/`build_windows`. Each later wave is gated behind a synthetic barrier build whose only job is to sleep `(k-1) * E2E_WAVE_BARRIER_INTERVAL_SEC` seconds. Crucially, every barrier depends ONLY on `upb`/`build_windows` — never on the prior barrier or the prior wave — so all barriers start in parallel right after the package upload and the stagger comes purely from their differing sleep durations. This cuts the per-scheduling-decision dispatch burst without serializing wave completion, deliberately avoiding the serial completion chain that timed out at the 240m batch limit previously. Both knobs are tunable constants near the top of the generator. Barriers are excluded from the e2e report wait list (`wait_for_ids.json`) since they are synthetic and must not be polled by `aggregate_e2e_reports`. Verified by regenerating `e2e_workflow_generated.yml`: 4 barriers present (l/w waves 2 and 3), waves reference their barrier, wave 1 still on `upb`/`build_windows`, no shard depends on another shard, no barrier depends on another barrier, and the yaml parses cleanly. --- Prompt: implement barrier-wave staggering, draft PR from dev, run e2e
1 parent aa9d525 commit f1277b2

2 files changed

Lines changed: 300 additions & 160 deletions

File tree

0 commit comments

Comments
 (0)