When a trace forks, RolloutLimits double-counts the shared prefix and stops the rollout too early.
RolloutLimits.reached() checks trace.num_input_tokens / num_total_tokens, and those sum each branch's counts (trace.py). Forked branches share the same prefix nodes, so the prefix is counted once per branch. A two-branch rollout sees ~2x its real input on the shared part, and trips the cap before it actually should.
Minimal case (shared prompt prefix, two sampled leaves):
branch[0] input=2 branch[1] input=2
unique graph input = 2
trace.num_input_tokens (summed) = 4
max_input_tokens = 3 -> reached() refuses (max_input_tokens); real usage is within budget
The check runs before each turn, so once a fork exists mid-rollout (subagents, compaction, retokenization drift) later turns get refused early.
The reason I'm filing instead of just sending a patch: there's no obviously-correct number.
- summing branches over-counts the shared prefix (current behavior)
- max over branches under-counts genuinely parallel work (N subagents each 50k -> reports 50k)
- provider
usage can't recover the unique input footprint either — each branch's prompt_tokens re-includes the prefix
So it comes down to what the caps are meant to bound. Roughly:
- unique context footprint (dedupe shared graph nodes; token-id based, training-only), or
- provider cost (re-sent prefix is real work and should count repeatedly), or
- keep usage-based but attribute incremental input per unique node/call.
I have a repro and a prototype for (1). Happy to send a PR once you decide which semantics you want — didn't want to bake in a behavior change unilaterally.
When a trace forks,
RolloutLimitsdouble-counts the shared prefix and stops the rollout too early.RolloutLimits.reached()checkstrace.num_input_tokens/num_total_tokens, and those sum each branch's counts (trace.py). Forked branches share the same prefix nodes, so the prefix is counted once per branch. A two-branch rollout sees ~2x its real input on the shared part, and trips the cap before it actually should.Minimal case (shared prompt prefix, two sampled leaves):
The check runs before each turn, so once a fork exists mid-rollout (subagents, compaction, retokenization drift) later turns get refused early.
The reason I'm filing instead of just sending a patch: there's no obviously-correct number.
usagecan't recover the unique input footprint either — each branch'sprompt_tokensre-includes the prefixSo it comes down to what the caps are meant to bound. Roughly:
I have a repro and a prototype for (1). Happy to send a PR once you decide which semantics you want — didn't want to bake in a behavior change unilaterally.