Problem
The "Overhead Time" metric in eval output frequently shows negative values:
SDK Duration: 3.77s
API Call Time: 11.71s
Overhead Time: -7938ms
The formula is sdkDurationMs - apiDurationMs (metrics.ts lines 266, 293), which assumes API time is a subset of SDK time. But the Agent SDK can make parallel API calls across multiple models, so aggregate API time exceeds wall-clock SDK time.
Suggestion
Options:
- Remove the overhead metric — it's not meaningful with parallel execution
- Rename to "API vs Wall-clock Delta" and clarify in the display that negative means parallel execution
- Clamp to zero with a note: `Math.max(0, sdkDuration - apiDuration)`
Files
- `evals/lib/metrics.ts` (lines 266, 293)
Problem
The "Overhead Time" metric in eval output frequently shows negative values:
The formula is
sdkDurationMs - apiDurationMs(metrics.ts lines 266, 293), which assumes API time is a subset of SDK time. But the Agent SDK can make parallel API calls across multiple models, so aggregate API time exceeds wall-clock SDK time.Suggestion
Options:
Files