Commit 772a6b1
ci(smoke): add token-usage sanity checks to smoke workflows
Add a verify_token_usage job to smoke-copilot, smoke-claude, and
smoke-codex that runs after the agent job on the downloaded agent
artifact and fails the workflow when token accounting looks wrong.
The checker (scripts/ci/check-token-usage.js) enforces two invariants:
- Internal consistency: the sum of per-response records in
token-usage.jsonl must exactly equal the aggregated agent_usage.json
(input/output/cache_read/cache_write). This is engine-independent.
- cache_read_tokens must not be 0 across multiple responses, which is
the symptom of the cached-token normalization bug.
ai_credits/ambient_context drift is reported as warnings only.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 8c46503 commit 772a6b1
8 files changed
Lines changed: 713 additions & 22 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
26 | 45 | | |
27 | 46 | | |
28 | 47 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
25 | 44 | | |
26 | 45 | | |
27 | 46 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
48 | 67 | | |
49 | 68 | | |
50 | 69 | | |
| |||
0 commit comments