You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All 5 audited runs consumed 3.6–7.1 M effective tokens to produce an issue confirming there was nothing to report. The log collection step (gh aw logs --start-date -1d) has consistently returned an empty .runs array, meaning every run executes the full four-phase workflow — Python scripting, snapshot persistence, chart generation attempts, and full template issue creation — before concluding with zero findings.
#1 — Enforce early exit after Phase 1 when the log window is empty (still unimplemented from June 9)
Estimated AIC savings: 20–30% per empty-data run
Evidence: 5/5 audited runs detected len(runs) == 0. Despite having nothing to report, every run completes the full workflow: Phase 1 writes and runs a Python script, Phase 2 copies an empty snapshot, Phase 3 attempts chart generation (skipped or generates empty output), and Phase 4 fills the entire report template with zero-valued placeholders. Issue footers confirm 3.6–7.1 M effective tokens consumed per run for this zero-value work. The June 9 optimizer issue first identified this pattern; it has not been acted on.
Action: After Phase 1 completes, add a conditional branch at the top of Phase 2:
If `audit_snapshot.json` shows `overall.total_runs == 0` and the raw `.runs` array was empty,
immediately publish the following minimal issue and exit — skip Phases 2, 3, and 4:
title: "Daily AIC Usage Audit — {YYYY-MM-DD}"
body:
> ⚠️ No agentic workflow runs captured in the last 24 hours.
> `gh aw logs --start-date -1d` returned an empty `.runs` array.
> No snapshot, charts, or trend entry recorded.
Eliminating Phases 2–4 on empty-data runs removes the Python snapshot copy, rolling-summary update, chart-generation attempt, and full template expansion — the primary sources of wasted effective tokens.
#2 — Prune the RunData schema table to used fields only
Estimated AIC savings: 10–12% per run
Evidence: The "Data Sources" section contains a 14-row schema table. Only 7 of those fields are referenced in Phase 1 processing or Phase 4 reporting (workflow_name, aic, token_usage, turns, error_count, warning_count, status, conclusion). The remaining fields — effective_tokens (marked "deprecated; use aic for billing"), url, duration, created_at, run_id, token_usage_summary — are described in detail but unused in the processing logic.
Action: Replace the 14-row RunData table with an 8-row table containing only the actively used fields. Remove the deprecated effective_tokens row entirely. The pruned table removes approximately 18 lines of dense context from every run.
Proposed replacement table
| Field | Type | Notes ||---|---|---||`workflow_name`| string | Human-readable name ||`aic`| float | AI Credits consumed (primary billing metric) ||`token_usage`| int | Total tokens (treat missing/null as 0) ||`turns`| int | Number of agent turns ||`error_count`| int | Errors encountered ||`warning_count`| int | Warnings encountered ||`status`| string |`completed`, `in_progress`, etc. ||`conclusion`| string |`success`, `failure`, etc. |
#3 — Compress the OTEL experiment span section to a 4-line note
Estimated AIC savings: 6–8% per run
Evidence: Lines 276–299 of agentic-token-audit.md contain a 25-line inline JavaScript template for emitting OTEL span attributes. This section fires only when /tmp/gh-aw/experiments/assignments.json exists. No observed run produced an experiment-related event in its step summary, indicating the assignments file was absent in every run. The full JavaScript template adds ~350 tokens of scaffolding to the prompt on every run regardless.
Action: Replace the 25-line OTEL code block with a 4-line prose description:
If `/tmp/gh-aw/experiments/assignments.json` exists, emit one OTEL span attribute per
assignment key using `otlp.cjs` `logSpan`: set `gh_aw.experiment.<name>` to the assigned
variant for each name, and `gh_aw.experiment.names` to a comma-separated list of all names.
Skip this step if the file is absent.
#4 — Remove "Important Notes" section; inline the one unique directive into Phase 2
Estimated AIC savings: 3–4% per run
Evidence: The "Important Notes" section (lines 263–272) contains three directives already present elsewhere:
Null coalescing → already in Phase 1 ("Handle null/missing aic and token_usage by treating them as 0").
Empty vs. zero-completed run distinction → already in Phase 2 conditional logic.
"Keep the issue concise" → editorial guidance with no structural effect.
All three are redundant. The section adds ~11 lines of context with no unique informational value.
Action: Delete the "Important Notes" section entirely. No content is lost — the Phase 1 and Phase 2 instructions already cover both substantive points.
🔧 Structural Optimization: Inline Sub-Agent for Chart Generation
Phase 3 (Generate Charts) qualifies as a strong inline sub-agent candidate. The workflow has no existing ## agent: blocks.
Dimension
Score
Rationale
Independence
2 / 3
Reads from files written to disk by Phase 1; separable once Phase 1 completes
Small-model adequacy
3 / 3
Pure data-formatting and chart creation; no strategic synthesis required
Parallelism
1 / 2
Must wait for Phase 1 output files, but can overlap with Phase 4 issue-writing
Size
2 / 2
Two distinct chart types with specific technical requirements (PYTHONPATH, dual-axis, 300 DPI)
Total
8 / 10 — Strong candidate
The chart generation task is entirely extractive: read two JSON files, produce two PNG files using matplotlib/seaborn, upload them, return two URLs. A smaller model handles this without the full reasoning capability of the main agent.
Note: Implement alongside or after Recommendation #1 to ensure the sub-agent is not invoked on empty-data runs.
Proposed prompt change — replace the Phase 3 body with:
## agent: charts (model: haiku)
Read `/tmp/gh-aw/token-audit/audit_snapshot.json` (workflow AIC totals) and
`/tmp/gh-aw/repo-memory/default/rolling-summary.json` (historical entries).
1. If fewer than 2 rolling-summary entries exist, skip the trend chart; return `null` for its URL.
2. Generate `ai_credits_by_workflow.png`: horizontal bar chart, top 15 workflows by `total_ai_credits`, 300 DPI, white background.
3. Generate `ai_credits_trend.png`: dual-axis line chart — primary y-axis `total_ai_credits`, secondary y-axis `active_workflows` labeled "Active workflows/day". 300 DPI, white background.
4.`export PYTHONPATH=/tmp/gh-aw/token-audit/site-packages` before any Python command.
5. Upload each PNG via `upload_asset`. Return the two URLs.
⚠️ Caveats
AIC billing figures are 0.00 across all snapshots; savings estimates are derived from effective-token footprints visible in issue footers (3.6–7.1 M/run) and the June 9 optimizer baseline (0.27 AIC/run ≈ 12% prompt reduction).
5 runs analyzed; the empty-data pattern may resolve independently if the gh aw logs data pipeline is fixed upstream.
The chart sub-agent recommendation delivers savings only when real data is present; all 5 recent runs had no data and chart generation was already being skipped.
Target workflow: Daily Agentic Workflow AIC Usage Audit (
agentic-token-audit.md)Reason selected: Highest-frequency AI workflow not self-targeted; prior June 9 critical recommendation (early exit on empty data) remains unimplemented; fresh June 10–11 run evidence confirms persistent waste pattern.
Analysis period: 2026-06-05 → 2026-06-11 · 5 runs audited
📊 Spend Profile
All 5 audited runs consumed 3.6–7.1 M effective tokens to produce an issue confirming there was nothing to report. The log collection step (
gh aw logs --start-date -1d) has consistently returned an empty.runsarray, meaning every run executes the full four-phase workflow — Python scripting, snapshot persistence, chart generation attempts, and full template issue creation — before concluding with zero findings.Per-run evidence
References: §27351174099, §27280081885, §27209129613
🏆 Ranked Recommendations
#1 — Enforce early exit after Phase 1 when the log window is empty (still unimplemented from June 9)
Estimated AIC savings: 20–30% per empty-data run
Evidence: 5/5 audited runs detected
len(runs) == 0. Despite having nothing to report, every run completes the full workflow: Phase 1 writes and runs a Python script, Phase 2 copies an empty snapshot, Phase 3 attempts chart generation (skipped or generates empty output), and Phase 4 fills the entire report template with zero-valued placeholders. Issue footers confirm 3.6–7.1 M effective tokens consumed per run for this zero-value work. The June 9 optimizer issue first identified this pattern; it has not been acted on.Action: After Phase 1 completes, add a conditional branch at the top of Phase 2:
Eliminating Phases 2–4 on empty-data runs removes the Python snapshot copy, rolling-summary update, chart-generation attempt, and full template expansion — the primary sources of wasted effective tokens.
#2 — Prune the RunData schema table to used fields only
Estimated AIC savings: 10–12% per run
Evidence: The "Data Sources" section contains a 14-row schema table. Only 7 of those fields are referenced in Phase 1 processing or Phase 4 reporting (
workflow_name,aic,token_usage,turns,error_count,warning_count,status,conclusion). The remaining fields —effective_tokens(marked "deprecated; useaicfor billing"),url,duration,created_at,run_id,token_usage_summary— are described in detail but unused in the processing logic.Action: Replace the 14-row RunData table with an 8-row table containing only the actively used fields. Remove the deprecated
effective_tokensrow entirely. The pruned table removes approximately 18 lines of dense context from every run.Proposed replacement table
#3 — Compress the OTEL experiment span section to a 4-line note
Estimated AIC savings: 6–8% per run
Evidence: Lines 276–299 of
agentic-token-audit.mdcontain a 25-line inline JavaScript template for emitting OTEL span attributes. This section fires only when/tmp/gh-aw/experiments/assignments.jsonexists. No observed run produced an experiment-related event in its step summary, indicating the assignments file was absent in every run. The full JavaScript template adds ~350 tokens of scaffolding to the prompt on every run regardless.Action: Replace the 25-line OTEL code block with a 4-line prose description:
#4 — Remove "Important Notes" section; inline the one unique directive into Phase 2
Estimated AIC savings: 3–4% per run
Evidence: The "Important Notes" section (lines 263–272) contains three directives already present elsewhere:
aicandtoken_usageby treating them as 0").All three are redundant. The section adds ~11 lines of context with no unique informational value.
Action: Delete the "Important Notes" section entirely. No content is lost — the Phase 1 and Phase 2 instructions already cover both substantive points.
🔧 Structural Optimization: Inline Sub-Agent for Chart Generation
Phase 3 (Generate Charts) qualifies as a strong inline sub-agent candidate. The workflow has no existing
## agent:blocks.The chart generation task is entirely extractive: read two JSON files, produce two PNG files using matplotlib/seaborn, upload them, return two URLs. A smaller model handles this without the full reasoning capability of the main agent.
Note: Implement alongside or after Recommendation #1 to ensure the sub-agent is not invoked on empty-data runs.
Proposed prompt change — replace the Phase 3 body with:
gh aw logsdata pipeline is fixed upstream.