Technique suggestion: inspect AI agent session traces after a run

[luoyuctl]

Technique

Inspect AI agent session traces after non-trivial coding runs so you can see where the assistant spent time, burned tokens, retried tools, or got stuck before you start the next iteration.

How I use it

After a Claude Code, Codex CLI, Gemini CLI, Aider, Cursor export, or similar run, I open the local session log in a terminal dashboard and check:

• token and cost spikes
• long gaps or hanging steps
• repeated tool failures and retries
• shallow or anomalous sessions
• session-to-session diffs when comparing two attempts

One concrete workflow:

agenttrace --demo
agenttrace --overview
agenttrace --overview -f json

For CI or team workflows, the same idea can become a lightweight health gate:

agenttrace --overview --fail-under-health 80 --fail-on-critical

Why it is valuable

AI coding assistants can produce a final diff that looks plausible while hiding the actual process: retry loops, stalled tool calls, context waste, or expensive sessions. A short post-run trace review makes the next prompt more grounded: you can ask the assistant to avoid the failing tool path, split the task, reset context, lower cost, or add missing logs/tests.

Tool used

agenttrace: https://github.com/luoyuctl/agenttrace

It is a local-first TUI/CLI for AI coding agent observability across Claude Code, Codex CLI, Gemini CLI, Aider, Cursor exports, and related logs.

[m13v]

the part that breaks down with post-run trace review isn't the data layer, it's that the most expensive failures don't show up as retry loops or token spikes. they show up as the model confidently doing the wrong refactor across many turns, each call cheap and fast. token-and-latency dashboards make those sessions look healthy because the failure is semantic, not mechanical. the diff-vs-prior-attempt view is the only one that catches it, and only if you have a reference attempt that didn't go off the rails.

[luoyuctl]

That is a fair distinction. I would split the technique into two layers:

• use trace review for mechanical/process failures: retry loops, long stalls, token spikes, missing tool calls, or shallow runs
• use session-to-session or diff-vs-reference review for semantic drift, because a cheap and fast run can still confidently move in the wrong direction

So the safer version of the technique is not “tokens and latency prove the run was good”; it is “inspect the trace before the next attempt, and compare against a known-good or previous attempt when the risk is semantic.” That catches a different class of failure than ordinary test output or cost dashboards.