Skip to content

Commit de73c25

Browse files
authored
Token tracking (#58)
* feat: session token tracking with /tokens command and exit summary Track cumulative input/output/cache tokens, request count, and turn count across the session. Display via: - /tokens command — show session totals at any time - Exit summary — printed at all exit points (interactive, --prompt, SIGINT) - Per-turn inline display unchanged (📊 line) New state fields: totalInputTokens, totalOutputTokens, totalCacheReadTokens, totalRequests, totalTurns Accumulation happens in the assistant.usage event handler. Turn count incremented on each onUserPromptSubmitted. Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * feat: teach LLM to build reusable modules proactively Add guidance to system message encouraging the LLM to use register_module when it identifies missing capabilities rather than just describing gaps. Modules persist across sessions. Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: update sourceHash on register_module to prevent hash mismatch warnings When register_module saves a user module, it now writes the sourceHash to the .json metadata. Previously only loadModuleAsync set this field, so re-registering a module with updated code would leave a stale hash that the validator flagged as a mismatch warning. Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> * fix: address PR #58 review feedback 1. event-handler.ts: guard against undefined token values before accumulating 2. llm-output.ts: format large token numbers with toLocaleString, fix alignment 3. index.ts: use guard function for exit summary (no tokens = no summary) 4. state.ts: add doc comment clarifying totalRequests tracks premium requests 5. module-store.ts: use computeTruncatedHash instead of manual slice Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com> --------- Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
1 parent b218169 commit de73c25

File tree

9 files changed

+111
-927
lines changed

9 files changed

+111
-927
lines changed

package-lock.json

Lines changed: 7 additions & 927 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

src/agent/commands.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,13 @@ const COMMANDS: readonly CommandEntry[] = Object.freeze([
5151
"Enables verbose SDK event logging (same as HYPERAGENT_DEBUG=1).\n" +
5252
"Shows every event the agent receives from the Copilot SDK.",
5353
},
54+
{
55+
completion: "/tokens",
56+
help: "Show session token usage summary",
57+
detail:
58+
"Displays cumulative input/output/cache tokens, request count,\n" +
59+
"and turn count for the current session. Also shown on exit.",
60+
},
5461
{
5562
completion: "/reasoning conversation ",
5663
help: "Set conversation reasoning effort (low|medium|high|xhigh)",

src/agent/event-handler.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -429,6 +429,14 @@ export function registerEventHandler(
429429
cost?: number;
430430
duration?: number;
431431
};
432+
433+
// Accumulate session totals. Count one request per usage event;
434+
// usageData.cost is premium request count, not a reliable API-call counter.
435+
state.totalInputTokens += usageData.inputTokens ?? 0;
436+
state.totalOutputTokens += usageData.outputTokens ?? 0;
437+
state.totalCacheReadTokens += usageData.cacheReadTokens ?? 0;
438+
state.totalRequests += 1;
439+
432440
// Ensure stats appear on a new line — streamed
433441
// message_delta writes don't end with \n.
434442
if (state.streamedContent) {

src/agent/index.ts

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ import {
116116
renderReasoningTransition,
117117
printUsageStats,
118118
printExtendedReasoningNotice,
119+
formatTokenSummary,
119120
} from "./llm-output.js";
120121

121122
// ── Session Timing ───────────────────────────────────────────────────
@@ -145,6 +146,19 @@ function formatSessionDuration(): string {
145146
return seconds > 0 ? `${minutes}m ${seconds}s` : `${minutes}m`;
146147
}
147148

149+
/**
150+
* Print session token summary to console.
151+
* Called at all exit points (interactive, --prompt, SIGINT).
152+
*/
153+
function printExitTokenSummary(): void {
154+
if (state.totalInputTokens === 0 && state.totalOutputTokens === 0) return;
155+
const lines = formatTokenSummary(state);
156+
console.log(); // blank line before summary
157+
for (const line of lines) {
158+
console.log(` ${line}`);
159+
}
160+
}
161+
148162
// ── Paths ────────────────────────────────────────────────────────────
149163

150164
const __agentFilename = fileURLToPath(import.meta.url);
@@ -4482,6 +4496,9 @@ function buildSessionConfig() {
44824496
// Guidance is stored in state.lastGuidance and re-injected on
44834497
// every turn so it survives compaction.
44844498
onUserPromptSubmitted: async (input: { prompt: string }) => {
4499+
// Track turn count
4500+
state.totalTurns++;
4501+
44854502
// Capture prompt and reset per-prompt tracking flags
44864503
state.currentUserPrompt = input.prompt;
44874504
state.hasCalledListModules = false;
@@ -5172,6 +5189,7 @@ async function main(): Promise<void> {
51725189
await processMessage(session, "continue");
51735190
}
51745191
}
5192+
printExitTokenSummary();
51755193
console.log(`\n✅ Prompt completed. (${formatSessionDuration()})\n`);
51765194
cleanupCtrlR?.();
51775195
rl.close();
@@ -5206,6 +5224,7 @@ async function main(): Promise<void> {
52065224
// Exit — either bare 'exit' or '/exit'
52075225
const lower = trimmed.toLowerCase();
52085226
if (lower === "exit" || lower === "/exit") {
5227+
printExitTokenSummary();
52095228
console.log(`\n👋 Goodbye! (session: ${formatSessionDuration()})\n`);
52105229
break;
52115230
}
@@ -5412,6 +5431,7 @@ const SHUTDOWN_TIMEOUT_MS = 5_000;
54125431

54135432
// Graceful shutdown — clean up on SIGINT (Ctrl+C)
54145433
process.on("SIGINT", async () => {
5434+
printExitTokenSummary();
54155435
console.log(`\n\n👋 Goodbye! (session: ${formatSessionDuration()})\n`);
54165436

54175437
// Stop transcript synchronously — async won't complete before exit

src/agent/llm-output.ts

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,33 @@ export function printUsageStats(stats: string, indent: string): void {
6060
console.log(`${indent}${C.dim("📊 " + stats)}`);
6161
}
6262

63+
/**
64+
* Format a session token summary for /tokens command or exit display.
65+
* Returns an array of lines (without leading newline).
66+
*/
67+
export function formatTokenSummary(state: {
68+
totalInputTokens: number;
69+
totalOutputTokens: number;
70+
totalCacheReadTokens: number;
71+
totalRequests: number;
72+
totalTurns: number;
73+
}): string[] {
74+
const total = state.totalInputTokens + state.totalOutputTokens;
75+
const lines: string[] = [];
76+
lines.push(`${C.label("Token Usage")} ${C.dim("(process total)")}`);
77+
lines.push(`Input: ${state.totalInputTokens.toLocaleString()} tokens`);
78+
lines.push(`Output: ${state.totalOutputTokens.toLocaleString()} tokens`);
79+
if (state.totalCacheReadTokens > 0) {
80+
lines.push(
81+
`Cache read: ${state.totalCacheReadTokens.toLocaleString()} tokens`,
82+
);
83+
}
84+
lines.push(`Total: ${total.toLocaleString()} tokens`);
85+
lines.push(`Requests: ${state.totalRequests}`);
86+
lines.push(`Turns: ${state.totalTurns}`);
87+
return lines;
88+
}
89+
6390
// ── Reasoning Rendering ──────────────────────────────────────────────
6491

6592
/**

src/agent/module-store.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -365,6 +365,8 @@ export async function saveModule(
365365
description,
366366
author,
367367
mutable,
368+
// Keep sourceHash in sync so the validator doesn't flag a mismatch
369+
sourceHash: computeTruncatedHash(source),
368370
metadataCache: {
369371
extractedFromHash: currentHash,
370372
exports,

src/agent/slash-commands.ts

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,16 @@ export async function handleSlashCommand(
143143
console.log();
144144
return true;
145145

146+
case "/tokens": {
147+
const { formatTokenSummary } = await import("./llm-output.js");
148+
const lines = formatTokenSummary(state);
149+
for (const line of lines) {
150+
console.log(` ${line}`);
151+
}
152+
console.log();
153+
return true;
154+
}
155+
146156
case "/reasoning": {
147157
// /reasoning conversation <level> — set conversation reasoning effort
148158
// /reasoning audit <level> — set audit reasoning effort (min: medium)

src/agent/state.ts

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,23 @@ export interface AgentState {
240240
* additionalContext on every onUserPromptSubmitted.
241241
*/
242242
lastGuidance: string | null;
243+
244+
// ── Token Tracking ───────────────────────────────────────────────
245+
246+
/** Cumulative input tokens across all LLM requests this session. */
247+
totalInputTokens: number;
248+
249+
/** Cumulative output tokens across all LLM requests this session. */
250+
totalOutputTokens: number;
251+
252+
/** Cumulative cache-read tokens across all LLM requests this session. */
253+
totalCacheReadTokens: number;
254+
255+
/** Total number of LLM API requests (one per assistant.usage event). */
256+
totalRequests: number;
257+
258+
/** Number of user turns (messages sent) this session. */
259+
totalTurns: number;
243260
}
244261

245262
// ── Factory ──────────────────────────────────────────────────────────
@@ -317,5 +334,12 @@ export function createAgentState(
317334
hasCalledListModules: false,
318335
modulesInspected: new Set<string>(),
319336
lastGuidance: null,
337+
338+
// Token tracking
339+
totalInputTokens: 0,
340+
totalOutputTokens: 0,
341+
totalCacheReadTokens: 0,
342+
totalRequests: 0,
343+
totalTurns: 0,
320344
};
321345
}

src/agent/system-message.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,12 @@ DISCOVERY (never guess — always check):
115115
Do NOT guess parameter names — they are ALL listed in typeDefinitions.
116116
For specific function details, call module_info(name, "functionName").
117117
118+
BUILDING REUSABLE MODULES:
119+
When you identify a missing capability (like a format library), don't just
120+
describe the gap — use register_module to build a reusable module that fills
121+
it. Modules persist across sessions and compound in value over time.
122+
Import your module with: import { fn } from "ha:<name>"
123+
118124
PLUGINS: Require explicit enable via manage_plugin.
119125
Host plugin functions return values directly (not Promises).
120126
You CAN use async/await — it works — but await on a plugin call

0 commit comments

Comments
 (0)