Skip to content

ROADMAP.md

Latest commit

 

History

History
60 lines (43 loc) · 4.76 KB

ROADMAP.md

File metadata and controls

60 lines (43 loc) · 4.76 KB
Raw

Roadmap

Language: English · 简体中文

This document sketches where context-tracker could go. It is a direction, not a commitment — priorities may shift with real usage and feedback.

Detailed, checklist-driven breakdown: see docs/implementation-plan.md.

Positioning

context-tracker can grow along three possible identities. The route below is shaped by which one we lean into:

  • A. Developer debugging tool (current) — see exactly what a single request sent; troubleshoot agent behavior.
  • B. Context & cost analysis — aggregate across requests to answer "where did my tokens go, and how do I trim them?"
  • C. Agent observability infrastructure — durable storage, session traces, and team workflows; an APM for LLM agents.

The strongest moat is B: "capture and view requests" has a low technical bar and is easily replaced, but "tell me how my context budget is spent and how to save it" is a real pain point in the agent era — and the protocol-extraction layer already gives us the foundation for it.

Themes

1. Capture depth — get more, more accurately

  • Response deep-parsing — responses are currently stored as raw SSE text. Parse out stop_reason, tool calls, and token usage so the viewer shows what a call actually consumed. This is the key step from "viewing the request" to "seeing the whole picture."
  • Request diff — diff between consecutive requests. Agents append messages every turn; a diff instantly shows what was added, whether the system prompt changed, and which history got truncated.
  • Precise token accounting — feed a tokenizer (or the response usage) to turn the N msg / N tools / N B summary into real token counts, broken down by system / tools / history.

2. Analysis & insight — the core of positioning B

  • Context composition — a breakdown showing "40% of this call's tokens are tool definitions, 35% is history, 10% is your actual question" — directly actionable for optimization.
  • Cost dashboard — aggregate spend by provider / model / time (usage + a pricing table) to answer "how much did this project cost today." Complements billing-oriented tools that lack per-request context detail.
  • Waste detection — automatically flag repeated large tool schemas, never-trimmed history, oversized system prompts, with concrete suggestions.

3. Coverage breadth

  • Transparent proxy mode (opt-in) — an optional MITM + CA mode for fully config-free capture and signature-authenticated providers (Bedrock / Vertex). The default stays the zero-CA baseURL mode; this is a path for those who need it.
  • More clients — Gemini CLI, Cursor, Cline, Aider, etc. Each is mostly a matter of confirming its base URL / protocol and adding an extractor to protocols.js.
  • WebSocket / bidirectional streams — for future realtime API scenarios.

4. Persistence & retrospection — the basis for positioning C

  • On-disk storage — everything is in memory today and lost on restart. Add optional SQLite/JSONL persistence for post-hoc review and cross-session search.
  • Session / trace correlation — group a task's multiple requests into one timeline by session_id, instead of a flat request list.
  • Export / share — export a single entry or a whole session to HAR / JSON / Markdown for issues or team review.

5. Ecosystem & distribution

  • Stability infra — 22 tests exist but there is no CI. Add GitHub Actions (test + lint), a CHANGELOG, and semantic versioning ahead of a public release.
  • Programmable API — turn the viewer's /__history into a documented read-only API so others can build analysis/alerting on top.
  • Hooks / plugin points — allow custom redactors, custom provider extractors, and request transformers.

Phased plan

Version Focus Highlights
v0.2 Polish current tool (A) Publish to npm · CI · response usage parsing · request diff
v0.3 Move toward analysis (B) Precise token accounting · context composition · cost aggregation
v0.4 Breadth Opt-in transparent proxy (CA) · more clients (Gemini/Cursor/Aider)
v0.5 Persistence & sessions (C) On-disk storage · session traces · export
v1.0 Stability & ecosystem Documented API · plugin points · semver commitment

Guiding principles

  • Stay zero-dependency by default. New heavy features (tokenizers, storage, transparent proxy) should be opt-in and degrade gracefully when absent.
  • Never touch the live path. Capture and analysis remain fire-and-forget side channels; the agent's request/response must never be blocked or altered.
  • Local-first. No telemetry, no data leaving the machine unless the user explicitly exports it.