Skip to content

feat: add ProxyBudgetWidget for LiteLLM/OpenRouter/compatible proxies#367

Open
sagy101 wants to merge 4 commits into
sirmalloc:mainfrom
sagy101:feat/proxy-budget-widget
Open

feat: add ProxyBudgetWidget for LiteLLM/OpenRouter/compatible proxies#367
sagy101 wants to merge 4 commits into
sirmalloc:mainfrom
sagy101:feat/proxy-budget-widget

Conversation

@sagy101
Copy link
Copy Markdown

@sagy101 sagy101 commented May 11, 2026

Summary

Adds a new opt-in proxy-budget widget that displays spend-vs-cap from a LiteLLM-compatible HTTP proxy, with green/yellow/red threshold tiers. Ships with two verified presets out of the box (LiteLLM and OpenRouter) plus full path-driven configuration for any other bearer-token JSON endpoint (Portkey, Helicone, custom proxies). Defaults to Claude Code's existing ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN env vars so the typical setup needs zero config.

Happy to rename, restructure, or close if scope doesn't fit. The implementation is included so the design is concrete to react to.

Display

Live render with the widget at the end of line 2 (Budget: $616.72/$1000.00 (62%) in green):

ProxyBudgetWidget rendering

Format options via metadata.format:

Format Output
spend-percent (default) Budget: $5.00/$100.00 (5%)
percent Budget: 5%
spend Budget: $5.00

Color tiers (default 80% warning / 95% critical, configurable):

Range Color
< warning green
warning..critical yellow
>= critical red

Presets

Two verified presets ship in this PR. Both auth via Authorization: Bearer <token> from the env var named by tokenEnv.

Preset Endpoint Spend path Cap path Reset path
litellm (default) ${baseUrl}/key/info info.spend info.max_budget info.budget_reset_at
openrouter ${baseUrl}/api/v1/key data.usage data.limit data.limit_reset

The presets live in PROXY_BUDGET_PRESETS in src/utils/proxy-budget-fetch.ts. Adding a new one is a single object entry — the exported ProxyBudgetPreset type, the isProxyBudgetPreset runtime guard, the metadata validator in usage-prefetch.ts, and the test matrix all derive from the registry keys. A test iterates the registry to assert every entry has a fully-specified shape, so adding a malformed preset fails the suite immediately.

OpenRouter shape verified against openrouter.ai/docs/api/api-reference/api-keys/get-current-key.

Implementation notes

  • Data flow. Adds prefetchProxyBudgetIfNeeded to src/utils/usage-prefetch.ts, called once per render in src/ccstatusline.ts (mirrors the existing prefetchUsageDataIfNeeded pattern for Anthropic usage). Result attached to RenderContext.proxyBudgetData; the widget reads synchronously.
  • HTTP / cache. New src/utils/proxy-budget-fetch.ts uses Node stdlib https (no new deps). Caches at ~/.cache/ccstatusline/proxy-budget.json with default 60s TTL, plus stale-cache fallback up to 10× TTL on transient failures. Lock file pattern adapted from src/utils/usage-fetch.ts. Sanity guard (spend > 2 × budget) returns null to suppress display when the proxy returns nonsense.
  • Configuration via metadata. All settings live in WidgetItem.metadata: Record<string, string> (preset, endpoint, baseUrlEnv, tokenEnv, authScheme, JSON paths, thresholds, cacheTtlSec, timeoutMs, format). v1 ships without a custom Ink editor — users configure in JSON. Matches early custom-command behavior; a renderEditor can be added in a follow-up if requested.
  • Color tiers via embedded chalk. render() returns a chalk-wrapped string (chalk.green/yellow/red). The renderer at src/utils/renderer.ts:756 wraps with its own outer SGR but the embedded codes survive intact — same mechanism that already works for any colored content. No changes needed in the renderer.
  • Forward compat. Implements the optional getNumericValue returning the percentage so future threshold-aware features in the renderer can consume it without re-implementing the widget.
  • Tokens come from env, never from config. tokenEnv (default ANTHROPIC_AUTH_TOKEN) selects the variable name; the value never touches settings.json, the disk cache, error logs, or anywhere else.
  • Defaults match Claude Code conventions. ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN are the env vars Claude Code itself sets when routing through any proxy. The typical setup needs nothing in metadata beyond "enabled": true.

Why a native widget vs CustomCommand?

The existing CustomCommand widget can run a shell script that hits the same endpoint, but:

  1. Subprocess overhead per render. Status line refreshes ~every 1-3s; spawning a separate shell each time plus serializing JSON over stdin is meaningful overhead compared to a sync read of context.proxyBudgetData.
  2. Cache and lock-file reimplementation. Each user installing the CustomCommand recipe has to reimplement the 60s TTL, stale fallback, and concurrent-fetch locking themselves in shell.
  3. Loss of typed config and first-class features. Threshold-aware tier colors, raw-value mode, preview mode, editor display, and getNumericValue all become script bookkeeping instead of native widget features.

The native widget pays a one-time integration cost for repeated correctness.

Files changed

  • src/utils/proxy-budget-fetch.ts (new) — fetch, cache, lock, sanity guard, preset registry, dotted JSON path resolver
  • src/utils/usage-prefetch.ts — new hasProxyBudgetWidget / prefetchProxyBudgetIfNeeded exports
  • src/types/RenderContext.ts — new proxyBudgetData? field
  • src/ccstatusline.ts — two lines: call prefetch, attach to context
  • src/widgets/ProxyBudget.ts (new) — the widget class
  • src/widgets/index.ts — re-export
  • src/utils/widget-manifest.ts — manifest entry proxy-budget in Usage cluster
  • src/utils/__tests__/proxy-budget-fetch.test.ts (new) — 18 cases (LiteLLM happy path, OpenRouter preset, custom paths, missing env, non-2xx, malformed JSON, missing field, sanity guard, timeout, bearer/x-api-key headers, fresh/stale cache, network down, registry guard + shape)
  • src/widgets/__tests__/ProxyBudget.test.ts (new) — 16 cases (null data, tier boundaries, custom thresholds, format variants, raw mode, preview mode, invalid format/threshold fallback, editor display, getNumericValue, supportsRawValue/supportsColors)
  • docs/USAGE.md — new "Proxy Budget Widget" section under existing widget docs

Test plan

  • bun tsc --noEmit clean
  • bun run lint clean (eslint, 0 warnings)
  • bun test — 1288 / 1288 passing (1254 baseline + 34 new)
  • bun run build produces dist/ccstatusline.js cleanly
  • Smoke-tested against a live LiteLLM proxy (built binary piped sample hook JSON):
    • Cold cache: fetches and renders Budget: $X.XX/$Y.YY (NN%) in green; cache written to ~/.cache/ccstatusline/proxy-budget.json
    • Warm cache: ~0.25s end-to-end, no network call (verified via wall-clock timing)
    • Missing token: segment silently absent, no errors on stdout
    • Black-hole IP (https://10.255.255.1): widget hides, lock file cleaned up
  • Visually verified end-to-end inside Claude Code with a full layout (model, thinking-effort, context bar, tokens, session-clock, cwd, git-branch, git-status, proxy-budget) — see screenshot above

Scope notes

Out of scope for v1: cloud-native cost APIs that don't fit the bearer-token-JSON pattern (AWS Bedrock SigV4 + Cost Explorer, Vertex AI service-account JWT + Cloud Billing, Azure RBAC + Cost Management). Documented in docs/USAGE.md under the widget's "Out of scope" note. Future: separate provider-specific widgets if there's demand.

sagy101 added 4 commits May 11, 2026 15:22
Adds the data layer for an upcoming proxy-budget widget that fetches
spend-vs-cap from an HTTP proxy (LiteLLM, OpenRouter, Portkey, Helicone,
and any other bearer-token JSON endpoint). The widget itself follows in
the next commit so this one is reviewable in isolation.

src/utils/proxy-budget-fetch.ts (new):
- HTTPS GET via Node stdlib (no new deps); AbortController-style timeout
  via req.setTimeout for the same effect.
- Dotted-string JSON path resolver for spend/budget/resetAt fields.
- Sanity guard: rejects spend > 2x budget (likely misconfigured path).
- Disk cache at ~/.cache/ccstatusline/proxy-budget.json with 60s TTL by
  default, plus stale fallback up to 10x TTL on transient fetch failure.
  Lock file at ~/.cache/ccstatusline/proxy-budget.lock prevents
  concurrent fetches.
- Token read from configurable env var (default ANTHROPIC_AUTH_TOKEN);
  never written to disk cache, never echoed in error paths.
- Bearer (default) and x-api-key auth schemes.

src/utils/usage-prefetch.ts:
- New hasProxyBudgetWidget(lines) and prefetchProxyBudgetIfNeeded(lines)
  exports. Keeps all prefetch coordination in one orchestrator file.

src/types/RenderContext.ts:
- Optional proxyBudgetData?: ProxyBudgetData | null field. Widgets read
  it synchronously after the prefetcher populates it.

src/ccstatusline.ts:
- Two lines: call prefetchProxyBudgetIfNeeded next to the existing usage
  prefetch and attach the result to RenderContext.

src/utils/__tests__/proxy-budget-fetch.test.ts (new, 13 cases):
- LiteLLM happy path, custom JSON paths (OpenRouter shape), missing env,
  non-2xx, malformed JSON, missing field, sanity guard, timeout, bearer
  vs x-api-key headers, fresh-cache hit, stale fallback, network down +
  no cache.

All 1267 tests pass (1254 baseline + 13 new). Lint clean.
Adds a 'preset' metadata key for the proxy-budget widget that bundles
the endpoint suffix, JSON paths, and auth scheme for a known proxy.
v1 ships two verified presets:

- litellm:    GET ${baseUrl}/key/info     -> info.spend / info.max_budget / info.budget_reset_at
- openrouter: GET ${baseUrl}/api/v1/key   -> data.usage / data.limit / data.limit_reset

The default (no preset) remains litellm-shaped so existing usage is
unchanged. Users override individual fields per-key as before; the per-
key value wins over the preset default.

The registry is designed for easy extension. Adding a new preset is a
single object entry in PROXY_BUDGET_PRESETS:

  newProxy: {
      endpoint: '${baseUrl}/...',
      spendPath: 'a.b.c',
      budgetPath: 'a.b.d',
      resetAtPath: 'a.b.e',
      authScheme: 'bearer'
  }

The exported ProxyBudgetPreset type, the isProxyBudgetPreset runtime
guard, the prefetcher's metadata.preset validator, and the test matrix
all derive from the registry keys via 'keyof typeof'. No other file
needs to change to add a preset.

Two new tests iterate the registry to assert (a) the guard accepts
every key and rejects unknowns, and (b) every entry has a fully-
specified shape. So adding a malformed preset entry fails the suite
immediately.

OpenRouter response shape verified against
https://openrouter.ai/docs/api/api-reference/api-keys/get-current-key
(spend = data.usage, cap = data.limit, reset = data.limit_reset).

All 1272 tests pass (1267 prior + 5 new preset cases). Lint clean.
Adds the user-facing widget that reads the prefetched proxyBudgetData
from RenderContext and renders an inline color-tiered indicator. Type
string 'proxy-budget'; category 'Usage'; default chalk color 'green';
implements Widget plus the optional getNumericValue() for forward
compatibility with the interface's threshold hook.

Display:
- Default format 'spend-percent' => '$5.00/$100.00 (5%)'
- 'percent' => '5%'
- 'spend'   => '$5.00'

Color tiers reuse chalk.green / chalk.yellow / chalk.red embedded in
the returned string. The rendering pipeline at src/utils/renderer.ts:756
wraps with its own outer SGR but the embedded codes survive intact —
same mechanism that already works for any colored content. Default
thresholds are warningThreshold=80 and criticalThreshold=95, both
overridable per-instance via metadata. Out-of-range values fall back
silently.

Configuration is read entirely from item.metadata (string-keyed) for
v1 — no custom Ink editor needed. Matches the early CustomCommand
pattern. The editor's modifierText shows the preset name or endpoint
host so users can tell instances apart at a glance.

Registration:
- src/widgets/index.ts: export ProxyBudgetWidget
- src/utils/widget-manifest.ts: manifest entry 'proxy-budget' in the
  Usage cluster

Tests (16 cases in src/widgets/__tests__/ProxyBudget.test.ts):
- null data returns null (segment hides gracefully)
- green / yellow / red at default tier boundaries
- custom warningThreshold / criticalThreshold from metadata
- format=percent / format=spend
- item.rawValue strips the 'Budget:' label
- isPreview returns hardcoded example with and without rawValue
- invalid format and out-of-range thresholds fall back to defaults
- getEditorDisplay shows preset hint
- getNumericValue returns percentage or null
- supportsRawValue and supportsColors both true

All 1288 tests pass (1272 prior + 16 new widget). Lint clean.
Adds a section in docs/USAGE.md mirroring the existing widget
documentation pattern. Covers:

- the two ships-with-the-PR presets (litellm default, openrouter
  verified against openrouter.ai's /api/v1/key)
- the full metadata table with defaults
- an example covering both the Claude-Code-routed setup (no env config
  needed) and OpenRouter direct
- explicit 'why a native widget vs Custom Command?' rationale to
  pre-empt the reviewer question
- an 'Out of scope (v1)' note for cloud-native cost APIs (Bedrock
  SigV4, Vertex JWT, Azure RBAC) so users aren't surprised

Also adds a one-line entry in the 'Available Widgets / Tokens, Usage &
Context' bulleted index so the new widget appears in the at-a-glance
catalog.
@sagy101 sagy101 marked this pull request as ready for review May 11, 2026 13:34
@sirmalloc sirmalloc force-pushed the main branch 3 times, most recently from 4f7a07b to ec28376 Compare May 12, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant