Skip to content

feat(pricing): add user price overrides for models#560

Open
ozymandiashh wants to merge 1 commit into
getagentseal:mainfrom
ozymandiashh:pr/price-override
Open

feat(pricing): add user price overrides for models#560
ozymandiashh wants to merge 1 commit into
getagentseal:mainfrom
ozymandiashh:pr/price-override

Conversation

@ozymandiashh

Copy link
Copy Markdown
Contributor

Closes #390.

What this adds

A user-defined per-model price override, so you can give a model a price that LiteLLM does not ship yet (for example DeepSeek V4 currently reports $0), or override an existing price, without waiting for an upstream pricing PR. Overrides take the highest precedence in cost computation.

# rates are USD per 1,000,000 tokens (the unit you read on pricing pages)
codeburn price-override deepseek-v4 --input 0.27 --output 1.10
codeburn price-override deepseek-v4 --input 0.27 --output 1.10 --cache-read 0.07 --cache-creation 0.27
codeburn price-override --list
codeburn price-override --remove deepseek-v4

Why

LiteLLM is the source of truth for pricing, but it lags new or promotional prices, and some models (or private/internal model ids) never get a public entry. When a model has no entry it prices to $0, which makes its usage invisible in reports. This lets you set a price locally and keep an eye on cost in codeburn until the official entry lands, then drop the override. It also covers business/negotiated rates that differ from list price.

How it works

  • Config: a new optional priceOverrides map in ~/.config/codeburn/config.json, keyed by model, with input / output / optional cacheRead / cacheCreation rates in USD per 1,000,000 tokens.
  • CLI: a price-override [model] command mirroring model-alias, with --input / --output / --cache-read / --cache-creation (per 1M), plus --list and --remove. Setting requires at least --input and --output, and every provided rate must be a finite number >= 0 (otherwise it exits non-zero with a clear message). The unit is documented in the command help.
  • Pricing: setPriceOverrides converts the per-1M rates to the internal per-token ModelCosts (dividing by 1e6, guarded by the existing safePerTokenRate). getModelCosts consults the overrides at each resolution tier it already uses for the snapshot, so an override wins wherever a snapshot entry of the same key would have matched.

Two correctness details

  1. An override wins wherever the snapshot would match, without over-applying. getModelCosts resolves a model by exact key, then longest-prefix, then case-insensitive. The override check is interleaved at each of those tiers (override-exact, snapshot-exact, override-prefix, snapshot-prefix, override-ci, snapshot-ci), so an override is never silently bypassed by a fuzzier snapshot match. A more-specific exact entry still wins, so overriding gpt-5 does not hijack a gpt-5-mini call that has its own exact entry.
  2. Overrides participate in daily-cache invalidation. The daily cache is keyed by a config hash (today the localModelSavings hash). Changing a price override now also changes that hash (savings and overrides are combined into the value passed to ensureCacheHydrated), so cached historical days recompute against the new price instead of showing the stale one. With no overrides configured, the hash is byte-for-byte the previous savings-only value, so nothing changes for existing users.

Default behavior is unchanged

With no priceOverrides configured, pricing resolution and the cache-invalidation hash are byte-for-byte identical to before. The feature is purely additive.

Verification

  • New unit tests (tests/models.test.ts): an override prices a model that has no snapshot entry to a non-zero cost; an override wins over both a snapshot price and a configured alias for the same name; the per-1M to per-token conversion is exact ($1.00 / 1M over 1,000,000 input tokens prices to exactly $1.00); cache rates default sanely when omitted; the override applies case-insensitively and by prefix but does not shadow a more-specific exact entry; two different override configs produce different cache-invalidation hashes while empty overrides leave the savings-only hash unchanged.
  • New CLI tests (tests/cli-price-override.test.ts, isolated HOME): set writes config.priceOverrides; --list shows it; --remove deletes it; an invalid rate is rejected with a non-zero exit.
  • npx vitest run tests/models.test.ts tests/cli-price-override.test.ts passes (118 tests). npm test is green except the pre-existing locale-dependent tests/overview.test.ts assertion. npx tsc --noEmit is clean. The offline build succeeds.

Files

  • src/config.ts (add priceOverrides to Config)
  • src/models.ts (setPriceOverrides, override-aware getModelCosts, getPriceOverridesConfigHash)
  • src/usage-aggregator.ts (combine the savings and overrides hashes for cache invalidation)
  • src/main.ts (load overrides at startup + the price-override command)
  • tests/models.test.ts + tests/cli-price-override.test.ts (new)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Pricing override function

1 participant