Feature/count token#8
Conversation
- Add new command line option `--claude-code-env` to generate environment variables for Claude Code - Update `runServer` function to use new `claudeCodeEnv` option - Add `claude-code-env` to `start` command options in README.md
…ge assignment in translateChunkToAnthropicEvents message_start
There was a problem hiding this comment.
Pull Request Overview
This pull request implements a comprehensive token counting feature for the Anthropic API translation layer, including adding cached token support and creating a new count_tokens endpoint.
Key changes:
- Added a new
/v1/messages/count_tokensendpoint handler for token counting - Replaced the simple tokenizer with a comprehensive token calculation system supporting multiple encoders
- Enhanced usage tracking with cached token support in response translations
Reviewed Changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/routes/messages/count-tokens-handler.ts |
New handler implementing token counting logic with model-specific adjustments |
src/lib/tokenizer.ts |
Complete rewrite from simple counting to comprehensive multi-encoder token calculation |
src/routes/messages/route.ts |
Added count_tokens endpoint routing |
src/routes/messages/non-stream-translation.ts |
Enhanced usage object with cached token support |
src/routes/messages/stream-translation.ts |
Updated streaming response to handle cached tokens |
src/routes/chat-completions/handler.ts |
Updated to use new async token counting with model context |
src/services/copilot/create-chat-completions.ts |
Added prompt_tokens_details interface for cached tokens |
src/start.ts |
Added claude-code-env CLI option |
src/routes/messages/anthropic-types.ts |
Made usage field optional in response type |
tests/anthropic-response.test.ts |
Updated test to handle optional usage field |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| let finalTokenCount = tokenCount.input + tokenCount.output | ||
| if (anthropicPayload.model.startsWith("claude")) { | ||
| finalTokenCount = Math.round(finalTokenCount * 1.15) | ||
| } else if (anthropicPayload.model.startsWith("grok")) { | ||
| finalTokenCount = Math.round(finalTokenCount * 1.03) | ||
| } |
There was a problem hiding this comment.
The magic numbers 1.15 and 1.03 should be extracted as named constants with comments explaining their purpose (e.g., model-specific token estimation adjustments).
| if ( | ||
| typeof func.parameters === "object" // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition |
There was a problem hiding this comment.
The eslint disable comment is placed incorrectly and creates confusing formatting. Move it to the line above the condition or use a block comment format.
| if ( | |
| typeof func.parameters === "object" // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition | |
| // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition | |
| if ( | |
| typeof func.parameters === "object" |
| if (anthropicPayload.model.startsWith("claude")) { | ||
| // https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview#pricing | ||
| tokenCount.input = tokenCount.input + 346 | ||
| } else if (anthropicPayload.model.startsWith("grok")) { | ||
| tokenCount.input = tokenCount.input + 480 | ||
| } |
There was a problem hiding this comment.
The magic numbers 346 and 480 should be extracted as named constants with clear comments explaining they represent tool usage token overhead for different model families.
No description provided.