Skip to content

Commit 4504ccb

Browse files
Lefteris GilmazLefteris Gilmaz
authored andcommitted
Add Anthropic Messages API front end
1 parent df018d6 commit 4504ccb

4 files changed

Lines changed: 852 additions & 3 deletions

File tree

README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -308,6 +308,22 @@ LiteLLM completion() model= gemini-1.5-flash-002; provider = gemini
308308
the same OpenAI API compatible chat completions endpoint. This should allow you to integrate it into any existing tools or frameworks easily. If the LLM you want to use
309309
doesn't have an OpenAI API compatible endpoint (like Google or Anthropic) you can use [LiteLLM proxy server](https://docs.litellm.ai/docs/proxy/quick_start) that supports most LLMs.
310310

311+
### Anthropic-compatible Messages API
312+
313+
OptiLLM also exposes an Anthropic-compatible API front end for tools that expect the Claude Messages API, such as Claude Code:
314+
315+
```bash
316+
export ANTHROPIC_BASE_URL="http://localhost:8000"
317+
export ANTHROPIC_API_KEY="optillm"
318+
```
319+
320+
Supported compatibility endpoints:
321+
322+
- `POST /v1/messages`
323+
- `POST /v1/messages/count_tokens`
324+
325+
The Anthropic front end translates requests into OptiLLM's existing OpenAI-compatible execution path, so text-only requests can still use approach prefixes such as `moa-gpt-4o-mini` or the `optillm_approach` request field. Requests that include Anthropic tools bypass optimization approaches and use direct proxy behavior to preserve Claude Code's structured `tool_use` and `tool_result` blocks.
326+
311327
The following sequence diagram illustrates how the request and responses go through optillm.
312328

313329
![Sequance diagram showing optillm in use](https://raw.githubusercontent.com/algorithmicsuperintelligence/optillm/main/optillm-sequence-diagram.png)

0 commit comments

Comments
 (0)