Commit e530ca7
feat: Add a default /v1/messages (Anthropic Messages) route to the base Gym… (#1627)
# What does this PR do?
[Design
note](#1620 (comment))
Makes **every NeMo Gym model server speak the Anthropic Messages API**
by adding a
`POST /v1/messages` route to the base `SimpleResponsesAPIModel`, with a
default implementation
that maps Anthropic Messages ↔ Responses around the server's own
`responses()`. External
harnesses that require an Anthropic endpoint — notably
`claude_code_agent` (the Claude Code CLI
only talks `/v1/messages`) — can now run against **any** Gym backend
(vLLM, OpenAI, an inference
provider), not just Anthropic's API.
The Messages ↔ Responses mapping lives in one shared, transport-free,
SDK-free converter,
`nemo_gym/anthropic_converter.py`, used in both directions and shared
with the egress
`anthropic_model` provider (#1146/#1546).
See `DESIGN-NOTE.md` for rationale.
# Why
Gym standardizes on the Responses API, but real blackbox agents speak
other protocols. Rather
than a bespoke per-agent bridge or a separate proxy server, the Messages
spoke becomes a
base-class capability: implement `responses()` once and get
`/v1/messages` for free. This is
the model-proxy layer the CustomAgent EPIC (#1042) assumes — its sidecar
"path-preserving
forwards" `/v1/messages` to these routes.
# Key design points
- **Route on the base class, not a separate proxy.** Every model server
inherits `/v1/messages`;
no extra process or network hop. (An earlier `anthropic_messages_proxy`
+ `SimpleMessagesAPIModel`
iteration was removed in favor of this.)
- **Delegates to the server's own `responses()`** — reuses whatever
backend the server has; a
small `inspect`-based dispatch handles both existing `responses()`
signatures.
- **Synthesized SSE.** Backends are non-streaming via Responses; the
converter fabricates the
Anthropic SSE event stream from the complete response (the Claude CLI
requires streaming).
- **Shared bidirectional converter**, transport-free and **no
`anthropic` SDK** (local shapes
over Gym's aiohttp). Fail-loud (`NotImplementedError`) on unsupported
content.
- **Eval-first.** Token-ids live only at the Responses hop and cannot
ride the Anthropic SSE
response; an RL path would side-channel them, not thread them through
Messages.
# Issues
Closes #1620 (*"Add a default `/v1/messages` (Anthropic
Messages) route to the
base Gym model server"*).
Closes #1566
Relationship to adjacent work:
- **#1620** — this PR is its implementation: the `/v1/messages` route +
default Messages ↔
Responses mapping on `SimpleResponsesAPIModel`, the shared converter,
and the
`claude_code_agent` end-to-end run satisfy its acceptance criteria.
- **#1146 / #1546** (egress `anthropic_model`) — inverse direction;
shares the same converter.
- **#1286** (`responses_converter`) — sibling Chat-Completions ↔
Responses converter; same pattern.
- **EPIC #1042** (CustomAgent) — its sidecar model proxy
"path-preserving forwards" `/v1/messages`
to the routes this PR adds.
# Usage
Every model server now answers Anthropic Messages directly — this is
config-independent and is
the core of the change. Against any running Gym model server:
```bash
curl -s http://127.0.0.1:<model_server_port>/v1/messages -H 'content-type: application/json' -d '{
"model": "x", "max_tokens": 64, "messages": [{"role": "user", "content": "hi"}]
}'
```
`claude_code_agent` reaches any Gym model server by setting its
`model_server` ref — the harness
resolves `ANTHROPIC_BASE_URL` from it and the CLI appends `/v1/messages`
(see
`responses_api_agents/claude_code_agent/app.py:_resolve_base_url`). The
end-to-end run in
**Testing** below used this path against a local vLLM. The verified
`reasoning_gym` Claude Code
config keeps its direct `anthropic_base_url` default; targeting a Gym
model server is an opt-in
`model_server` ref.
# Additional information
- **Changed:** `nemo_gym/base_responses_api_model.py` (new
`/v1/messages` route + default
`messages()`); **added** `nemo_gym/anthropic_converter.py` (shared
bidirectional converter);
`responses_api_agents/claude_code_agent/README.md` (documents the
model-server path). No
change to the verified `reasoning_gym_claude_code_agent.yaml` —
targeting a Gym model server is
an opt-in `model_server` ref on the agent.
- **Converter surface:** `responses_to_anthropic` /
`anthropic_to_responses` (egress, shared
with #1546); `anthropic_request_to_responses` /
`responses_to_anthropic_response` /
`anthropic_response_to_sse` (ingress). Supports text, base64 images,
function tools,
tool_use/tool_result, thinking, stop-reason. Anthropic-API egress
*policy* stays in the
`anthropic_model` server, not the converter.
# Testing
- `tests/unit_tests/test_anthropic_converter.py` (ingress + round-trip)
and `…_egress.py`
(egress) — **100%** converter coverage.
- `tests/unit_tests/test_responses_api_model_messages.py` — base-class
`/v1/messages` for both
`responses()` signatures + SSE framing.
- Full unit suite green (284); `openai_model` tests pass unchanged.
- **Live e2e:** `claude_code_agent → policy_model (vLLM) /v1/messages`
on reasoning_gym —
`finished_naturally=1`, 1 turn, ~5.5k tokens, 0 failures (reward
reflects the small model's
answer, not the plumbing).
- Verified: `ruff check`, `ruff format --check`, `pre-commit`.
# Open questions
- Versioning/regression suite for the converters (OpenAI/Anthropic bump
frequently).
- Audit/align the existing Chat-Completions ↔ Responses tests.
- Confirm fail-loud vs drop policy for items absent in a target schema
(e.g. computer-use).
---------
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Signed-off-by: Lin Jia <linj@nvidia.com>
Co-authored-by: Lin Jia <linj@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent 797db29 commit e530ca7
12 files changed
Lines changed: 2304 additions & 3 deletions
File tree
- nemo_gym
- resources_servers/reasoning_gym/configs
- responses_api_agents/claude_code_agent
- tests/unit_tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
262 | 262 | | |
263 | 263 | | |
264 | 264 | | |
| 265 | + | |
265 | 266 | | |
266 | 267 | | |
267 | 268 | | |
| |||
0 commit comments