Skip to content

Commit 3cbd61a

Browse files
authored
add anthropic json schema support (#339)
* feat: add Anthropic json_schema support via output_config.format Add native structured output support for the Anthropic provider by mapping the common response_format: :json_schema to Anthropic's output_config.format API. Changes: - transforms.rb: add normalize_response_format to convert json_schema response_format to { format: { type: "json_schema", schema: ... } } for output_config. json_object and text remain unhandled (nil) to preserve existing behavior. - transforms.rb: call normalize_response_format in normalize_params, deleting response_format and setting output_config when applicable. - request.rb: read response_format without deleting so @response_format is preserved for json_object emulation; normalize_params handles deletion and conversion. Design decisions: - JSON Schema Hashes are passed through as-is; not converted to Anthropic::BaseModel. - name and strict from the common format are not forwarded; Anthropic's output_config.format does not use them. - Beta header is not required for output_config (stable API). * test: enable Anthropic json_schema integration tests Activate previously commented-out json_schema tests now that the Anthropic provider supports output_config.format. Changes: - Update REQUEST_JSON_SCHEMA to include output_config.format.schema as the expected serialized request shape. - Enable test_request_builder for response_json_schema_inline, response_json_schema_implicit, response_json_schema_named, and response_json_schema_implicit_bare, verifying all four input forms produce the same output_config payload. - Enable live VCR test asserting response.format.type == "json_schema" and response.message.parsed_json[:colors] is an Array. - Add VCR cassettes for the four json_schema variants and update text cassettes to reflect current request format. * docs: update Anthropic provider docs for json_schema support Document Anthropic's native json_schema structured output support. Changes: - structured_output.md: update provider support table, marking Anthropic json_schema as supported (🟩) with a note about output_config.format on supported Claude models. - anthropic.md: add JSON Schema Support section with usage example, serialized request format, and notes on name/strict not forwarded and model availability. - anthropic.md: update Response Format parameter reference to include JSON Schema Support link. - anthropic.md: update Limitations section to reflect that Anthropic now natively supports json_schema via output_config.format.
1 parent b569eb5 commit 3cbd61a

9 files changed

Lines changed: 287 additions & 20 deletions

File tree

docs/actions/structured_output.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Two JSON response formats:
2020
| Provider | `json_object` | `json_schema` | Notes |
2121
|:---------------|:-------------:|:-------------:|:------|
2222
| **OpenAI** | 🟩 | 🟩 | Native support with strict mode (Responses API only for json_schema) |
23-
| **Anthropic** | 🟦 | | Emulated via prompt engineering technique |
23+
| **Anthropic** | 🟦 | 🟩 | json_object emulated; json_schema native via output_config.format on supported Claude models |
2424
| **OpenRouter** | 🟩 | 🟩 | Native support, depends on underlying model |
2525
| **Ollama** | 🟨 | 🟨 | Model-dependent, support varies by model |
2626
| **RubyLLM** | 🟨 | 🟨 | Depends on underlying provider/model |

docs/providers/anthropic.md

Lines changed: 55 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -119,12 +119,65 @@ Anthropic provides access to the Claude model family. For the complete list of a
119119

120120
### Response Format
121121

122-
- **`response_format`** - Control output format (see [Emulated JSON Object Support](#emulated-json-object-support))
122+
- **`response_format`** - Control output format (see [JSON Object Support](#emulated-json-object-support) and [JSON Schema Support](#json-schema-support))
123123

124124
### Streaming
125125

126126
- **`stream`** - Enable streaming responses (boolean, default: false)
127127

128+
## JSON Schema Support
129+
130+
ActiveAgent maps `response_format: :json_schema` (or the hash form) to Anthropic's native `output_config.format` API. JSON Schema Hashes are passed through as-is; ActiveAgent does not convert them to `Anthropic::BaseModel`.
131+
132+
### Usage
133+
134+
```ruby
135+
class ColorsAgent < ApplicationAgent
136+
generate_with :anthropic, model: "claude-haiku-4-5"
137+
138+
def primary_colors
139+
prompt(
140+
"Return the three primary colors.",
141+
response_format: :json_schema
142+
)
143+
end
144+
end
145+
```
146+
147+
Place a schema file at `app/views/agents/colors_agent/primary_colors.json`:
148+
149+
```json
150+
{
151+
"schema": {
152+
"type": "object",
153+
"properties": {
154+
"colors": { "type": "array", "items": { "type": "string" } }
155+
},
156+
"required": ["colors"],
157+
"additionalProperties": false
158+
}
159+
}
160+
```
161+
162+
ActiveAgent serializes the request as:
163+
164+
```ruby
165+
{
166+
output_config: {
167+
format: {
168+
type: "json_schema",
169+
schema: { ... }
170+
}
171+
}
172+
}
173+
```
174+
175+
### Notes
176+
177+
- `name` and `strict` from the common format are not forwarded; Anthropic's `output_config.format` does not use them.
178+
- JSON Schema support availability depends on model version. Check [Anthropic's documentation](https://platform.claude.com/docs/en/build-with-claude/structured-outputs) for supported models.
179+
- `json_object` emulation via prompt engineering is separate and unchanged.
180+
128181
## Emulated JSON Object Support
129182

130183
While Anthropic does not natively support structured response formats like OpenAI's `json_object` mode, ActiveAgent provides emulated support through a prompt engineering technique.
@@ -155,7 +208,7 @@ Unlike OpenAI's native JSON mode:
155208
- **Prompt-dependent reliability**: Success depends on clear prompt instructions
156209
- **No strict mode**: Cannot guarantee specific field requirements
157210

158-
For applications requiring guaranteed schema conformance, consider using the [Structured Output](/actions/structured_output) feature with providers that support native JSON schema validation.
211+
For applications requiring guaranteed schema conformance, use the [Structured Output](/actions/structured_output) feature with `response_format: :json_schema`. Anthropic natively supports JSON schema validation via `output_config.format` on supported Claude models.
159212

160213
## Constitutional AI
161214

lib/active_agent/providers/anthropic/request.rb

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,9 @@ class Request < SimpleDelegator
7171
# @raise [ArgumentError] when gem model validation fails
7272
def initialize(**params)
7373
# Step 1: Extract custom fields that gem doesn't support
74-
@response_format = params.delete(:response_format)
74+
# Read response_format without deleting - normalize_params will delete and convert it
75+
# to output_config for json_schema, or drop it for other types.
76+
@response_format = params[:response_format]
7577
@stream = params.delete(:stream)
7678
anthropic_beta = params.delete(:anthropic_beta)
7779

lib/active_agent/providers/anthropic/transforms.rb

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,12 @@ def normalize_params(params)
3131
params[:tools] = normalize_tools(params[:tools]) if params[:tools]
3232
params[:tool_choice] = normalize_tool_choice(params[:tool_choice]) if params[:tool_choice]
3333

34+
# Normalize json_schema response_format → output_config (Anthropic's native structured output field)
35+
if params[:response_format]
36+
output_config = normalize_response_format(params.delete(:response_format))
37+
params[:output_config] = output_config if output_config
38+
end
39+
3440
# Handle mcps parameter (common format) -> transforms to mcp_servers (provider format)
3541
if params[:mcps]
3642
params[:mcp_servers] = normalize_mcp_servers(params.delete(:mcps))
@@ -159,6 +165,52 @@ def normalize_tool_choice(tool_choice)
159165
end
160166
end
161167

168+
# Normalizes response_format to Anthropic output_config structure.
169+
#
170+
# Supported (ActiveAgent common format):
171+
# { type: "json_schema", json_schema: { name: "...", schema: {...}, strict: true } }
172+
# → { format: { type: "json_schema", schema: {...} } }
173+
#
174+
# Notes:
175+
# - Anthropic does not use OpenAI's `name` or `strict` fields in output_config.format.
176+
# - json_object is not handled here; it remains prompt-emulated.
177+
# - text is not handled here; Anthropic returns plain text by default.
178+
#
179+
# @param format [Hash, Symbol, String] ActiveAgent common response_format
180+
# @return [Hash] Anthropic output_config hash, or nil if not applicable
181+
def normalize_response_format(format)
182+
case format
183+
when Hash
184+
format_hash = format.deep_symbolize_keys
185+
186+
if format_hash[:type].to_s == "json_schema"
187+
{
188+
format: {
189+
type: "json_schema",
190+
schema: format_hash[:json_schema]&.dig(:schema)
191+
}
192+
}
193+
elsif format_hash[:type].to_s == "json_object"
194+
# json_object is not handled here; it remains prompt-emulated.
195+
nil
196+
elsif format_hash[:type].to_s == "text"
197+
# text is not handled here; it remains prompt-emulated.
198+
nil
199+
else
200+
# Pass through (already properly structured or Anthropic native format)
201+
format_hash
202+
end
203+
when Symbol, String
204+
if format.to_s == "json_schema"
205+
{ format: { type: "json_schema" } }
206+
else
207+
nil
208+
end
209+
else
210+
format
211+
end
212+
end
213+
162214
# Merges consecutive same-role messages into single messages with multiple content blocks.
163215
#
164216
# Required by Anthropic API - consecutive messages with the same role must be combined.

test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit.yml

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit_bare.yml

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_inline.yml

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_named.yml

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

test/integration/anthropic/common_format/response_format_test.rb

Lines changed: 32 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,23 @@ def response_json_object
7373
content: "Return the three primary colors."
7474
}
7575
],
76-
max_tokens: 1024
76+
max_tokens: 1024,
77+
output_config: {
78+
format: {
79+
type: "json_schema",
80+
schema: {
81+
type: "object",
82+
properties: {
83+
colors: {
84+
type: "array",
85+
items: { type: "string" }
86+
}
87+
},
88+
required: [ "colors" ],
89+
additionalProperties: false
90+
}
91+
}
92+
}
7793
}
7894

7995
def response_json_schema_inline
@@ -141,10 +157,10 @@ def response_json_schema_named
141157
end
142158

143159
[
144-
# :response_json_schema_inline,
145-
# :response_json_schema_implicit,
146-
# :response_json_schema_named,
147-
# :response_json_schema_implicit_bare
160+
:response_json_schema_inline,
161+
:response_json_schema_implicit,
162+
:response_json_schema_named,
163+
:response_json_schema_implicit_bare
148164
].each do |action_name|
149165
test_request_builder(TestAgent, action_name, :generate_now, TestAgent::REQUEST_JSON_SCHEMA)
150166
end
@@ -175,19 +191,19 @@ def response_json_schema_named
175191
end
176192
end
177193

178-
# test "response format: json_schema (implicit bare)" do
179-
# agent_name = TestAgent.name.demodulize.underscore
180-
# action_name = "response_json_schema_implicit_bare"
181-
# cassette_name = [ self.class.name.underscore, "#{agent_name}_#{action_name}" ].join("/")
194+
test "response format: json_schema (implicit bare)" do
195+
agent_name = TestAgent.name.demodulize.underscore
196+
action_name = "response_json_schema_implicit_bare"
197+
cassette_name = [ self.class.name.underscore, "#{agent_name}_#{action_name}" ].join("/")
182198

183-
# VCR.use_cassette(cassette_name) do
184-
# response = TestAgent.response_json_schema_implicit_bare.generate_now
199+
VCR.use_cassette(cassette_name) do
200+
response = TestAgent.response_json_schema_implicit_bare.generate_now
185201

186-
# assert_equal "json_schema", response.format.type
187-
# assert_not_nil response.message.parsed_json
188-
# assert_kind_of Array, response.message.parsed_json[:colors]
189-
# end
190-
# end
202+
assert_equal "json_schema", response.format.type
203+
assert_not_nil response.message.parsed_json
204+
assert_kind_of Array, response.message.parsed_json[:colors]
205+
end
206+
end
191207
end
192208
end
193209
end

0 commit comments

Comments
 (0)