Skip to content

feat(bedrock): enable OpenAI model via mantle model endpoint#2224

Closed
JackYPCOnline wants to merge 3 commits intostrands-agents:mainfrom
JackYPCOnline:feat_mantle_model
Closed

feat(bedrock): enable OpenAI model via mantle model endpoint#2224
JackYPCOnline wants to merge 3 commits intostrands-agents:mainfrom
JackYPCOnline:feat_mantle_model

Conversation

@JackYPCOnline
Copy link
Copy Markdown
Contributor

@JackYPCOnline JackYPCOnline commented Apr 29, 2026

Motivation

Amazon Bedrock now exposes an OpenAI-compatible endpoint (Mantle) at
bedrock-mantle.<region>.api.aws, which provides the Responses API, Chat Completions
API, and built-in features like server-side stateful conversations and reasoning
controls. These capabilities are not available through the Converse API that
BedrockModel uses today.

Today, users who want to reach those capabilities have to bypass BedrockModel and
configure OpenAIModel / OpenAIResponsesModel by hand with the correct base URL and
auth. That's workable but leaks Bedrock routing details into application code and
splits the Bedrock experience across two provider classes depending on which API
surface the user wants.

This change lets users stay on BedrockModel and opt into the OpenAI-compatible
transport via config.

Public API Changes

BedrockConfig gains an optional openai_endpoint field. When set, requests route
through the Mantle endpoint via the OpenAI Python SDK instead of Converse; when unset,
behavior is unchanged.

# Converse (existing, unchanged)
BedrockModel(model_id="openai.gpt-oss-20b-1:0", region_name="us-west-2")

# Chat Completions via Mantle
BedrockModel(
    model_id="openai.gpt-oss-120b",
    region_name="us-east-1",
    max_tokens=1024,
    temperature=0.7,
    openai_endpoint={"api": "chat_completions"},
)

# Responses API with reasoning and stateful sessions
BedrockModel(
    model_id="openai.gpt-oss-120b",
    region_name="us-east-1",
    openai_endpoint={
        "api": "responses",
        "stateful": True,
        "params": {"reasoning": {"effort": "medium"}},
    },
)

The nested OpenAIEndpointConfig holds the fields that are meaningful only on the
OpenAI-compatible surface:

  • api (required): "responses" or "chat_completions".
  • api_key: Bedrock-issued API key used as the bearer token. Omit to let the OpenAI
    SDK read OPENAI_API_KEY per AWS's documented flow.
  • stateful: server-side conversation state (Responses only).
  • params: pass-through to the OpenAI SDK for Responses-only options like reasoning.
  • client_args: escape hatch for a custom http_client (e.g. SigV4 for tests) or
    other SDK constructor options.

Generic inference parameters (temperature, top_p, max_tokens) stay on
BedrockConfig and are forwarded to the delegate with naming translated where the
APIs differ (for example max_tokensmax_output_tokens on the Responses path).
stop_sequences is forwarded as stop for Chat Completions but is rejected with
api="responses" since the Responses API does not accept stop sequences.

Config that only applies to the Converse transport cannot be combined with
openai_endpoint. This includes guardrail_*, cache_*, service_tier,
additional_args, streaming=False (both OpenAI delegate APIs always stream), and
stop_sequences when api="responses". All such combinations raise at init time
with a specific, actionable message so misconfigurations fail loudly instead of
silently no-opping.

Use Cases

  • Reach Responses-only features on Bedrock-hosted OpenAI models (stateful sessions,
    reasoning effort, built-in tools) without swapping provider classes.
  • Migrate existing OpenAI SDK code to Bedrock with minimal changes. Setting
    OPENAI_API_KEY to a Bedrock API key and passing openai_endpoint={"api": "chat_completions"}
    is typically all that's needed.
  • Mix transports in the same app. Use Converse for Claude with guardrails, and
    openai_endpoint for gpt-oss with reasoning, both through BedrockModel.

Backward Compatibility

Not a breaking change. The Converse path is untouched; all existing BedrockModel
configurations continue to work exactly as before. The only new surface is the
opt-in openai_endpoint field.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 82.35294% with 15 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/strands/models/bedrock.py 82.35% 11 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

# ``include_tool_result_status`` is intentionally excluded: it is always auto-defaulted by
# ``__init__`` and only affects Converse-side tool-result serialization. It has no effect on
# the Mantle path either way, so requiring users to clear it would be a pure footgun.
_CONVERSE_ONLY_CONFIG_KEYS: frozenset[str] = frozenset(
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue: additional_args is missing from _CONVERSE_ONLY_CONFIG_KEYS. This field is splat into the Converse API request dict at line ~536 and would silently no-op on the OpenAI delegate path.

Suggestion: Add "additional_args" to _CONVERSE_ONLY_CONFIG_KEYS so that setting it alongside openai_endpoint raises at init time with a clear message, consistent with the other Converse-only fields.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Resolved — "additional_args" is now at line 98 of _CONVERSE_ONLY_CONFIG_KEYS, and there's a corresponding test case (additional_args_conflict) in the parametrized validation test.

Comment thread src/strands/models/bedrock.py Outdated
Comment thread src/strands/models/bedrock.py Outdated
Comment thread src/strands/models/bedrock.py
Comment thread src/strands/models/bedrock.py
@github-actions
Copy link
Copy Markdown

Issue: This PR introduces a new public API surface (OpenAIEndpointConfig as a typed dict with 5 fields, plus changes to BedrockConfig and the constructor behavior). Per the API Bar Raising process, this warrants a needs-api-review label since it introduces a new abstraction that customers will use to configure routing behavior.

Key API design questions worth explicit review:

  1. Should openai_endpoint be a nested config dict, or would a simpler api_surface="responses"|"chat_completions" top-level field (with additional fields promoted to BedrockConfig) better align with the "simple at any scale" tenet?
  2. The client_args escape hatch is powerful but underdocumented in terms of lifecycle ownership — who closes the http_client?
  3. Is params the right name when it shadows OpenAIResponsesConfig.params? Could this cause confusion about what's forwarded vs what's not?

Comment thread src/strands/models/bedrock.py Outdated
Comment thread src/strands/models/bedrock.py
@github-actions
Copy link
Copy Markdown

Assessment: Comment

Well-structured feature that cleanly extends BedrockModel with OpenAI-compatible endpoint routing via delegation. The validation-at-init approach and the clear separation between transports is solid.

Review Categories
  • API Design: This introduces a meaningful new public API surface that likely warrants formal API bar-raising review (needs-api-review label). The nested openai_endpoint dict approach is reasonable but alternatives should be explicitly discussed.
  • Documentation accuracy: The module docstring overclaims streaming support on both transports; the additional_args field is missing from the Converse-only conflict check.
  • Silent config drops: stop_sequences on the Responses path and streaming=False on the delegate path are silently ignored — prefer warnings over debug logs for dropped user configuration.
  • Code hygiene: Minor typo in docstring, implicit **kwargs forwarding of system_prompt_content.

The delegation pattern and test coverage (unit + integration with SigV4 signing) are well-executed.

Comment thread src/strands/models/bedrock.py Outdated
@github-actions
Copy link
Copy Markdown

Assessment: Comment

Good iteration. The previous review's critical issues have all been addressed — additional_args is now in the conflict set, streaming=False and stop_sequences with Responses both raise clearly, the typo is fixed, and the system_prompt_content / lifecycle ownership concerns are handled.

Remaining items
  • Documentation consistency: The OpenAIEndpointConfig class docstring still claims stop_sequences and streaming are "forwarded" — this is now inconsistent with the actual raise-on-misconfig behavior.
  • Test coverage: 78.48% patch coverage due to untested delegation paths (stream, structured_output, count_tokens). Lightweight mocked unit tests would close this gap.

Neither is blocking — the code logic is correct and well-validated. Nice work addressing the feedback.

@JackYPCOnline JackYPCOnline changed the title feat(bedrock): route through OpenAI-compatible endpoint via openai_en… feat(bedrock): enable OpenAI model via mantle model endpoint Apr 29, 2026
@github-actions
Copy link
Copy Markdown

Assessment: Approve

All previously raised issues have been addressed. The code is well-structured, documentation is accurate, validation is comprehensive, and error messages are actionable. The remaining Codecov gap (82.35%, 11 missing + 4 partials) is the stream()/structured_output() delegation paths — acceptable given integration tests cover these end-to-end.

Non-blocking note
  • The needs-api-review label question from the first review round remains open as a process item — this introduces a customer-facing config surface that may benefit from designated API reviewer sign-off per the bar-raising process. This is a team process decision, not a code quality concern.

@JackYPCOnline JackYPCOnline added the needs-api-review Makes changes to the public API surface label Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-api-review Makes changes to the public API surface size/l

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant