feat: add JSON structured output support to `BedrockChatGenerator` by davidsbatista · Pull Request #3108 · deepset-ai/haystack-core-integrations

davidsbatista · 2026-04-07T09:07:42Z

Related Issues

fixes #3067

Proposed Changes:

This PR introduces two major changes.,

1. Structured output support (json_schema)

Pass a json_schema dict in generation_kwargs to request validated JSON responses from the model. The generator builds the correct outputConfig for the Bedrock Converse API and stores the parsed result in reply.meta["structured_output"].

  result = generator.run(
      messages=[ChatMessage.from_user("Return user data as JSON")],
      generation_kwargs={
          "json_schema": {
              "name": "user_schema",
              "schema": {"type": "object", "properties": {"name": {"type": "string"}}},
          }
      },
  )
  # result["replies"][0].meta["structured_output"] -> {"name": "..."}

2. `aioboto3` → `aiobotocore` (async path)

The outputConfig requires boto3>=1.42.84, but aioboto3 pins aiobotocore==2.25.1 which is incompatible with that version of botocore at runtime. Replaced aioboto3 with aiobotocore>=3.4.0 directly — same underlying library, no functional change to async behaviour, resolves the version conflict.

Files changed

pyproject.toml — updated deps and mypy overrides
common/amazon_bedrock/utils.py — async session now uses aiobotocore.session.AioSession
chat/chat_generator.py — run_async() uses session.create_client() with explicit credentials
chat/utils.py — _parse_structured_output() helper
tests/ — unit + integration tests for both paths

How did you test it?

Added new integrations tests for both the run and run_async() when having a json schema in the generation_kwargs
added unit tests for the _parse_structured_output
added a few more tests to covering previously untested paths in AmazonBedrockChatGenerator

Notes for the reviewer

there are also some changes which are purely cosmetic, due to applying a new ruff/linting to the existing code

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

…icts + updating tests

github-actions · 2026-04-07T09:10:47Z

Coverage report (amazon_bedrock)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
integrations/amazon_bedrock/src/haystack_integrations/common/amazon_bedrock
utils.py
integrations/amazon_bedrock/src/haystack_integrations/components/generators/amazon_bedrock/chat
chat_generator.py					671-672, 688-689, 692
utils.py					715
Project Total

_{This report was generated by python-coverage-comment-action}

anakin87

I left some comments

anakin87 · 2026-04-07T15:31:54Z

integrations/amazon_bedrock/src/haystack_integrations/common/amazon_bedrock/utils.py

-                region_name=aws_region_name,
-                profile_name=aws_profile_name,
-            )
+            session = aiobotocore.session.AioSession()


could you explain why the aws_* variables are not passed in? I'm not familiar with aiobotocore and haven't found comprehensive docs...

Ok I see we use them at client creation below. In any case, feel free to add details if relevant

anakin87 · 2026-04-07T15:34:01Z

...edrock/src/haystack_integrations/components/generators/amazon_bedrock/chat/chat_generator.py

+        aws_access_key_id: Secret | None = None,
+        aws_secret_access_key: Secret | None = None,
+        aws_session_token: Secret | None = None,
+        aws_region_name: Secret | None = None,
+        aws_profile_name: Secret | None = None,


we use this pattern across all integrations, so I don't see the need for changing it.
The docstring also generates the API reference, which makes it clear to users what the default value is.

this was a mistake, thanks for catching it!

anakin87 · 2026-04-07T15:36:22Z

...edrock/src/haystack_integrations/components/generators/amazon_bedrock/chat/chat_generator.py

            - `stopSequences`: List of stop sequences to stop generation.
            - `temperature`: Sampling temperature.
            - `topP`: Nucleus sampling parameter.
+            - `json_schema`: Request structured JSON output validated against a schema. Provide a dict with:


Instead of json_schema, I'd use response_format, which is more adopted across integrations

I can expose it like that, but then I guess for the API to process it, I have to rename it to json_schema before it is sent

actually it might not be needed

anakin87 · 2026-04-07T15:38:27Z

...edrock/src/haystack_integrations/components/generators/amazon_bedrock/chat/chat_generator.py

            - `temperature`: Sampling temperature.
            - `topP`: Nucleus sampling parameter.
+            - `json_schema`: Request structured JSON output validated against a schema. See
+              :meth:`_prepare_request_params` for full details.


we generally don't use :meth: syntax

I would not point users to a private method (not available on the API reference). If we want to avoid duplications, let's point users to __init__

anakin87 · 2026-04-07T15:40:33Z

.../amazon_bedrock/src/haystack_integrations/components/generators/amazon_bedrock/chat/utils.py

    return replies


+def _parse_structured_output(replies: list[ChatMessage]) -> list[ChatMessage]:


I'd prefer to leave JSON parsing to users, as we do in other integrations.

anakin87 · 2026-04-07T15:41:06Z

...edrock/src/haystack_integrations/components/generators/amazon_bedrock/chat/chat_generator.py

            - `temperature`: Sampling temperature.
            - `topP`: Nucleus sampling parameter.
+            - `json_schema`: Request structured JSON output validated against a schema. See
+              :meth:`_prepare_request_params` for full details.


same comment as in run

Akash504-ai and others added 13 commits April 2, 2026 10:55

feat(amazon-bedrock): add structured output support via outputConfig

963cccd

test: add unit and integration test for outputConfig support

c930c1e

style: format test file with black

db6231d

style: format amazon bedrock tests with black

6eebbb9

style: fix remaining E501 line length issues

c10cf98

style: fix final E501 issues

ef21b44

fmt 8 specific test files

e0575fa

docs: add outputConfig to parameter documentation

49d7fb9

fix: move outputConfig into additionalModelRequestFields

42b174d

fix: remove unsupported outputConfig and resolve lint issues

2ec338f

fix: ignore outputConfig to avoid validation errors and lint issues

09b9dd3

increasing dependencies to boto3>=1.42.84 and dealing with deps confl…

f240292

…icts + updating tests

Merge branch 'main' into feat/add-structured-ouput-BedrockChatGenerator

8ecd6f8

github-actions bot added the integration:amazon-bedrock label Apr 7, 2026

davidsbatista changed the title ~~Feat/add structured ouput bedrock chat generator~~ Feat: add JSON structured output support to BedrockChatGenerator Apr 7, 2026

github-actions bot added the type:documentation Improvements or additions to documentation label Apr 7, 2026

davidsbatista changed the title ~~Feat: add JSON structured output support to BedrockChatGenerator~~ feat: add JSON structured output support to BedrockChatGenerator Apr 7, 2026

formatting

f819810

davidsbatista mentioned this pull request Apr 7, 2026

feat: add structured output support via outputConfig to AmazonBedrockChatGenerator #3088

Closed

4 tasks

davidsbatista added 2 commits April 7, 2026 14:37

wip

d46a729

replacing aiboto by aiobotocore

e5ea9a6

davidsbatista removed the type:documentation Improvements or additions to documentation label Apr 7, 2026

Merge branch 'main' into feat/add-structured-ouput-BedrockChatGenerator

45b7624

github-actions bot added the type:documentation Improvements or additions to documentation label Apr 7, 2026

davidsbatista added 5 commits April 7, 2026 16:03

Merge branch 'main' into feat/add-structured-ouput-BedrockChatGenerator

13ea263

addin tests for utils

88b4700

addin tests for utils

13300cb

fixing imports

f17b3b8

increasing test coverage for ChatGenerator

2926e3c

davidsbatista added 2 commits April 7, 2026 16:38

fixing imports

6c424f0

adding integrations test for run_async as well

f95e39a

davidsbatista marked this pull request as ready for review April 7, 2026 15:05

davidsbatista requested a review from a team as a code owner April 7, 2026 15:05

davidsbatista requested review from anakin87 and bogdankostic and removed request for a team and bogdankostic April 7, 2026 15:05

anakin87 requested changes Apr 7, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add JSON structured output support to `BedrockChatGenerator`#3108

feat: add JSON structured output support to `BedrockChatGenerator`#3108
davidsbatista wants to merge 24 commits intomainfrom
feat/add-structured-ouput-BedrockChatGenerator

davidsbatista commented Apr 7, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

anakin87 left a comment

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

davidsbatista Apr 8, 2026

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

davidsbatista Apr 8, 2026

Uh oh!

davidsbatista Apr 8, 2026

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

anakin87 Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		return replies


		def _parse_structured_output(replies: list[ChatMessage]) -> list[ChatMessage]:

Conversation

davidsbatista commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues

Proposed Changes:

1. Structured output support (json_schema)

2. aioboto3 → aiobotocore (async path)

How did you test it?

Notes for the reviewer

Checklist

Uh oh!

github-actions bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report (amazon_bedrock)

Uh oh!

anakin87 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

davidsbatista commented Apr 7, 2026 •

edited

Loading

2. `aioboto3` → `aiobotocore` (async path)

github-actions bot commented Apr 7, 2026 •

edited

Loading