Skip to content

feat: add metrics and usage to conversation streaming responses#3148

Draft
pjkroker wants to merge 12 commits intoaws-amplify:mainfrom
pjkroker:feature/bedrock-model-usage
Draft

feat: add metrics and usage to conversation streaming responses#3148
pjkroker wants to merge 12 commits intoaws-amplify:mainfrom
pjkroker:feature/bedrock-model-usage

Conversation

@pjkroker
Copy link
Copy Markdown

@pjkroker pjkroker commented Mar 26, 2026

Problem

Streaming conversation responses from the Bedrock Converse API do not expose model usage or latency information. Consumers have no way to track token consumption or response latency for conversation turns.

Issue number, if available: #2355

Related PRs (all required for #2355)

This feature spans 4 repositories. PRs should be merged in this order:

  1. amplify-category-apiaws-amplify/amplify-category-api#3448 — Defines AmplifyAIMetrics/AmplifyAIUsage GraphQL types and updates resolvers
  2. amplify-dataaws-amplify/amplify-data#697 — Adds TypeScript types and client-side deserializers
  3. amplify-backendaws-amplify/amplify-backend#3148 — Captures metrics/usage from Bedrock streaming response
  4. amplify-codegenaws-amplify/amplify-codegen#1008 — Adds fields to generated introspection metadata

Changes

  • Added metrics (latencyMs) and usage (inputTokens, outputTokens, totalTokens) fields to the StreamingResponseChunk type, included in the final chunk when a turn completes.
  • Extracted metrics and usage from the Bedrock metadata stream chunk in BedrockConverseAdapter.
  • Serialized metrics and usage to JSON strings in ConversationTurnResponseSender before sending to AppSync.
  • Added metrics and usage to the GraphQL message selection set in ConversationMessageHistoryRetriever.
  • Updated the integration test schema with MockMetrics and MockUsage custom types on ConversationMessageAssistantResponse.
    Corresponding docs PR, if applicable:

Validation

  • Added unit test: includes metrics and usage in the final streaming chunk (bedrock converse adapter)
  • Added unit test: serializes metrics and usage to JSON when streaming (response sender)
  • Updated existing streaming test assertions to account for the new metadata chunk
  • All 77 unit tests in @aws-amplify/ai-constructs pass

Checklist

  • If this PR includes a functional change to the runtime behavior of the code, I have added or updated automated test coverage for this change.
  • If this PR requires a change to the Project Architecture README, I have included that update in this PR.
  • If this PR requires a docs update, I have linked to that docs PR above.
  • If this PR modifies E2E tests, makes changes to resource provisioning, or makes SDK calls, I have run the PR checks with the run-e2e label set.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Mar 26, 2026

🦋 Changeset detected

Latest commit: 5c1bcbf

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@aws-amplify/ai-constructs Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pjkroker pjkroker force-pushed the feature/bedrock-model-usage branch from 7bee268 to beb71bd Compare March 30, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant