Skip to content

Add documentation for limiting token usage in AG-UI, conversational, …#12453

Open
GugaGlonti wants to merge 1 commit into
opensearch-project:mainfrom
GugaGlonti:main
Open

Add documentation for limiting token usage in AG-UI, conversational, …#12453
GugaGlonti wants to merge 1 commit into
opensearch-project:mainfrom
GugaGlonti:main

Conversation

@GugaGlonti
Copy link
Copy Markdown

Description

Documents the parameters.max_tokens agent-level token budget for conversational agents, AG-UI agents, plan-execute-reflect agents, and the Execute Agent API.

This update explains that max_tokens limits covered agent-runner LLM calls by applying the remaining budget before each request, preserving lower per-call model limits, and stopping execution when reported usage exhausts the budget. It also clarifies that include_token_usage is not required for token limiting; it only controls whether detailed token usage metrics are returned in the response.

Issues Resolved

Closes #12430

Version

3.7

Frontend features

N/A. This PR documents ML Commons agent API behavior and does not include OpenSearch Dashboards frontend changes.

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions Bot added the Tech review PR: Tech review in progress label May 21, 2026
@github-actions
Copy link
Copy Markdown

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference).

…plan-execute-reflect agents, and Execute Agent API

Signed-off-by: Guga Glonti <110097877+GugaGlonti@users.noreply.github.com>
@kolchfa-aws kolchfa-aws added release-notes PR: Include this PR in the automated release notes v3.7.0 labels May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes PR: Include this PR in the automated release notes Tech review PR: Tech review in progress v3.7.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DOC] Agent-level Token Limit parameter

2 participants