-
Notifications
You must be signed in to change notification settings - Fork 828
feat: add context window limit lookup table #2249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
opieter-aws
merged 1 commit into
strands-agents:main
from
opieter-aws:feat/context-window-limit-lookup
May 5, 2026
+425
−7
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,177 @@ | ||
| """Default model metadata lookup tables. | ||
|
|
||
| Provides context window limits for known model IDs across all providers. | ||
| Values sourced from provider documentation and | ||
| https://github.com/BerriAI/litellm/blob/litellm_internal_staging/model_prices_and_context_window.json | ||
|
|
||
| Applied to providers with well-known, fixed model IDs: Bedrock, Anthropic, OpenAI, | ||
| OpenAI Responses, Gemini, and Mistral. Providers that use local/custom model IDs | ||
| (Ollama, LlamaCpp, SageMaker) or proxy to other providers with their own prefixed | ||
| ID format (LiteLLM) are excluded — their context windows depend on deployment config, | ||
| not a static table. | ||
| """ | ||
|
|
||
| import logging | ||
| from collections.abc import Mapping | ||
| from typing import TypeVar | ||
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
| _C = TypeVar("_C", bound=Mapping[str, object]) | ||
|
|
||
| # Context window limits (in tokens) for known model IDs. | ||
| # | ||
| # Best-effort lookup table — unknown models return None and callers | ||
| # fall back gracefully (e.g. proactive compression is disabled). | ||
|
opieter-aws marked this conversation as resolved.
|
||
| # Users can always override with an explicit context_window_limit in their model config. | ||
| # | ||
| # For Bedrock models with cross-region prefixes (e.g. us., eu., global.), | ||
| # get_context_window_limit strips the prefix before lookup so only the base model ID is needed here. | ||
| _CONTEXT_WINDOW_LIMITS: dict[str, int] = { | ||
| # Anthropic (direct API) | ||
| "claude-sonnet-4-6": 1_000_000, | ||
| "claude-sonnet-4-20250514": 1_000_000, | ||
| "claude-sonnet-4-5": 200_000, | ||
| "claude-sonnet-4-5-20250929": 200_000, | ||
| "claude-opus-4-6": 1_000_000, | ||
| "claude-opus-4-6-20260205": 1_000_000, | ||
| "claude-opus-4-7": 1_000_000, | ||
| "claude-opus-4-7-20260416": 1_000_000, | ||
| "claude-opus-4-5": 200_000, | ||
| "claude-opus-4-5-20251101": 200_000, | ||
| "claude-opus-4-20250514": 200_000, | ||
| "claude-opus-4-1": 200_000, | ||
| "claude-opus-4-1-20250805": 200_000, | ||
| "claude-haiku-4-5": 200_000, | ||
| "claude-haiku-4-5-20251001": 200_000, | ||
| "claude-3-7-sonnet-20250219": 200_000, | ||
| "claude-3-5-sonnet-20241022": 200_000, | ||
| "claude-3-5-sonnet-20240620": 200_000, | ||
| "claude-3-5-haiku-20241022": 200_000, | ||
| "claude-3-opus-20240229": 200_000, | ||
| "claude-3-haiku-20240307": 200_000, | ||
| # Bedrock Anthropic (base model IDs — cross-region prefixes stripped by get_context_window_limit) | ||
| "anthropic.claude-sonnet-4-6": 1_000_000, | ||
| "anthropic.claude-sonnet-4-20250514-v1:0": 1_000_000, | ||
| "anthropic.claude-sonnet-4-5-20250929-v1:0": 200_000, | ||
| "anthropic.claude-opus-4-6-v1": 1_000_000, | ||
| "anthropic.claude-opus-4-7": 1_000_000, | ||
| "anthropic.claude-opus-4-5-20251101-v1:0": 200_000, | ||
| "anthropic.claude-opus-4-20250514-v1:0": 200_000, | ||
| "anthropic.claude-opus-4-1-20250805-v1:0": 200_000, | ||
| "anthropic.claude-haiku-4-5-20251001-v1:0": 200_000, | ||
| "anthropic.claude-haiku-4-5@20251001": 200_000, | ||
| "anthropic.claude-3-7-sonnet-20250219-v1:0": 200_000, | ||
| "anthropic.claude-3-7-sonnet-20240620-v1:0": 200_000, | ||
| "anthropic.claude-3-5-sonnet-20241022-v2:0": 200_000, | ||
| "anthropic.claude-3-5-sonnet-20240620-v1:0": 200_000, | ||
| "anthropic.claude-3-5-haiku-20241022-v1:0": 200_000, | ||
| "anthropic.claude-3-opus-20240229-v1:0": 200_000, | ||
| "anthropic.claude-3-haiku-20240307-v1:0": 200_000, | ||
| "anthropic.claude-3-sonnet-20240229-v1:0": 200_000, | ||
| "anthropic.claude-mythos-preview": 1_000_000, | ||
| # Bedrock Amazon Nova | ||
| "amazon.nova-pro-v1:0": 300_000, | ||
| "amazon.nova-lite-v1:0": 300_000, | ||
| "amazon.nova-micro-v1:0": 128_000, | ||
| "amazon.nova-premier-v1:0": 1_000_000, | ||
| "amazon.nova-2-lite-v1:0": 1_000_000, | ||
| "amazon.nova-2-pro-preview-20251202-v1:0": 1_000_000, | ||
| # OpenAI | ||
| "gpt-5.5": 1_050_000, | ||
| "gpt-5.5-pro": 1_050_000, | ||
| "gpt-5.4": 1_050_000, | ||
| "gpt-5.4-pro": 1_050_000, | ||
| "gpt-5.4-mini": 272_000, | ||
| "gpt-5.4-nano": 272_000, | ||
| "gpt-5.2": 272_000, | ||
| "gpt-5.2-pro": 272_000, | ||
| "gpt-5.1": 272_000, | ||
| "gpt-5": 272_000, | ||
| "gpt-5-mini": 272_000, | ||
| "gpt-5-nano": 272_000, | ||
| "gpt-5-pro": 128_000, | ||
| "gpt-4.1": 1_047_576, | ||
| "gpt-4.1-mini": 1_047_576, | ||
| "gpt-4.1-nano": 1_047_576, | ||
| "gpt-4o": 128_000, | ||
| "gpt-4o-mini": 128_000, | ||
| "gpt-4-turbo": 128_000, | ||
| "o3": 200_000, | ||
| "o3-mini": 200_000, | ||
| "o3-pro": 200_000, | ||
| "o4-mini": 200_000, | ||
| "o1": 200_000, | ||
| # Google Gemini | ||
| "gemini-2.5-flash": 1_048_576, | ||
| "gemini-2.5-flash-lite": 1_048_576, | ||
| "gemini-2.5-pro": 1_048_576, | ||
| "gemini-2.0-flash": 1_048_576, | ||
| "gemini-2.0-flash-lite": 1_048_576, | ||
| "gemini-3-pro-preview": 1_048_576, | ||
| "gemini-3-flash-preview": 1_048_576, | ||
| "gemini-3.1-pro-preview": 1_048_576, | ||
| "gemini-3.1-flash-lite-preview": 1_048_576, | ||
| # Mistral | ||
| "mistral-large-latest": 262_144, | ||
| "mistral-large-2512": 262_144, | ||
| "mistral-large-3": 262_144, | ||
| "mistral-medium-latest": 131_072, | ||
| "mistral-medium-2505": 131_072, | ||
| "mistral-small-latest": 131_072, | ||
| "mistral-small-3-2-2506": 131_072, | ||
| } | ||
|
|
||
|
|
||
| def get_context_window_limit(model_id: str) -> int | None: | ||
| """Look up the context window limit for a model ID. | ||
|
|
||
| For Bedrock cross-region model IDs (e.g. ``us.anthropic.claude-sonnet-4-6``), | ||
| the region prefix is stripped as a fallback if the direct lookup fails. | ||
|
|
||
| Args: | ||
| model_id: The model ID to look up. | ||
|
|
||
| Returns: | ||
|
opieter-aws marked this conversation as resolved.
|
||
| The context window limit in tokens, or None if not found. | ||
| """ | ||
| direct = _CONTEXT_WINDOW_LIMITS.get(model_id) | ||
| if direct is not None: | ||
| return direct | ||
|
|
||
| # Fallback: strip prefix before first dot and retry (handles cross-region prefixes) | ||
| dot_index = model_id.find(".") | ||
| if dot_index != -1: | ||
| stripped = model_id[dot_index + 1 :] | ||
| result = _CONTEXT_WINDOW_LIMITS.get(stripped) | ||
| if result is not None: | ||
| logger.debug( | ||
| "model_id=<%s>, stripped_id=<%s> | resolved context window limit via prefix strip", model_id, stripped | ||
| ) | ||
| return result | ||
|
|
||
| return None | ||
|
|
||
|
|
||
| def resolve_config_metadata(config: _C, model_id: str) -> _C: | ||
| """Resolve model metadata fields on a config dict from built-in lookup tables. | ||
|
|
||
| When ``context_window_limit`` is not explicitly set, looks it up from the built-in table. | ||
| Explicit values pass through unchanged. Returns a new dict only when resolution adds a field; | ||
| otherwise returns the original config to avoid unnecessary allocation. | ||
|
|
||
| Args: | ||
| config: The stored model config dict. | ||
| model_id: The model ID to look up. | ||
|
|
||
| Returns: | ||
| The config with resolved metadata, or the original config if nothing to resolve. | ||
| """ | ||
| if "context_window_limit" in config: | ||
| return config | ||
|
|
||
| limit = get_context_window_limit(model_id) | ||
| if limit is None: | ||
| return config | ||
|
lizradway marked this conversation as resolved.
|
||
|
|
||
| return {**config, "context_window_limit": limit} # type: ignore[return-value] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.