Skip to content

Commit eb2b902

Browse files
allquixoticoz-agent
andcommitted
Bedrock max-tokens UX: probe via inference profile, widen slider, fix layout
Issues fixed (reported by allquixotic): 1. The Detect probe failed with "on-demand throughput isn't supported" when the user selected an inference profile (e.g. `us.anthropic.claude-opus-4-7`). The probe was sending the bare base model id to AWS Bedrock. Now the probe uses a new `resolveBedrockInvokeTargetId` helper that mirrors the runtime's target resolution (system / application profile, custom ARN, or foundation model with optional global / cross-region prefix). 2. The Max Tokens slider was clamped to ~16K even when the model's static cap was much higher (e.g. Opus 4.7 at 128K). Root cause: `useSelectedModel`'s Bedrock branch was passing `modelMaxTokens` (the user's request-time slider value) to `resolveBedrockModelInfo`, which clobbered `info.maxTokens` and capped the slider at the user's previous selection. The webview now passes only `awsModelMaxOutputTokens` (the empirical Detect override) to widen the ceiling; the runtime path keeps applying `modelMaxTokens` for the actual API request via `getModelById`. 3. The "Detect max output tokens" button text was clipped by the numeric field. `MaxOutputTokensControl` now uses `flex-1 min-w-0` for the slider column and renders the provider-specific extra slot on its own line below the slider+input row, so wide labels can no longer be occluded by the text field's absolute focus border. Bumps src/package.json to 3.53.13. Co-Authored-By: Oz <oz-agent@warp.dev>
1 parent 989b3e1 commit eb2b902

5 files changed

Lines changed: 120 additions & 17 deletions

File tree

packages/types/src/providers/bedrock.ts

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -792,6 +792,80 @@ export const usesBedrockDefault1MContext = (baseModelId?: string) =>
792792
!!baseModelId &&
793793
BEDROCK_1M_CONTEXT_DEFAULT_MODEL_IDS.includes(baseModelId as (typeof BEDROCK_1M_CONTEXT_DEFAULT_MODEL_IDS)[number])
794794

795+
const getBedrockRegionPrefix = (region?: string): string | undefined => {
796+
if (!region) return undefined
797+
for (const [pattern, prefix] of AWS_INFERENCE_PROFILE_MAPPING) {
798+
if (region.startsWith(pattern)) return prefix
799+
}
800+
return undefined
801+
}
802+
803+
/**
804+
* Returns the AWS-side target id that the Bedrock runtime would invoke against, given a
805+
* provider-settings snapshot. Mirrors the resolution `AwsBedrockHandler.getModel()` does
806+
* before sending a Converse command, so that callers outside the runtime (e.g. the
807+
* settings-page max-tokens probe) can hit the exact same target the user's profile is
808+
* configured to invoke.
809+
*
810+
* Resolution order:
811+
* 1. `awsCustomArn` wins if present (the user provided a literal ARN).
812+
* 2. If `awsBedrockTargetKind` (or the inferred kind) is an explicit profile / prompt
813+
* router selection, use `awsBedrockInvokeTarget` verbatim, stripping the synthetic
814+
* `:1m` UI suffix.
815+
* 3. Otherwise we have a foundation-model selection. Apply Global Inference (`global.`)
816+
* when enabled and supported, else apply the regional cross-region inference prefix
817+
* (`us.`, `eu.`, etc.) when enabled.
818+
*/
819+
export interface ResolveBedrockInvokeTargetIdOptions {
820+
awsCustomArn?: string
821+
awsBedrockInvokeTarget?: string
822+
awsBedrockTargetKind?: BedrockInvokeTargetKind
823+
apiModelId?: string
824+
awsUseGlobalInference?: boolean
825+
awsUseCrossRegionInference?: boolean
826+
awsRegion?: string
827+
}
828+
829+
export const resolveBedrockInvokeTargetId = (options: ResolveBedrockInvokeTargetIdOptions): string => {
830+
if (options.awsCustomArn) {
831+
return options.awsCustomArn
832+
}
833+
834+
const configuredTargetId = options.awsBedrockInvokeTarget || options.apiModelId || ""
835+
const explicitKind = options.awsBedrockTargetKind
836+
const targetKind = inferBedrockInvokeTargetKind({
837+
targetId: configuredTargetId,
838+
explicitKind,
839+
})
840+
841+
if (
842+
targetKind === "system-profile" ||
843+
targetKind === "application-profile" ||
844+
targetKind === "prompt-router" ||
845+
targetKind === "custom-arn"
846+
) {
847+
return stripBedrock1MContextSuffix(configuredTargetId)
848+
}
849+
850+
const baseModelId = parseBedrockBaseModelId(configuredTargetId)
851+
852+
if (
853+
options.awsUseGlobalInference &&
854+
BEDROCK_GLOBAL_INFERENCE_MODEL_IDS.includes(baseModelId as (typeof BEDROCK_GLOBAL_INFERENCE_MODEL_IDS)[number])
855+
) {
856+
return `global.${baseModelId}`
857+
}
858+
859+
if (options.awsUseCrossRegionInference) {
860+
const prefix = getBedrockRegionPrefix(options.awsRegion)
861+
if (prefix) {
862+
return `${prefix}${baseModelId}`
863+
}
864+
}
865+
866+
return baseModelId
867+
}
868+
795869
export const shouldUseBedrock1MContext = ({
796870
targetId,
797871
baseModelId,

src/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"displayName": "%extension.displayName%",
44
"description": "%extension.description%",
55
"publisher": "allquixotic",
6-
"version": "3.53.12",
6+
"version": "3.53.13",
77
"icon": "assets/icons/icon.png",
88
"galleryBanner": {
99
"color": "#617A91",

webview-ui/src/components/settings/MaxOutputTokensControl.tsx

Lines changed: 23 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -82,29 +82,40 @@ export const MaxOutputTokensControl = ({
8282

8383
return (
8484
<div className="flex flex-col gap-2">
85+
{/*
86+
* Row 1: slider + numeric input. Slider gets `flex-1 min-w-0` so it can shrink without
87+
* pushing siblings off-screen; the number field is a fixed-width flex-shrink-0 column.
88+
*/}
8589
<div className="flex items-center gap-3">
86-
<Slider
87-
min={min}
88-
max={sliderMax}
89-
step={step}
90-
value={[sliderValue]}
91-
onValueChange={handleSliderChange}
92-
disabled={disabled}
93-
data-testid="max-output-tokens-slider"
94-
/>
95-
<div className="flex-shrink-0">
90+
<div className="flex-1 min-w-0">
91+
<Slider
92+
min={min}
93+
max={sliderMax}
94+
step={step}
95+
value={[sliderValue]}
96+
onValueChange={handleSliderChange}
97+
disabled={disabled}
98+
data-testid="max-output-tokens-slider"
99+
/>
100+
</div>
101+
<div className="flex-shrink-0" style={{ width: "10ch" }}>
96102
<FormattedTextField
97103
value={effectiveValue}
98104
onValueChange={handleInputChange}
99105
formatter={unlimitedIntegerFormatter}
100106
disabled={disabled}
101107
aria-label={inputAriaLabel}
102-
style={{ width: "9ch" }}
108+
style={{ width: "100%" }}
103109
data-testid="max-output-tokens-input"
104110
/>
105111
</div>
106-
{extraSlot}
107112
</div>
113+
{/*
114+
* Row 2: provider-specific actions (e.g. the Bedrock probe button). Rendered on its own
115+
* line so wide button labels can't be clipped by the absolutely-positioned focus border
116+
* of the VSCode text field above.
117+
*/}
118+
{extraSlot ? <div className="flex flex-wrap items-center gap-2">{extraSlot}</div> : null}
108119
{helperText ? <div className="text-sm text-vscode-descriptionForeground">{helperText}</div> : null}
109120
</div>
110121
)

webview-ui/src/components/settings/providers/BedrockMaxTokensProbeButton.tsx

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { useCallback, type ReactNode } from "react"
22

3-
import type { ProviderSettings } from "@roo-code/types"
3+
import { resolveBedrockInvokeTargetId, type ProviderSettings } from "@roo-code/types"
44

55
import { Button, StandardTooltip } from "@src/components/ui"
66
import { useAppTranslation } from "@src/i18n/TranslationContext"
@@ -16,7 +16,12 @@ interface BedrockMaxTokensProbeButtonProps {
1616
value: ProviderSettings[K],
1717
isUserAction?: boolean,
1818
) => void
19-
/** Resolved model id (may differ from `apiConfiguration.apiModelId` for inference-profile targets). */
19+
/**
20+
* Optional UI hint of the resolved base model id (only used as a fallback when no AWS-side
21+
* invoke target can be derived from `apiConfiguration`). The probe always sends the actual
22+
* AWS target id (system profile, application profile, ARN, or prefixed foundation model)
23+
* computed via {@link resolveBedrockInvokeTargetId}.
24+
*/
2025
modelId?: string
2126
}
2227

@@ -80,7 +85,14 @@ export const useBedrockMaxTokensProbeUi = ({
8085
const { probe, isProbing, lastResult, lastError } = useBedrockMaxTokensProbe()
8186

8287
const onDetect = useCallback(async () => {
83-
const targetModelId = modelId || apiConfiguration.apiModelId || ""
88+
// Mirror the runtime's invoke-target resolution (system/application profile id,
89+
// custom ARN, or foundation model with optional cross-region/global prefix) so the
90+
// probe hits the same AWS target the actual chat requests would. Without this we'd
91+
// send the bare base model id (e.g. `anthropic.claude-opus-4-7`) and AWS rejects
92+
// it with "on-demand throughput isn't supported" for models that require an
93+
// inference profile.
94+
const resolvedTargetId = resolveBedrockInvokeTargetId(apiConfiguration)
95+
const targetModelId = resolvedTargetId || modelId || apiConfiguration.apiModelId || ""
8496
if (!targetModelId) {
8597
return
8698
}

webview-ui/src/components/ui/hooks/useSelectedModel.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,13 @@ function getSelectedModel({
188188
baseModelId: apiConfiguration.apiModelId,
189189
targetId,
190190
optIn1MContext: apiConfiguration.awsBedrock1MContext,
191-
modelMaxTokens: apiConfiguration.modelMaxTokens,
191+
// Intentionally NOT passing `modelMaxTokens` here. That's the user's request-time
192+
// slider value; passing it would clamp `info.maxTokens` to the user's current
193+
// preference and prevent the slider's upper bound from showing the model's true
194+
// ceiling. The runtime path (AwsBedrockHandler.getModelById) still applies it for
195+
// the actual API request. The empirical detect-button override is honored via
196+
// `maxOutputTokensOverride` so a user-confirmed cap widens the slider as expected.
197+
maxOutputTokensOverride: apiConfiguration.awsModelMaxOutputTokens,
192198
contextWindowOverride: apiConfiguration.awsModelContextWindow,
193199
})
194200
const displayId = apiConfiguration.apiModelId ?? resolved.baseModelId

0 commit comments

Comments
 (0)