-
Notifications
You must be signed in to change notification settings - Fork 212
fix(router-provider): fetch model metadata before context management decisions #1053
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
101b490
47e0732
885d8e0
c7ead3e
1264874
f8302c0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -61,6 +61,21 @@ export abstract class RouterProvider extends BaseProvider { | |
| return this.getModel() | ||
| } | ||
|
|
||
| private modelFetchPromise?: Promise<void> | ||
|
|
||
| async ensureModelFetched(): Promise<void> { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For auth-scoped providers, does the first request now fetch the model list twice — once here and again unconditionally in |
||
| if (Object.keys(this.models).length === 0) { | ||
| const fetchPromise = (this.modelFetchPromise ??= this.fetchModel().then(() => undefined)) | ||
| try { | ||
| await fetchPromise | ||
| } finally { | ||
| if (this.modelFetchPromise === fetchPromise) { | ||
| this.modelFetchPromise = undefined | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| override getModel(): { id: string; info: ModelInfo } { | ||
| // Use `||` (not `??`) so an empty-string modelId also falls back to the default, | ||
| // guaranteeing a non-empty id rather than forwarding "" to the API as an invalid | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2762,6 +2762,8 @@ export class Task extends EventEmitter<TaskEvents> implements TaskLike { | |
|
|
||
| await this.diffViewProvider.reset() | ||
|
|
||
| await this.api.ensureModelFetched?.() | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The litellm/deepseek/moonshot fetchers re-throw, so a failure here would reject into the catch at L3743, which |
||
|
|
||
| // Cache model info once per API request to avoid repeated calls during streaming | ||
| // This is especially important for tools and background usage collection | ||
| this.cachedStreamingModel = this.api.getModel() | ||
|
|
@@ -3842,6 +3844,7 @@ export class Task extends EventEmitter<TaskEvents> implements TaskLike { | |
| const { profileThresholds = {}, mode, apiConfiguration } = state ?? {} | ||
|
|
||
| const { contextTokens } = this.getTokenUsage() | ||
| await this.api.ensureModelFetched?.() | ||
| const modelInfo = this.api.getModel().info | ||
|
|
||
| const maxTokens = getModelMaxOutputTokens({ | ||
|
|
@@ -4042,6 +4045,7 @@ export class Task extends EventEmitter<TaskEvents> implements TaskLike { | |
| const { contextTokens } = this.getTokenUsage() | ||
|
|
||
| if (contextTokens) { | ||
| await this.api.ensureModelFetched?.() | ||
| const modelInfo = this.api.getModel().info | ||
|
coderabbitai[bot] marked this conversation as resolved.
Outdated
|
||
|
|
||
| const maxTokens = getModelMaxOutputTokens({ | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the rejection path covered anywhere? If
getModelsrejects once and thefinallycleanup in router-provider.ts (L71-74) ever regresses, the stored rejected promise would poison every subsequent call. A reject-then-resolve test here would lock that in.