fix: correct finishReason in Google streaming#109
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the API gateway’s streaming behavior to better match upstream provider specs—most notably Google’s streamGenerateContent finish semantics—while also adjusting several streaming-related concerns (cancellation, error mapping, token counting) across endpoints.
Changes:
- Google streaming: stop emitting
finishReason: 'STOP'on every chunk and instead emit the stop reason only at the end of the stream. - OpenAI-compatible streaming: add early request cancellation on client disconnect, map
LanguageModelErrorto more appropriate HTTP statuses, and reuse token counts for usage + logging. - Internal refactors: ensure
CancellationTokenSourceinstances are disposed; align WebSocket enablement default with configuration schema.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| parts: [{ text: textValue }] | ||
| }, | ||
| finishReason: 'STOP', | ||
| finishReason: null, |
Comment on lines
2961
to
2967
| } catch (error) { | ||
| const errorChunk = { | ||
| error: { | ||
| message: error instanceof Error ? error.message : 'Unknown error', | ||
| type: 'server_error' | ||
| type: error instanceof ApiError ? error.type : 'server_error', | ||
| code: error instanceof ApiError ? error.code : undefined | ||
| } |
Comment on lines
+2747
to
+2753
| // Create CTS early so client-disconnect cancels the model request (#98) | ||
| const cts = new vscode.CancellationTokenSource(); | ||
| req.on('close', () => { | ||
| cts.cancel(); | ||
| this.logInfo(`[OpenAI] Client disconnected, cancelling request ${logRequestId || ''}`); | ||
| }); | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #100