feat(session/llm): scaffold OpenAI Realtime (WebSocket) transport#29359
Draft
WonderJL wants to merge 1 commit into
Draft
feat(session/llm): scaffold OpenAI Realtime (WebSocket) transport#29359WonderJL wants to merge 1 commit into
WonderJL wants to merge 1 commit into
Conversation
Adds a scaffold for the OpenAI Realtime API (wss://api.openai.com/v1/realtime) behind a new `experimental.openai_realtime` config flag. Not wired into the live stream path yet — see REALTIME-DESIGN.md for protocol mapping, lifecycle plan, and open questions for maintainers. What ships: - realtime.ts: pure event mapper (Realtime JSON -> LLMEvent) + adapterState + a stream() stub that fails with RealtimeNotImplemented so accidental wiring is loud. - Config flag (default off) so the live wiring can land incrementally. - Unit tests for the mapper covering text turns, function-call argument streaming, unknown-event passthrough, and failed-response status. Default behavior unchanged.
Contributor
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
Contributor
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
experimental.openai_realtimeflag (defaultfalse) — no behavior change until the flag is enabled and the wiring lands.LLMEvent) with unit tests, a stubstream()that fails loudly, and a design note documenting the protocol mapping and open questions.Why a scaffold, not a full transport?
The Realtime event surface is wide and additive; the right dispatch shape inside
session/llm.tsis a maintainer call. Landing the scaffold first lets reviewers shape that integration without a churny diff. This PR is intentionally inert at runtime —stream()throwsRealtimeNotImplementedif accidentally invoked.What changes
packages/opencode/src/session/llm/realtime.ts(new) —adapterState(),toLLMEvents(),REALTIME_URL,RealtimeNotImplemented, stubstream(). The mapper shape mirrorsai-sdk.ts'sadapterState()/toLLMEvents()on purpose so the eventual dispatch site treats both adapters uniformly.packages/opencode/src/session/llm/REALTIME-DESIGN.md(new) — scope, non-goals, protocol mapping table, lifecycle plan, auth, open questions for maintainers.packages/opencode/src/config/config.ts— addsexperimental.openai_realtime: Schema.optional(Schema.Boolean).packages/opencode/test/session/llm-realtime.test.ts(new) — four unit tests covering text turns, function-call argument streaming, unknown-event passthrough, and failed-response status.Realtime → LLMEvent mapping (summary)
response.createdstepStartresponse.output_item.added(message)textStartresponse.output_text.deltatextDeltaresponse.output_text.donetextEndresponse.output_item.added(function_call)toolInputStartresponse.function_call_arguments.deltatoolInputDeltaresponse.function_call_arguments.donetoolInputEndresponse.output_item.done(function_call)toolCall(with parsedarguments)response.donestepFinish+finishUnknown event types (audio., rate_limits., etc.) drop to
[]rather than fail.Open questions (called out in the design note)
providerID(openai-realtime) or keep it underopenaiand switch transport by flag?ProviderTransform.toolsfor tool-schema translation, or build a Realtime-specific path?gpt-5*reasoning items (which aren't first-class in Realtime yet) — block atstatus()or pass through?Test plan
bun run typecheck(packages/opencode) — clean.bun test test/session/llm-realtime.test.ts— 4/4 pass.gpt-4o-realtime-preview/gpt-5.x-realtimewith a tool-call prompt.Notes for reviewers
Stream.timeoutOrElseidle guard so both transports fail-fast on stalls.