feat: add bidirectional live sessions (SendLiveMessage)#1975
Conversation
Add a bidirectional streaming `SendLiveMessage` RPC and supporting types so clients and agents can exchange continuous media (audio/video) and multiple turns over a single live session, beyond the existing server-streaming `SendStreamingMessage`. The change is additive and capability-gated; existing methods and types are unchanged (the existing `StreamResponse` is embedded, not modified): - `SendLiveMessage(stream LiveRequest) returns (stream LiveResponse)`. - `LiveRequest` / `LiveResponse` wrappers composing `SendMessageRequest` / `StreamResponse` with a media arm. - `MediaFrame` media-plane frame (start/delta/stop/pause/interrupt events and per-modality sequencing), `ModalitySpec`, and the `Directionality` enum. - `AgentCapabilities.bidi_streaming` (live/duplex marker) and `live_session` (per-modality media support). - `SendMessageConfiguration.live_session` media offer.
There was a problem hiding this comment.
Code Review
This pull request introduces bidirectional live sessions to the A2A protocol by adding the SendLiveMessage streaming RPC and its associated data structures (LiveRequest, LiveResponse, MediaFrame, and configuration/capability messages). The review feedback highlights two key improvement opportunities: first, removing the REQUIRED constraint on the part field in MediaFrame to avoid forcing dummy payloads for control-only events (like pause or interrupt); second, renaming the message field in LiveRequest to message_request to prevent confusing nested references like request.message.message in downstream code.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| message MediaFrame { | ||
| // The media content of this frame. The bytes typically reside in `Part.raw` | ||
| // with `Part.media_type` set (e.g., "audio/l16;rate=16000"). | ||
| Part part = 1 [(google.api.field_behavior) = REQUIRED]; |
There was a problem hiding this comment.
Since MediaFrame is also used for control and lifecycle events (such as EVENT_MEDIA_STOP, EVENT_MEDIA_PAUSE, and EVENT_INTERRUPT) which do not carry any media payload, marking the part field as REQUIRED forces callers to construct dummy Part objects for these events. Removing the REQUIRED annotation makes the API cleaner and more robust for control-only frames.
| Part part = 1 [(google.api.field_behavior) = REQUIRED]; | |
| Part part = 1; |
| // A turn, or the opening message of the session. Semantically identical to a | ||
| // standalone `SendMessageRequest`. The opening message MAY carry a media | ||
| // offer in its `SendMessageConfiguration.live_session`. | ||
| SendMessageRequest message = 1; |
There was a problem hiding this comment.
Naming this field message when its type is SendMessageRequest can be confusing, especially since SendMessageRequest itself contains a field named message of type Message (leading to repetitive patterns like request.message.message in code). Renaming it to message_request or send_message_request improves clarity and consistency with LiveResponse's event field.
| SendMessageRequest message = 1; | |
| SendMessageRequest message_request = 1; |
|
This seems very similar to https://github.com/a2aproject/A2A/pull/1549/changes, which was originally based on a PR/proposal from @mikeas1 #1120 I think we need to consolidate into a single epic issue with design discussion instead of a bunch of PRs, because otherwise we end up missing things which have already been discussed in other places, for example 1549 handles the reconnection to a stream but it doesn't look like this one covers that yet. We should look at how this relates to Tasks, there are a number of issues open on multi-turn for tasks, I've proposed a consolidated issue for this here: #1942 (comment) I think decisions made on this will influence how/where we hook BiDi into the protocol. |
Description
A2A v1.0 supports server→client streaming only (
SendStreamingMessage): a client cannot stream additional input on the same call, and there is no full-duplex media channel for real-time audio/video. This PR adds a bidirectional live session so a client and an agent can exchange continuous media and multiple turns over a single connection.The change is additive and capability-gated — existing methods and types are unchanged, so v1.0 clients/servers are unaffected. In particular, the existing
StreamResponseis embedded in the newLiveResponserather than modified, soSendStreamingMessage/SubscribeToTaskare untouched.What's added (
specification/a2a.proto)SendLiveMessage(stream LiveRequest) returns (stream LiveResponse)— a bidirectional streaming RPC (requires a bidi-capable transport, e.g. gRPC).LiveRequest/LiveResponse— two-arm wrappers that compose existing types:LiveRequest = oneof { SendMessageRequest message | MediaFrame frame }LiveResponse = oneof { StreamResponse event | MediaFrame frame }MediaFrame— a media-plane frame (not a turn; nomessage_id/role) carrying audio/video/binary inPart.raw, withstart/delta/stop/pause/interrupt(barge-in) events and per-modality sequencing; plusModalitySpecand theDirectionalityenum (SDP-style directions).AgentCapabilities.bidi_streaming(live/duplex marker) andAgentCapabilities.live_session(LiveSessionCapabilities, per-modality media support).SendMessageConfiguration.live_session(LiveSessionOffer) — the media offer carried on the opening message.Design notes
SendMessageRequest(existing contract); amessageframe is semantically identical to a standaloneSendMessageRequest, so this does not change the semantics of sending a message to a running task.SendMessageConfiguration) — no separate signaling/control message family: acceptance is a successful stream, rejection is a standard error, teardown is the transport close.Taskfrom the opening message; transports come from the existingsupported_interfaces.Scope / follow-ups
This PR is the protocol (proto) definition. A WebSocket binding (for browser/edge clients that can't use gRPC bidi) and the prose update to
docs/specification.mdare intended as separate follow-ups, to keep this change focused and shippable incrementally.Validation
buf lintpasses locally (no new findings).Related to #656, #1549, #1864.