Skip to content

feat: add bidirectional live sessions (SendLiveMessage)#1975

Open
herczyn wants to merge 1 commit into
a2aproject:devfrom
herczyn:bidi-live-session
Open

feat: add bidirectional live sessions (SendLiveMessage)#1975
herczyn wants to merge 1 commit into
a2aproject:devfrom
herczyn:bidi-live-session

Conversation

@herczyn

@herczyn herczyn commented Jun 23, 2026

Copy link
Copy Markdown
Member

Description

A2A v1.0 supports server→client streaming only (SendStreamingMessage): a client cannot stream additional input on the same call, and there is no full-duplex media channel for real-time audio/video. This PR adds a bidirectional live session so a client and an agent can exchange continuous media and multiple turns over a single connection.

The change is additive and capability-gated — existing methods and types are unchanged, so v1.0 clients/servers are unaffected. In particular, the existing StreamResponse is embedded in the new LiveResponse rather than modified, so SendStreamingMessage / SubscribeToTask are untouched.

What's added (specification/a2a.proto)

  • SendLiveMessage(stream LiveRequest) returns (stream LiveResponse) — a bidirectional streaming RPC (requires a bidi-capable transport, e.g. gRPC).
  • LiveRequest / LiveResponse — two-arm wrappers that compose existing types:
    • LiveRequest = oneof { SendMessageRequest message | MediaFrame frame }
    • LiveResponse = oneof { StreamResponse event | MediaFrame frame }
  • MediaFrame — a media-plane frame (not a turn; no message_id/role) carrying audio/video/binary in Part.raw, with start/delta/stop/pause/interrupt (barge-in) events and per-modality sequencing; plus ModalitySpec and the Directionality enum (SDP-style directions).
  • AgentCapabilities.bidi_streaming (live/duplex marker) and AgentCapabilities.live_session (LiveSessionCapabilities, per-modality media support).
  • SendMessageConfiguration.live_session (LiveSessionOffer) — the media offer carried on the opening message.

Design notes

  • The stream opens with a SendMessageRequest (existing contract); a message frame is semantically identical to a standalone SendMessageRequest, so this does not change the semantics of sending a message to a running task.
  • Media negotiation is capability-based (Agent Card + offer in SendMessageConfiguration) — no separate signaling/control message family: acceptance is a successful stream, rejection is a standard error, teardown is the transport close.
  • The session binds to the Task from the opening message; transports come from the existing supported_interfaces.

Scope / follow-ups

This PR is the protocol (proto) definition. A WebSocket binding (for browser/edge clients that can't use gRPC bidi) and the prose update to docs/specification.md are intended as separate follow-ups, to keep this change focused and shippable incrementally.

Validation

  • buf lint passes locally (no new findings).
  • Additive-only (new field numbers, new messages, new RPC, new enum) — no breaking changes.

Related to #656, #1549, #1864.

Add a bidirectional streaming `SendLiveMessage` RPC and supporting types so
clients and agents can exchange continuous media (audio/video) and multiple
turns over a single live session, beyond the existing server-streaming
`SendStreamingMessage`.

The change is additive and capability-gated; existing methods and types are
unchanged (the existing `StreamResponse` is embedded, not modified):

- `SendLiveMessage(stream LiveRequest) returns (stream LiveResponse)`.
- `LiveRequest` / `LiveResponse` wrappers composing `SendMessageRequest` /
  `StreamResponse` with a media arm.
- `MediaFrame` media-plane frame (start/delta/stop/pause/interrupt events and
  per-modality sequencing), `ModalitySpec`, and the `Directionality` enum.
- `AgentCapabilities.bidi_streaming` (live/duplex marker) and `live_session`
  (per-modality media support).
- `SendMessageConfiguration.live_session` media offer.
@herczyn
herczyn requested a review from a team as a code owner June 23, 2026 12:32

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces bidirectional live sessions to the A2A protocol by adding the SendLiveMessage streaming RPC and its associated data structures (LiveRequest, LiveResponse, MediaFrame, and configuration/capability messages). The review feedback highlights two key improvement opportunities: first, removing the REQUIRED constraint on the part field in MediaFrame to avoid forcing dummy payloads for control-only events (like pause or interrupt); second, renaming the message field in LiveRequest to message_request to prevent confusing nested references like request.message.message in downstream code.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread specification/a2a.proto
message MediaFrame {
// The media content of this frame. The bytes typically reside in `Part.raw`
// with `Part.media_type` set (e.g., "audio/l16;rate=16000").
Part part = 1 [(google.api.field_behavior) = REQUIRED];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since MediaFrame is also used for control and lifecycle events (such as EVENT_MEDIA_STOP, EVENT_MEDIA_PAUSE, and EVENT_INTERRUPT) which do not carry any media payload, marking the part field as REQUIRED forces callers to construct dummy Part objects for these events. Removing the REQUIRED annotation makes the API cleaner and more robust for control-only frames.

Suggested change
Part part = 1 [(google.api.field_behavior) = REQUIRED];
Part part = 1;

Comment thread specification/a2a.proto
// A turn, or the opening message of the session. Semantically identical to a
// standalone `SendMessageRequest`. The opening message MAY carry a media
// offer in its `SendMessageConfiguration.live_session`.
SendMessageRequest message = 1;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

Naming this field message when its type is SendMessageRequest can be confusing, especially since SendMessageRequest itself contains a field named message of type Message (leading to repetitive patterns like request.message.message in code). Renaming it to message_request or send_message_request improves clarity and consistency with LiveResponse's event field.

Suggested change
SendMessageRequest message = 1;
SendMessageRequest message_request = 1;

@Tehsmash

Copy link
Copy Markdown
Contributor

This seems very similar to https://github.com/a2aproject/A2A/pull/1549/changes, which was originally based on a PR/proposal from @mikeas1 #1120

I think we need to consolidate into a single epic issue with design discussion instead of a bunch of PRs, because otherwise we end up missing things which have already been discussed in other places, for example 1549 handles the reconnection to a stream but it doesn't look like this one covers that yet.

We should look at how this relates to Tasks, there are a number of issues open on multi-turn for tasks, I've proposed a consolidated issue for this here: #1942 (comment) I think decisions made on this will influence how/where we hook BiDi into the protocol.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants