Skip to content

Commit e4ea2dd

Browse files
秦奇qwencoder
秦奇
and
qwencoder
committed
docs(daemon): design — ACP-over-HTTP → official ACP Streamable HTTP transport
Adds design for a second northbound transport on `qwen serve` implementing the official ACP Streamable HTTP transport (RFD QwenLM#721) at a single /acp endpoint, reusing the existing HttpAcpBridge + EventBus. Dual-transport (additive, REST stays); custom features via _qwen/ extension methods. Generated with AI Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
1 parent 0c04309 commit e4ea2dd

1 file changed

Lines changed: 315 additions & 0 deletions

File tree

‎docs/design/daemon-acp-http/README.md‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
# Daemon ACP-over-HTTP → Official ACP Streamable HTTP Transport
2+
3+
> Targets `daemon_mode_b_main`. Branch: `feat/daemon-acp-http-streamable`.
4+
> Author: arnoo.gao. Date: 2026-05-24. Status: **Design v1 → implementation**.
5+
> Design-first per repo workflow: this doc lands before/with the implementation PR so the wire contract is reviewable.
6+
7+
---
8+
9+
## 0. TL;DR
10+
11+
The daemon (`qwen serve`) today speaks a **bespoke REST + SSE** dialect to web/SDK
12+
clients, while speaking **real ACP JSON-RPC over stdio** to the spawned `qwen --acp`
13+
child. This proposal adds a **second northbound transport** that implements the
14+
**official ACP Streamable HTTP transport** (RFD #721) at a single `/acp` endpoint,
15+
so any ACP-native client (Zed, Goose, future SDKs) can drive the daemon directly
16+
over the standard protocol — no qwen-specific REST knowledge required.
17+
18+
**Decision: dual-transport, additive.** The new `/acp` endpoint is mounted
19+
alongside the existing REST surface, reusing the same `HttpAcpBridge` +
20+
`EventBus` underneath. The REST API is *not* removed. Rationale in §6.
21+
22+
**Decision: extension namespace = `_qwen/…`** (single-underscore prefix, the
23+
ACP-spec-reserved form for custom methods) for daemon features that have no
24+
standard ACP method (model switch, workspace introspection, heartbeat,
25+
multi-client permission policy, SSE backpressure tuning). Rationale in §5.
26+
27+
A complete, locally-runnable reference implementation ships in this PR
28+
(`packages/cli/src/serve/acpHttp/`) plus a verification harness
29+
(`scripts/acp-http-smoke.mjs`).
30+
31+
---
32+
33+
## 1. Background — what "ACP over HTTP" means today
34+
35+
Three tiers (verified at commit `0c0430939`):
36+
37+
```
38+
┌──────────────┐ bespoke REST + SSE (HTTP/1.1) ┌────────────┐ ACP JSON-RPC ┌──────────────┐
39+
│ web / SDK │ ───────────────────────────────► │ qwen │ (stdio NDJSON) │ qwen --acp │
40+
│ client │ ◄─── GET /session/:id/events ──── │ serve │ ◄─────────────► │ child (Agent)│
41+
│ (ACP client) │ (text/event-stream) │ (daemon) │ ndJsonStream │ │
42+
└──────────────┘ └────────────┘ └──────────────┘
43+
northbound: NOT ACP wire bridge southbound: real ACP
44+
```
45+
46+
### 1.1 Northbound (client ↔ daemon) — bespoke, today
47+
48+
- Express 5 app in `packages/cli/src/serve/server.ts` (~30 routes).
49+
- Discrete REST verbs, **not** JSON-RPC:
50+
- `POST /session` (create), `POST /session/:id/prompt`, `POST /session/:id/cancel`,
51+
`POST /session/:id/load|resume`, `POST /session/:id/model`,
52+
`POST /session/:id/permission/:requestId`, `POST /session/:id/heartbeat`,
53+
`DELETE /session/:id`, plus `/workspace/*`, `/capabilities`, `/health`.
54+
- Server→client streaming: `GET /session/:id/events` → `text/event-stream`.
55+
- Frames: `id: <n>\nevent: <type>\ndata: <json>\n\n` (`server.ts:formatSseFrame`, ~2626).
56+
- Per-session **monotonic `id`** + `Last-Event-ID` resume backed by a
57+
ring-buffer `EventBus` (`acp-bridge/src/eventBus.ts`).
58+
- Event `type`s: `session_update`, `client_evicted`, `slow_client_warning`,
59+
`state_resync_required`, `stream_error`, …
60+
- Auth: `Authorization: Bearer <token>` (`serve/auth.ts`), CORS deny + host allowlist.
61+
- Backpressure: per-connection serialized write chain + 15 s heartbeat comments.
62+
63+
### 1.2 Southbound (daemon ↔ child) — already ACP
64+
65+
- `acp-bridge/src/spawnChannel.ts` spawns `qwen --acp`, wraps stdin/stdout with
66+
`ndJsonStream` from `@agentclientprotocol/sdk` (`^0.14.1`).
67+
- `acp-bridge/src/bridge.ts:729` `new ClientSideConnection(() => client, channel.stream)`
68+
— the daemon is the ACP **client**, the child is the ACP **agent**.
69+
- Extension methods already in use on this leg: `unstable_setSessionModel`,
70+
`unstable_resumeSession`, `unstable_listSessions` (`acp-integration/acpAgent.ts`).
71+
72+
### 1.3 Why migrate the northbound
73+
74+
- Every client (webui, TS SDK, Java SDK, Python SDK, VSCode companion) re-implements
75+
the bespoke REST mapping. An ACP-standard endpoint lets ACP-native editors attach
76+
with zero qwen-specific glue.
77+
- Aligns the daemon's remote surface with the protocol it already speaks internally.
78+
79+
---
80+
81+
## 2. Target: ACP Streamable HTTP (RFD #721)
82+
83+
Merged **Draft** RFD (`agentclientprotocol/agent-client-protocol#721`, merged 2026-04-22).
84+
Not yet normative; not yet in any SDK. We implement against the RFD wire design.
85+
86+
### 2.1 Endpoint & verbs (single `/acp`)
87+
88+
| Verb | Behavior |
89+
|------|----------|
90+
| `POST /acp` | Send JSON-RPC. `initialize` → **`200`** + JSON body (capabilities) and sets `Acp-Connection-Id`. All other requests/notifications → **`202 Accepted`**, empty body; the *response* (if any) is delivered on the matching long-lived SSE stream. |
91+
| `GET /acp` | Open a long-lived **SSE** stream. (`Upgrade: websocket` → WebSocket; **deferred**, see §7.) |
92+
| `DELETE /acp` | Terminate the connection → `202`. |
93+
94+
### 2.2 Two-tier long-lived streams
95+
96+
- **Connection-scoped stream**: `GET /acp` with header `Acp-Connection-Id`, no session
97+
header. Carries connection-level responses (`session/new`, `session/load`,
98+
`authenticate`) and connection-level notifications.
99+
- **Session-scoped stream**: `GET /acp` with `Acp-Connection-Id` **and** `Acp-Session-Id`.
100+
Carries `session/update` notifications, **agent→client requests**
101+
(`session/request_permission`, `fs/read_text_file`, …), and responses to
102+
session POSTs (`session/prompt`, `session/cancel`).
103+
104+
### 2.3 Identity (3 layers)
105+
106+
- `Acp-Connection-Id` (HTTP header) — transport binding, minted at `initialize`.
107+
- `Acp-Session-Id` (HTTP header) — required on session-scoped GET + session POSTs.
108+
- `sessionId` (JSON-RPC param) — inside method params (must match the header).
109+
110+
### 2.4 Divergences from MCP StreamableHTTP
111+
112+
ACP uses **long-lived** streams (not per-request SSE), **two** ID headers (connection
113+
vs session), `202`-for-non-initialize, HTTP/2-required, WebSocket-required-client. We
114+
borrow the single-endpoint + POST/GET-SSE + session-header skeleton but adapt to the
115+
long-lived dual-ID model. We do **not** reuse `@modelcontextprotocol/sdk`'s
116+
`StreamableHTTPServerTransport` (its per-request stream model and single
117+
`Mcp-Session-Id` don't fit).
118+
119+
### 2.5 Standard methods (confirmed from current schema)
120+
121+
- Client→Agent requests: `initialize`, `authenticate`, `session/new`, `session/load`,
122+
`session/prompt`, `session/resume`, `session/close`, `session/list`,
123+
`session/set_mode`, `session/set_config_option`, `logout`.
124+
- Client→Agent notification: `session/cancel`.
125+
- Agent→Client requests: `fs/read_text_file`, `fs/write_text_file`,
126+
`session/request_permission`, `terminal/create|output|wait_for_exit|kill|release`.
127+
- Agent→Client notification: `session/update`.
128+
129+
---
130+
131+
## 3. Architecture of the new transport
132+
133+
The daemon must present an **ACP Agent surface over HTTP** northbound, while it
134+
remains an ACP **client** to the child southbound. The `/acp` layer is therefore a
135+
**JSON-RPC router** that terminates the HTTP transport and bridges into the existing
136+
`HttpAcpBridge`.
137+
138+
```
139+
POST /acp (JSON-RPC requests/responses/notifs)
140+
client ──────────────────────────────────────────────► ┌───────────────────────────┐
141+
(editor) │ AcpHttpTransport │
142+
◄── GET /acp (connection-scoped SSE) ────────── │ - connection registry │
143+
◄── GET /acp (session-scoped SSE) ───────────── │ - JSON-RPC id correlation│
144+
│ - method dispatch │
145+
└────────────┬──────────────┘
146+
│ reuses
147+
┌────────────▼──────────────┐
148+
│ HttpAcpBridge + EventBus │ (unchanged)
149+
└────────────┬──────────────┘
150+
│ ACP stdio (unchanged)
151+
qwen --acp child
152+
```
153+
154+
### 3.1 New module layout (`packages/cli/src/serve/acpHttp/`)
155+
156+
| File | Responsibility |
157+
|------|----------------|
158+
| `index.ts` | `mountAcpHttp(app, bridge, opts)` — registers `/acp` routes on the existing Express app. |
159+
| `connectionRegistry.ts` | `Acp-Connection-Id` → `AcpConnection` (connection SSE writer, `Map<sessionId, SessionStream>`, pending agent→client requests by JSON-RPC id, monotonic id allocator). TTL + DELETE cleanup. |
160+
| `jsonRpc.ts` | JSON-RPC 2.0 parse/validate/serialize helpers; error codes (`-32600` etc.); `_qwen/` namespace guard. |
161+
| `dispatch.ts` | Maps inbound JSON-RPC methods → `HttpAcpBridge` calls. Maps `BridgeEvent`s → outbound JSON-RPC frames. The translation table (§4). |
162+
| `sseStream.ts` | Long-lived SSE writer (reuses the backpressure/heartbeat pattern from `server.ts`). Distinct from REST `/events` (different framing: full JSON-RPC objects, not qwen event envelopes). |
163+
164+
No change to `bridge.ts` / `eventBus.ts` (additive consumer only).
165+
166+
### 3.2 Connection & session lifecycle
167+
168+
1. `POST /acp {initialize}` → mint `connectionId`, create `AcpConnection`, reply `200`
169+
with `{protocolVersion, agentCapabilities, _meta:{qwen:{…}}}` + `Acp-Connection-Id` header.
170+
2. Client opens `GET /acp` (connection-scoped) carrying `Acp-Connection-Id`.
171+
3. `POST /acp {session/new}` → `202`; daemon calls `bridge.createSession(...)`; pushes
172+
the JSON-RPC response (with `sessionId`) down the **connection** stream.
173+
4. Client opens `GET /acp` (session-scoped) with `Acp-Connection-Id`+`Acp-Session-Id`;
174+
daemon `bridge.subscribeEvents(sessionId)` and pipes translated frames.
175+
5. `POST /acp {session/prompt}` → `202`; `bridge.sendPrompt(...)`; `session/update`
176+
notifications stream live on the session stream; the final prompt **response**
177+
(`{id, result:{stopReason}}`) is pushed on the session stream when it settles.
178+
6. Agent→client request (e.g. `session/request_permission`) is emitted as a JSON-RPC
179+
**request** on the session stream with a daemon-allocated id; the client answers via
180+
`POST /acp {id, result}`; `dispatch` resolves it through the bridge's permission API.
181+
7. `DELETE /acp` (or connection-stream close + TTL) tears down sessions/subscriptions.
182+
183+
---
184+
185+
## 4. Translation table (bridge ⇄ ACP/HTTP)
186+
187+
### 4.1 Inbound (client POST → bridge)
188+
189+
| ACP method | Bridge call | Response routed to |
190+
|------------|-------------|--------------------|
191+
| `initialize` | (none; capabilities from `capabilities.ts`) | inline `200` |
192+
| `authenticate` | existing auth provider (`serve/auth/*`) | connection stream |
193+
| `session/new` | `bridge.createSession` | connection stream |
194+
| `session/load` / `session/resume` | `bridge.restoreSession('load'|'resume')` | connection stream |
195+
| `session/prompt` | `bridge.sendPrompt` | session stream (deferred until settle) |
196+
| `session/cancel` (notif) | `bridge.cancel` | — |
197+
| `session/list` | `bridge.listSessions` (`unstable_listSessions`) | connection stream |
198+
| `session/set_mode` | approval-mode route logic | session stream |
199+
| JSON-RPC **response** (to agent→client req) | resolve pending (`§4.3`) | — |
200+
| `_qwen/session/set_model` | `bridge.setSessionModel` (`unstable_setSessionModel`) | session stream |
201+
| `_qwen/workspace/list` etc. | workspace introspection routes | connection stream |
202+
| `_qwen/session/heartbeat` | `bridge.heartbeat` | connection stream |
203+
204+
### 4.2 Outbound (BridgeEvent → JSON-RPC on session stream)
205+
206+
| BridgeEvent.type | Emitted as |
207+
|------------------|-----------|
208+
| `session_update` | `{method:"session/update", params:<data>}` notification |
209+
| permission request | `{id:<n>, method:"session/request_permission", params}` request |
210+
| `client_evicted` / `slow_client_warning` / `state_resync_required` | `{method:"_qwen/notify", params:{kind,…}}` notification |
211+
| `stream_error` | JSON-RPC error response on the active prompt id (or `_qwen/notify`) |
212+
| prompt settle | `{id:<promptId>, result:{stopReason}}` |
213+
214+
### 4.3 Pending agent→client requests
215+
216+
`AcpConnection` keeps `Map<jsonRpcId, {sessionId, kind, bridgeRequestId, resolve}>`.
217+
When the client POSTs a JSON-RPC response object, `dispatch` matches `id`, then calls the
218+
bridge resolution path (e.g. permission `POST /session/:id/permission/:requestId`
219+
internal equivalent). `fs/*` requests: the daemon already satisfies file reads via its
220+
own workspace FS for the REST path; for the ACP surface we **forward** `fs/*` to the
221+
client (true ACP semantics) and only fall back to local FS when the client lacks the
222+
`fs` capability (declared in `initialize`).
223+
224+
---
225+
226+
## 5. Extension strategy (requirement #2)
227+
228+
ACP reserves any method starting with `_` for custom extensions and provides `_meta`
229+
on every type. The codebase's southbound leg already uses `unstable_*` method names.
230+
231+
**Northbound choice:** vendor-namespaced **`_qwen/<area>/<verb>`** method names
232+
(spec-compliant `_` prefix). Capabilities advertised under
233+
`agentCapabilities._meta.qwen` at `initialize` so clients feature-detect before use.
234+
235+
| Need | No standard ACP method? | Extension |
236+
|------|------------------------|-----------|
237+
| Model switch | yes | `_qwen/session/set_model` |
238+
| Workspace MCP/skills/providers/env introspection | yes | `_qwen/workspace/list`, `_qwen/workspace/<area>` |
239+
| Heartbeat / last-seen | yes | `_qwen/session/heartbeat` |
240+
| Multi-client permission policy (consensus/designated) | partial | `session/request_permission` + `_meta.qwen.policy` |
241+
| SSE backpressure tuning (`maxQueued`) | yes | `Acp-Qwen-Max-Queued` header on session GET |
242+
| Resume cursor (ring `Last-Event-ID`) | RFD Phase 4 | `Last-Event-ID` header + `_meta.qwen.eventId` on frames |
243+
244+
Standard methods are **never** renamed; extensions are strictly additive and ignorable.
245+
246+
---
247+
248+
## 6. Dual-transport vs. replace (requirement #4)
249+
250+
**Decision: dual-transport (additive).**
251+
252+
- The official transport is a **Draft** RFD, not normative, and absent from every SDK —
253+
hard-replacing would couple us to an unratified design and break webui + 3 SDKs +
254+
VSCode companion at once.
255+
- The REST surface carries features with no clean ACP mapping yet (workspace
256+
introspection, multi-client permission mediation, ring-buffer resume, capability
257+
registry). Those degrade to `_qwen/*` extensions on `/acp` but the REST surface stays
258+
authoritative until the RFD ratifies.
259+
- Both transports share **one** `HttpAcpBridge` + `EventBus` instance, so there is no
260+
state duplication — `/acp` and `/session/*` can even drive the same live session
261+
concurrently (multi-client is already supported by the bridge).
262+
- Toggle: on by default; `QWEN_SERVE_ACP_HTTP=0` (or `--no-acp-http`) disables mount.
263+
Advertised via a `acp_http` tag in `/capabilities`.
264+
265+
Migration path: once the RFD ratifies and SDKs ship, REST routes can be reframed as a
266+
thin compat shim over `/acp` (separate, later PR).
267+
268+
---
269+
270+
## 7. Scope of the implementation PR
271+
272+
**In scope (runnable + verified locally):**
273+
- `POST /acp` dispatch for `initialize`, `session/new`, `session/prompt`,
274+
`session/cancel`, `session/load`, JSON-RPC response handling.
275+
- Connection-scoped + session-scoped `GET /acp` SSE streams with JSON-RPC framing.
276+
- `session/update` streaming + final prompt response correlation.
277+
- `session/request_permission` agent→client round-trip.
278+
- `_qwen/session/set_model` extension as the worked example of #2.
279+
- Bearer-auth + host allowlist reuse (same middleware as REST).
280+
- Unit tests (`acpHttp/*.test.ts`) + a black-box smoke script driving a real daemon.
281+
282+
**Deferred (documented, not built now):**
283+
- WebSocket upgrade path (RFD-required client cap; SSE suffices for local verify).
284+
- HTTP/2 multiplexing (we run HTTP/1.1; POST and long-lived GET use separate sockets,
285+
which works for CLI/Node clients and ≤6-connection browsers). Documented divergence.
286+
- Full `fs/*` + `terminal/*` agent→client forwarding (permission path proves the
287+
mechanism; rest is mechanical follow-up).
288+
- SSE resumability hardening parity with the ring buffer (Phase 4 in RFD).
289+
290+
---
291+
292+
## 8. Local verification plan
293+
294+
1. `npm run build` (or workspace build of `cli` + `acp-bridge`).
295+
2. Start daemon: `qwen serve --listen 127.0.0.1:0 --token <t>` (or env token).
296+
3. Run `node scripts/acp-http-smoke.mjs`:
297+
- `POST /acp {initialize}` → assert `200` + `Acp-Connection-Id`.
298+
- Open connection SSE; `POST {session/new}` → assert response on stream.
299+
- Open session SSE; `POST {session/prompt:"say hi"}` → assert ≥1 `session/update`
300+
then a final `{result:{stopReason}}`.
301+
- Trigger a tool needing permission → assert `session/request_permission` request,
302+
POST a grant response → assert prompt completes.
303+
- `POST {_qwen/session/set_model}` → assert model switch + `session/update`.
304+
4. Vitest: `acpHttp/*.test.ts` green.
305+
306+
---
307+
308+
## 9. Risks
309+
310+
| Risk | Mitigation |
311+
|------|-----------|
312+
| RFD changes before ratification | Behind capability tag + `_qwen` namespace; isolated module; easy to revise. |
313+
| HTTP/1.1 vs required HTTP/2 | Localhost/CLI clients unaffected; documented; h2 is a transport swap later. |
314+
| Two transports on one bridge race | Bridge already supports multi-client; reuse its locking. |
315+
| `fs/*` forwarding vs daemon-local FS | Capability-gated: forward when client declares `fs`, else local. |

0 commit comments

Comments
 (0)