Move agent execution into a node-host utilityProcess with direct MessagePort streaming#3365
Draft
adamleithp wants to merge 6 commits into
Draft
Move agent execution into a node-host utilityProcess with direct MessagePort streaming#3365adamleithp wants to merge 6 commits into
adamleithp wants to merge 6 commits into
Conversation
Runtime-neutral tRPC transport over MessagePorts, protocol-compatible with @posthog/electron-trpc's wire envelope. The link side is written against the DOM MessagePort interface (Electron renderer today, Web Worker on web later); the server side runs against Electron MessagePortMain or worker_threads ports via structural adapters, so the package imports neither electron nor node. PortBridge handles late-arriving and replaced ports (utility-process restarts): operations queue until the first port connects, and a replacement or close fails all in-flight operations so callers' existing reconnect paths run. Groundwork for hosting the agent loop in an Electron utilityProcess with direct renderer-to-utility streaming. Generated-By: PostHog Code Task-Id: 30be96e9-d9a7-4b5a-8dfb-87be8ca4f4f6
… ports AgentService injected four wide main-process services (PosthogPluginService, IWorkspaceRepository, IWorkspaceSettings, FoldersService) plus IPowerManager, but only ever calls one method on each. Replace them with narrow async interfaces in the existing agent ports seam (AgentPluginDir, AgentWorkspaceDirectories, AgentWorktreeSettings, AgentKnownFolders, AgentPowerMonitor), bound in main to the same concrete services. No behavior change. This makes every AgentService dependency either portable or a small async port, so the service can next move into a node-host utilityProcess with these proxied back to main — without dragging sqlite or electron-store into that process. Generated-By: PostHog Code Task-Id: 30be96e9-d9a7-4b5a-8dfb-87be8ca4f4f6
AgentService (the agent loop, ACP stream parsing, gateway/MCP HTTP proxies,
agent process tracking) ran in the Electron main process, competing with
window management and every IPC round-trip. It now runs in a dedicated
utilityProcess ("node host"), the modern replacement for the
ELECTRON_RUN_AS_NODE child pattern:
- packages/node-host: the utility runtime — DI container binding agentModule +
auth/mcp proxies + its own process tracking, env-derived platform adapters,
and proxies for main-resident capabilities. Serves the agent routers over
MessagePorts via port-trpc; entry handshakes on process.parentPort
(init/ready/ping/shutdown) with ports typed structurally, so the package
never imports electron.
- host-capabilities router (served by main over a reverse port): the narrow
surface the moved AgentService still needs — sleep blocking, auth tokens,
MCP apps, the repo-fs bridge, plugin dir, sqlite-backed workspace lookups,
settings, power resume. Token calls only; proxy bodies stream locally in the
utility, and sqlite/electron-store never enter its bundle (verified: the
emitted node-host.js has no native-module or electron requires).
- NodeHostService supervisor in main, modeled on WorkspaceServerService:
spawn/handshake/heartbeat, exponential-backoff restart, stdio piped to the
app log. Its control client is one PortBridge-backed tRPC client that
survives restarts by swapping ports.
- AgentBridge mirrors agent events for main-side consumers (usage monitor,
workspace branch watcher, archive/suspension cancellation, git session env,
dev-toolbar snapshot), resubscribing on every Ready; hasActiveSessions stays
sync via cached idle/activity transitions.
- Main's agent.* routes become one-line forwards over the control channel, so
the renderer keeps working unchanged (it gets its own direct port next).
Generated-By: PostHog Code
Task-Id: 30be96e9-d9a7-4b5a-8dfb-87be8ca4f4f6
The renderer's agent.* operations now flow over a MessagePort wired straight to the node-host utilityProcess, so agent event streams bypass the main process entirely (the VS Code pattern). Everything else stays on the electron-trpc IPC link; main also keeps serving agent.* as a forwarding fallback so a mis-route fails loudly. Ports can't cross the contextBridge, so the preload relays them into the main world with window.postMessage; the renderer bridge registers its listener before requesting a port, queues operations until one arrives, and swaps ports (failing in-flight work into SessionService's existing auto-recovery) when the supervisor re-issues one with a bumped generation after a utility restart. Non-Electron hosts (web, storybook) see no preload and are unaffected — the same bridge later accepts a Web Worker's port unchanged. Generated-By: PostHog Code Task-Id: 30be96e9-d9a7-4b5a-8dfb-87be8ca4f4f6
A small nodeHost router (getStatus / restart / onStatusChanged), mirroring the workspaceServer one, so the renderer can observe supervisor state and offer a retry when the utility lands in the failed state. Architecture docs gain the desktop process-topology diagram; the supervisor joins the host-boundary allowlist alongside WorkspaceServerService. Generated-By: PostHog Code Task-Id: 30be96e9-d9a7-4b5a-8dfb-87be8ca4f4f6
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
Adding the node-host entry changed the three-entry rollup graph: the main chunk now imports the workspace-server.js and node-host.js ENTRY chunks to reach modules they share, executing their top-level bootstrap inside the Electron main process. workspace-server's env fail-fast then exit(2)'d the whole app at launch — every CI e2e smoke test died with "[workspace-server] missing or invalid WORKSPACE_SERVER_SECRET / WORKSPACE_SERVER_PORT". Both serve entries now gate their bootstrap on actually being the spawned child (argv entry-script basename for the workspace-server child, parentPort presence for the utility), the moral equivalent of require.main === module: a bare chunk import is a no-op, while a genuine mis-spawn still fails fast with the same loud error. Verified all four modes against the built output: direct-exec → loud exit 2, chunk-import → silent success. Also formats scripts/host-boundary-allowlist.json the way Biome wants it (the quality check's one error). Generated-By: PostHog Code Task-Id: 30be96e9-d9a7-4b5a-8dfb-87be8ca4f4f6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Agent execution — the agent loop, ACP stream parsing, the gateway/MCP HTTP proxies — runs in the Electron main process, where it competes with window management and every IPC round-trip the app makes. Borrowing VS Code's process-architecture lesson: UI-critical processes should never host CPU-ish or streaming work. Electron's
utilityProcess+ MessagePorts is the modern tool for exactly this split, and the seam has to stay clean enough that the same core logic can later run on web (Web Worker or remote server).Why: keep agent/orchestration work off the UI-critical processes so streaming sessions can't jank the app — this is the structural decision the rest of the perf work tunes around.
Changes
@posthog/port-trpc(new): tRPC-over-MessagePort transport, wire-compatible withelectron-trpc. The link side is written against the DOMMessagePortinterface with zero electron/node imports (Biome-enforced), so it works unchanged against a Web Worker on web later; the server side runs onMessagePortMainvia structural adapters.PortBridgequeues ops until a port arrives and swaps ports across restarts, failing in-flight work into callers' existing recovery paths.@posthog/node-host(new): the utilityProcess runtime.AgentService, the auth/MCP proxies, and agent process tracking move here; main-resident capabilities (power blocking, MCP apps, auth tokens, repo-fs bridge, sqlite-backed lookups, settings) are consumed over a reverse host-capabilities port as narrow async interfaces — sqlite and electron-store never enter the utility bundle (verified against the emittednode-host.js).agent.*traffic (token streams included) bypasses main entirely. Everything else stays on the electron-trpc IPC link, and main keeps servingagent.*as a forwarding fallback.NodeHostServicein main (modeled onWorkspaceServerService) handles handshake, heartbeat, backoff restart, stdio piping, and port re-issuance with generation counters; anAgentBridgemirrors agent events for main-side consumers.SessionService, all ofcore/ui, andapps/webare untouched.Follow-up phases (not in this PR): fold the workspace-server child's services (git/fs/watchers) into the node host and retire the HTTP/SSE loopback; move enrichment/tree-sitter and session-log tail parsing behind node-host routes; serve
HostRouter["agent"]from a Web Worker on web via the same link.How did you test this?
port-trpctransport suite (10 tests over realMessageChannels: query/mutation/subscription lifecycle, stop/cancel aborts, close-aborts-server, port replacement, stale-generation rejection); existing suites pass (apps/code286, workspace-server agent/proxy/process-tracking 126,electron-trpc16).pnpm typecheckacross the monorepo (25/25), including the compile-time check that the node host serves everyHostRouter["agent"]route.electron-vite buildemits the thirdnode-host.jsentry; grepped it to confirm nobetter-sqlite3/node-pty/@parcel/watcher/electron-store/electronrequires.check-host-boundariesclean.posthog-node-hostprocess mid-stream and watch the supervisor restart + session auto-recovery. Suggested reviewer checklist before merging.Automatic notifications
Created with PostHog Code