feat(remote): web remote access — drive the desktop app from any browser on your LAN or tailnet#163
Merged
Merged
Conversation
Embed a default-off HTTP+WebSocket server in the desktop process so a browser on another device (laptop, phone on the LAN or the user's mesh VPN) loads the same UI bundle and drives the same running instance — same projects, terminals, agent chats. A UI transport (second frontend for one backend) that composes with the remote-hosts compute transport: a paired web client transparently sees and drives the desktop's hosts. - Embedded auth'd axum HTTP+WS server (src-tauri/src/web_remote/): binds 0.0.0.0:4377 only when the user enables it; AssetResolver static bundle, /api/* control surface, /ws transport, and endpoint enumeration (loopback/LAN/tailnet/MagicDNS) with per-endpoint secure-context notes. Config persists and restores on boot. - WebSocket runtime shim (src/remote/): production twin of the dev mock — installs window.__TAURI_INTERNALS__ backed by a real socket so every invoke/listen/Channel/convertFileSrc routes over the wire with zero app-component changes. Invoke dispatch reuses the main webview's on_message; a Builder-registered channel interceptor re-routes each command's Channel frames (incl. binary PTY frames) to the owning browser; a listen_any hub multiplexes event subscribers. - Auth: one-time pairing token -> persistent revocable SQLite session (SHA-256 at rest, constant-time compare, no early-return), 30s single-use WS tickets, optional approval mode, HttpOnly+SameSite cookie and bearer through one authenticate path, origin checks on every state-touching route, per-IP pairing rate limit. Revocation severs live sockets immediately. - Remote Access settings pane: master toggle + exposure warning, port field, copy-ready endpoint list, QR/link pairing generator, paired devices list with revoke/revoke-all, approval-mode toggle + pending approval flow. - Multi-client mirror mode: terminal + agent-chat streams fan out to N subscribers (per-generation attach/detach, replay only to the new attacher, pause only when all subscribers pause); the desktop window stays the sole scrollback-serialization owner. - Web fallbacks (gated on isRemoteClient(), desktop byte-identical): in-app path browser replacing the native file dialog (unlocks opening projects / creating workspaces from the browser), Web Notifications with toast fallback, /api/assets file streamer for convertFileSrc, an authed browser-pane proxy to the loopback agent-browser daemons, hidden window chrome, and update-defer while remote + a web-triggered desktop update flow. - Docs: new docs/features/web-remote-access.md; INDEX/STATUS/plan updates. Debug-only e2e affordances stay #[cfg(debug_assertions)] + env-gated.
…ign update-flow doc wording
- relativeTime treated SQLite's timezone-less datetime('now') strings as
local time, skewing 'paired/last seen' by the machine's UTC offset;
normalize to ISO-8601 UTC before parsing, with regression tests.
- Feature doc described the update flow as deferring an auto-restart;
restarts are always explicit user actions — reword to match behavior.
…socket connects
Add GET /api/snapshot: an authed, versioned {api_version, app_state, status}
envelope built from the same AppStateStore::snapshot() the get_app_state command
returns (no duplicated serialization), Cache-Control: no-store. The web shim
prefetches it concurrently with the WS ticket + upgrade and answers the app's
first get_app_state from it, so the UI paints real state without a post-mount
socket round-trip. WS stays authoritative: the seed is applied only while a
monotonic wsAppStateSeq (bumped on each app-state-changed event) is still 0, so
a WS snapshot always wins and no stale seed can overwrite newer state; the
prefetch is never awaited on the hot path and any failure falls back to the WS
path silently. On reconnect the shim refetches and re-applies via the
app-state-changed path (discarded if a fresher WS event lands first) to catch up
on state missed during the outage. Desktop/Tauri and dev-mock paths are
unchanged (the fetchSnapshot shim option is supplied only by the web bootstrap).
Shaped as the first piece of a versioned API for future native (non-web) clients.
Also fixes two defects found in live end-to-end verification:
- Terminal input could scramble over the web transport: write_to_pty invokes
were each dispatched on their own task, letting rapid keystrokes race to the
PTY. They now drain through a per-connection serial lane (is_ordered_invoke)
in client-send order; every other invoke keeps the concurrent path.
- Disabling remote access left live sockets open: graceful shutdown only stops
new connections, so a connected device kept control until it reloaded.
stop_server now calls ConnectionRegistry::close_all() to sever every socket
immediately (the same mechanism revocation uses); port changes rebind the
same way.
Tests: Rust snapshot route (api_version, envelope shape, 401/403/200 admission),
ordered-lane + close_all; TS snapshot seeding (fetch validation/failure, seed
with no wire frame, discard-when-superseded, reconnect re-seed + stale-race).
# Conflicts: # docs/core/STATUS.md # src/components/layout/title-bar.tsx # src/dev/tauri-mock.ts
Widen the injected cancelIdle param in scrollback-idle-serializer.test.ts from `number` to the source's exported `IdleHandle` type. In a Node type environment `ReturnType<typeof setTimeout>` widens `IdleHandle` beyond `number`, so a `number`-only scheduler mock is not assignable to `(handle: IdleHandle) => void` and `tsc` fails `npm run check` / `tsc -b`. Pre-existing on main (reproduces on a clean origin/main checkout); surfaced by the matrix after the merge. Type-only change, no runtime effect.
The web-remote connection indicator was a floating pill anchored bottom-left of the viewport. It overlapped the sidebar footer, the setup-scripts hint card, and the notification bell. Replace it with a store-driven pair of surfaces, split by loudness: - Connected → a compact, non-overlapping chip in the title bar's right cluster (the slot the hidden native window controls free up on the web client). Small sky dot + "Remote", host in the tooltip and inline at wide widths; flex-shrinks/truncates so it can never collide at narrow widths. - Reconnecting / offline → a loud, centered banner overlay so a dropped or revoked session stays impossible to miss. Desktop never runs the bootstrap, so the store stays null and both surfaces render nothing — chrome is byte-identical to before.
…esolve the agent-chat sidecar in dev Endpoints: - Skip Docker/veth/virbr/CNI/VirtualBox/VMware/ZeroTier/tailscale0 interfaces by name (is_virtual_iface) so 172.x/link-local noise never masquerades as a reachable URL. The `br-` prefix is hyphenated so a real bridge `br0` survives. - Give each endpoint a coarse `group` (this_device/local_network/tailscale/ other) and a single `recommended` hint; tailnet IPv4+IPv6 and MagicDNS come from the CLI and land under `tailscale`. mark_recommended picks the best from-anywhere option (MagicDNS, else tailnet, else local-network). - Render the "Reachable at" list and the pairing device-picker as labelled groups (empty groups dropped, `other` behind a default-closed disclosure) with a Recommended chip; groupEndpoints degrades unknown groups into `other` and pickPrimaryEndpoint now prefers the recommended endpoint. Sidecar: - resolve_sidecar() adds a debug-only fallback to the dev-tree src-tauri/binaries/ copy so `npm run tauri:dev` resolves the Claude agent-chat sidecar even when the resource dir doesn't carry it; the fallback is compiled out of release via debug_assertions, so release resolution stays override -> resource_dir -> None. lib.rs pins the winner into CODEMUX_CLAUDE_SIDECAR_PATH. Docs: refresh the web-remote-access endpoint-enumeration section (grouping + virtual-interface filtering + recommended) and the agent-chat sidecar resolution note (debug-only dev fallback).
…cope - `codemux remote pair [--name <label>]` mints a one-time pairing code over the same-machine control socket and prints a scannable terminal QR plus link, so a phone or laptop can be paired over SSH without the desktop GUI. Errors clearly when remote access is disabled, reuses the GUI's PairingStore mint path, and carries an optional fallback device name that the /api/pair handler uses only when the client sends none. - Add a `bind_scope` config (all / tailscale / loopback) with a "Who can connect" Settings control. `tailscale` binds every tailnet address plus loopback and fails loudly rather than silently falling back to 0.0.0.0 when no tailnet address exists; `loopback` binds 127.0.0.1 only. Enable and set_config roll back to the last-good state on bind failure so the server is never "enabled but not running", and a scope or port change rebinds in place.
…s scope from a web client Changing the bind scope or port from the web client rebinds the server, which drops every socket before web_remote_set_config can answer. The invoke then rejected with "connection lost" and the handler treated that as a failure — error toast, control snapped back — even though the backend had applied the change (confirmed via ss; correct on reload). Desktop (native IPC) never drops its own transport, so it was unaffected. On a web client (isRemoteClient()), a port/scope change now: - Reflects the requested value optimistically and shows an "Applying — reconnecting" state instead of an error. isRebindDisconnectError() classifies the transport's connection-lost/not-connected rejection as the expected rebind drop, not a backend failure; a genuine backend rejection (e.g. Tailscale scope with no tailnet) still surfaces and reverts. - Settles on reconnect: once the shim's reconnect loop re-establishes the socket, it refetches web_remote_status and reconciles the control to the server's persisted value, silently (no toast) when they match. - Predicts a cutoff before applying a scope change and asks first. The origin the browser loaded from is classified (loopback / tailnet / lan) and checked against what the target scope still serves — loopback survives every scope, a tailnet/MagicDNS origin survives all+tailscale, a LAN origin survives only all. If the change would sever this device, a confirm dialog explains it; on confirm (or when a reconnect never succeeds) a clear terminal state replaces the endless reconnect spinner. Desktop behavior is byte-identical. Adds unit tests for the cutoff prediction (origin host + scope -> survives?) and the rebind-disconnect classification, plus component tests that a remote-initiated rebind disconnect shows no error toast while a real backend error still does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Turn on Remote Access in Settings and Codemux's desktop process starts an embedded server. Pair a browser on any other machine on your LAN or tailnet — scan a QR from your phone or open a one-time link — and the full app loads in that browser: the same projects, terminals, and agent chats, driving the same running desktop instance. Work keeps going when the browser disconnects (the PTY daemon already guarantees that), and reconnecting picks up exactly where it left off.
Architecture
src-tauri/src/web_remote/). It binds only when you enable it; disabling severs live sockets and unbinds the port. The frontend served is the app's own bundle, resolved through Tauri'sAssetResolver(no separate build).src/remote/) that installs__TAURI_INTERNALS__over the same internals seam the dev mock uses, so app components can't tell they're running in a browser vs. the desktop WebView. Invokes, events, and channels all ride oneGET /ws?ticket=…socket.AppHandle, so behavior is byte-identical to in-process IPC. Terminal input (write_to_pty) drains through a per-connection serial lane so rapid keystrokes can't race to the PTY.listen_anyhub — one desktop-side listener per event no matter how many browsers watch.GET /api/snapshot): a versioned{api_version, app_state, status}envelope built from the sameAppStateStore::snapshot()theget_app_statecommand returns. The shim prefetches it in parallel with the WS handshake and seeds the app's first render without a post-mount round-trip; the WS stays authoritative (a stale seed can never overwrite newer state), and it re-syncs on every reconnect.Security
Screenshots
Pairing screen — no token yet: the browser lands here; nothing is reachable until you pair.
Paired, full UI — the whole app loads in the browser with the Remote indicator, same projects and sessions as the desktop.
Web path browser — opening a project from the browser uses a transport-served path picker (no native file dialog).
Live terminal — real PTY output streams to the browser, including keystrokes typed in the web client.
Terminal replay — after a reload the PTY scrollback replays, so the terminal comes back exactly where it was.
QR pairing — create a pairing link + QR from Settings → Remote Access to pair a phone by scanning.
Approval flow — with approval mode on, a new device stays pending until you approve it in the desktop UI.
Devices panel — connected browsers are listed with platform info, and every session is revocable.
Toggle off — flipping the master switch off severs every live connection immediately and unbinds the port; the desktop keeps running.
Agent chat over web remote
The beta agent-chat GUI — the primary surface many users drive Codemux from — works end-to-end over a paired browser: enabling the beta from the web, running a live streamed turn, coexisting with a terminal, and rehydrating history after a reload. (Endpoint addresses in the pairing/devices shots above are boxed out; only loopback is shown.)
Enable Agent Chat from the web — the Beta toggle flipped on from the web client's own Settings, over the remote transport.
Live chat turn — a real streamed assistant reply in the chat GUI; the composer and the "Remote — connected" indicator are both visible.
Chat and terminal together — a live terminal and the agent-chat GUI both alive as sibling tabs in one paired web session.
History restored after reload — reloading the browser rehydrates the full chat transcript from the desktop backend (late-join / history hydration over web remote).
Cleaner endpoint list & agent chat
Follow-up polish on top of the above: the "Reachable at" list and the pairing card now curate what they show, and the run-from-source dev build resolves the agent-chat sidecar. (Endpoint addresses are redacted and the QR is blanked in these shots.)
Grouped endpoints — "Reachable at" buckets addresses into This device / Local network / Tailscale with a one-line explainer each; Docker bridges and other virtual interfaces are filtered out, and the best "from anywhere" endpoint carries a Recommended marker.
Curated pairing card — the QR/link device picker uses the same curated grouping (no Docker noise), with the recommended endpoint pre-selected.
Agent chat in run-from-source dev —
npm run tauri:devnow resolves the Claude sidecar, so a real streamed turn works over the paired web client.SSH pairing & access scope
Two follow-ups that make the server practical to run headless and to lock down: mint a pairing code straight from a terminal (handy over SSH — no desktop GUI needed), and choose which networks the server binds.
Pair from the terminal —
codemux remote pairmints a one-time pairing code over the same-machine control socket and prints a scannable QR + link, so you can pair a phone or laptop without touching the desktop GUI. Errors clearly when remote access is off;--namerides along as a fallback device label. (Real MagicDNS host and token redacted; the QR is regenerated to encode a loopback placeholder.)Access scope — a "Who can connect" control chooses which interfaces the server binds: All networks (
0.0.0.0), Tailscale only (tailnet addresses + loopback), or This device only (127.0.0.1). Changing it rebinds immediately and drops existing connections. Verified live at the socket level:allbinds0.0.0.0(LAN reachable),tailscalebinds the tailnet v4/v6 addresses plus loopback while dropping the LAN interface, andloopbackbinds127.0.0.1only.Testing
cargo test,src-taurilib): 1879 passed, 0 failed, 5 ignored. Theweb_remotesuite covers the pairing lifecycle, ticket single-use/binding/expiry, the rate limiter, the session-admission gate (bearer + cookie, missing/unknown/pending/cross-origin), the connection registry (targetedclose_sessionandclose_all), the channel router and event fan-out, endpoint enumeration, the asset route, the proxy guards, the new/api/snapshotroute (api_version, envelope shape built from the realAppStateStore::snapshot, and 401/403/200 admission), and the terminal ordered-input lane (write_to_ptyrouting + client-send-order preservation).npm run test): 2158 passed / 151 files. Includes shim contract tests against a fake WS (invoke round-trip, channel ordering, listen refcounting), transport reconnect/ticket flow, and the new snapshot seeding (fetch validation/failure →null, firstget_app_stateserved from the seed with no wire frame, seed discarded once a newer WS snapshot arrives, clean fall-back on failure, reconnect re-seed + stale-race discard).npm run check(tsc) clean;npm run buildsucceeds.npm run tauri:dev) with the server enabled, driven through the browser as a user — pair via link + QR, paired full UI, open an existing project and create a workspace via the web path-browser, live terminal (replay + live bytes + input ordering), an agent-chat turn, approval mode + approve + revoke, the devices panel, and flipping the master switch off (live connections severed, port unbound, desktop still alive).GET /api/snapshotwas confirmed fetched in parallel with the WS handshake (both HTTP 200 in the same millisecond) with the full authed envelope, and the auth matrix (401 unauth / 403 cross-origin / 401 bad bearer) held live.Honest caveat: the desktop webview is WebKitGTK with no CDP, so it can't be script-driven; the first server enable used the documented debug autostart bootstrap, but every management action (pairing, QR, approval, approve, revoke, master toggle) was then performed through the real web-client Settings UI. On loopback the seed-vs-WS race for the very first
get_app_stateisn't externally observable (the seed's whole point is to emit no wire frame), so that path is covered by the TS unit tests; the reconnect reseed was observed firing live (a second/api/snapshotfetch after a forced socket close).Deferred (not blocking v1)
/api/snapshotendpoint is deliberately shaped as its first API: a versioned envelope a client with no WebView and no@tauri-apps/apishim can bootstrap its whole view from, withapi_versionto detect breaking changes.