Skip to content

Commit 899b80b

Browse files
nnemirovskynnemirovsky
committed
docs: add macOS VM backend via tart implementation plan
1 parent a35cfe0 commit 899b80b

1 file changed

Lines changed: 239 additions & 0 deletions

File tree

‎docs/plans/20260405-macos-vm-backend.md‎

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
# macOS VM Backend via tart
2+
3+
## Overview
4+
5+
Add a macOS VM backend using `tart` CLI for running OpenClaw in a macOS guest VM with access to Apple frameworks (iMessage, EventKit, Keychain, Shortcuts). This is the third container backend alongside Docker and Apple Container (Linux). The macOS VM runs OpenClaw with full Apple ecosystem access while sluice governs all network traffic through pf rules + tun2proxy.
6+
7+
**Problem:** Apple Container (Plan 16) runs Linux containers. They cannot access Apple frameworks. For AI agents that need to interact with iMessage, Calendar, Reminders, FaceTime, or other Apple services, a macOS guest VM is required.
8+
9+
**Solution:** Add a `TartManager` implementing the existing `ContainerManager` interface, wrapping the `tart` CLI. Sluice can run anywhere (host, Docker, Apple Container). tun2proxy runs on the host. pf rules redirect macOS VM traffic through tun2proxy to sluice's SOCKS5 proxy.
10+
11+
**Architecture:**
12+
```
13+
Host (macOS):
14+
tun2proxy (native) -- creates TUN device, routes to SOCKS5
15+
pf rules -- redirect VM bridge traffic to TUN device
16+
17+
Docker / Apple Container / host:
18+
sluice -- SOCKS5 proxy + MCP gateway + API
19+
20+
macOS VM (tart):
21+
OpenClaw -- has iMessage, EventKit, Keychain, etc.
22+
Traffic: bridge100 -> pf -> tun2proxy -> SOCKS5 -> sluice -> internet
23+
```
24+
25+
**EULA note:** macOS EULA allows 2 additional macOS VM instances per Apple-branded host. Only OpenClaw runs in a macOS VM. sluice and tun2proxy are host processes or Linux containers and do not count.
26+
27+
## Context (from discovery)
28+
29+
**Existing infrastructure to reuse:**
30+
- `container.ContainerManager` interface (Plan 16) -- TartManager implements this
31+
- `container.NetworkRouter` for pf rules (Plan 16) -- same bridge routing
32+
- VirtioFS volume sharing for phantom tokens and CA certs
33+
- `container exec` pattern maps directly to `tart exec`
34+
35+
**Files/components involved:**
36+
- `internal/container/types.go` -- Runtime enum, ContainerManager interface
37+
- `internal/container/tart.go` -- new: TartManager
38+
- `internal/container/network.go` -- reuse pf routing (NetworkRouter, SetupNetworkRouting)
39+
- `cmd/sluice/main.go` -- runtime selection
40+
- `e2e/macos_vm_test.go` -- macOS VM e2e tests (separate from apple_test.go)
41+
42+
**Dependencies:**
43+
- `tart` CLI (Homebrew: `brew install cirruslabs/cli/tart`)
44+
- macOS with Apple Silicon (M1+)
45+
- tun2proxy on host
46+
47+
**tart CLI mapping:**
48+
```
49+
tart clone <image> <name> -- create VM from OCI image (can take minutes for macOS images)
50+
tart run <name> --dir=... --no-graphics -- start VM (BLOCKING: runs until VM shuts down, must use cmd.Start not cmd.Run)
51+
tart exec <name> -- <cmd> -- run command inside VM (requires tart agent in guest)
52+
tart stop <name> -- stop VM
53+
tart delete <name> -- remove VM
54+
tart list --format json -- list VMs as JSON
55+
tart ip <name> -- get VM IP address
56+
```
57+
58+
**tart agent requirement:** `tart exec` requires the tart helper agent running inside the guest VM. The `--vm-image` should be an OCI image with the tart agent pre-installed (e.g., images from cirruslabs).
59+
60+
**tun2proxy lifecycle:** Sluice does NOT manage tun2proxy. The user must start tun2proxy on the host before running `--runtime macos`. Create `scripts/macos-vm-setup.sh` that starts tun2proxy, applies pf rules, and enables IP forwarding. Sluice logs a warning if tun2proxy is not reachable.
61+
62+
## Development Approach
63+
64+
- **Testing approach**: Regular (code first, then tests)
65+
- Complete each task fully before moving to the next
66+
- All tests must pass before starting next task
67+
- Do NOT create new migration files. Edit 000001_init.up.sql if schema changes needed (they shouldn't be).
68+
- Reuse existing `ContainerManager` interface and `NetworkRouter` from Plan 16.
69+
70+
## Testing Strategy
71+
72+
- **Unit tests**: TartManager with mock exec (CommandRunner interface from Apple Container backend). All tests run on any platform.
73+
- **E2e tests**: `//go:build e2e && darwin` tag. Skip if `tart` not installed.
74+
75+
## Solution Overview
76+
77+
TartManager wraps the `tart` CLI the same way AppleManager wraps the `container` CLI. Both use the `CommandRunner` interface for testability. The `--runtime macos` flag selects TartManager. Network routing reuses the same pf + tun2proxy approach from Plan 16 since macOS VMs also use a bridge interface.
78+
79+
Key difference from Apple Container: macOS VMs need `security add-trusted-cert` for CA injection (Keychain-based trust) instead of `update-ca-certificates` (Linux cert bundle).
80+
81+
## Technical Details
82+
83+
### Runtime comparison
84+
85+
| Feature | Docker | Apple Container | macOS VM (tart) |
86+
|---------|--------|----------------|-----------------|
87+
| Guest OS | Linux | Linux | macOS |
88+
| Isolation | Namespaces | Hypervisor micro-VM | Hypervisor VM |
89+
| Boot time | ~1s | Sub-second | 2-4 seconds |
90+
| Memory | ~50MB | ~50MB | 1.5-2GB |
91+
| Apple frameworks | No | No | Yes |
92+
| CLI tool | docker | container | tart |
93+
| Network routing | tun2proxy container | pf + tun2proxy host | pf + tun2proxy host |
94+
| EULA limit | Unlimited | Unlimited | 2 macOS VMs |
95+
96+
### Volume sharing
97+
98+
```bash
99+
# tart VirtioFS
100+
tart run openclaw \
101+
--dir=phantoms:/path/to/phantoms \
102+
--dir=ca:/path/to/ca \
103+
--no-graphics
104+
```
105+
106+
### CA cert injection (macOS guest)
107+
108+
```bash
109+
tart exec openclaw -- security add-trusted-cert \
110+
-d -r trustRoot \
111+
-k /Library/Keychains/System.keychain \
112+
/Volumes/ca/sluice-ca.crt
113+
```
114+
115+
## Implementation Steps
116+
117+
### Task 1: Add RuntimeMacOS to enum and detect tart
118+
119+
**Files:**
120+
- Modify: `internal/container/types.go`
121+
- Modify: `cmd/sluice/main.go`
122+
- Modify: `cmd/sluice/main_test.go`
123+
124+
- [ ] Add `RuntimeMacOS Runtime = 3` to the Runtime enum with `String()` returning `"macos"` (value 2 is already RuntimeNone)
125+
- [ ] Add `"macos"` to `--runtime` flag accepted values in main.go
126+
- [ ] Update `detectRuntime`: add `tartAvailable bool` parameter. Auto-detection priority: `apple` > `docker` (unchanged). `macos` (tart) is explicit-only via `--runtime macos` because macOS VMs are heavyweight (2-4s boot, 1.5GB+ RAM) and auto-selecting them would be surprising. Update all `detectRuntime` call sites.
127+
- [ ] Add `--vm-image` flag for specifying the OCI image for tart (e.g., `ghcr.io/cirruslabs/macos-sequoia-base:latest`)
128+
- [ ] Write tests for runtime detection with tart available / not available
129+
- [ ] Run tests: `go test ./... -v -timeout 30s`
130+
131+
### Task 2: Implement TartManager CLI wrapper
132+
133+
**Files:**
134+
- Create: `internal/container/tart.go`
135+
- Create: `internal/container/tart_test.go`
136+
137+
- [ ] Implement `TartManager` struct using the existing `CommandRunner` interface for testability (same pattern as `AppleManager`)
138+
- [ ] Implement `tart clone` for creating VM from OCI image
139+
- [ ] Implement `tart run` with `--dir` flags for VirtioFS volumes and `--no-graphics` for headless. IMPORTANT: `tart run` is a BLOCKING command (unlike `container run`). Must use `cmd.Start()` (not `cmd.Run()`) and manage the process in a background goroutine. Add a `StartBackground(name string, args ...string) (*exec.Cmd, error)` method to CommandRunner or use a separate launch path.
140+
- [ ] Implement `tart exec` for running commands inside the VM (requires tart agent in guest image)
141+
- [ ] Implement `tart stop` and `tart delete` for lifecycle management
142+
- [ ] Implement `tart list --format json` and `tart ip` for status and IP retrieval (parse JSON output, not table)
143+
- [ ] Check if `tart` binary exists on creation (clear error if not installed)
144+
- [ ] Write tests with mock CommandRunner (capture commands, return canned output)
145+
- [ ] Write tests for error cases (binary not found, VM not running, exec failure)
146+
- [ ] Run tests: `go test ./internal/container/ -v -timeout 30s`
147+
148+
### Task 3: Implement ContainerManager interface
149+
150+
**Files:**
151+
- Modify: `internal/container/tart.go`
152+
- Modify: `internal/container/tart_test.go`
153+
154+
- [ ] Implement `ReloadSecrets`: write phantom token files to VirtioFS shared directory, run `tart exec <name> -- openclaw secrets reload`
155+
- [ ] Implement `RestartWithEnv`: stop VM, then re-run with new env (NOT delete+clone, which takes minutes for macOS images). tart VMs persist state across stop/run cycles.
156+
- [ ] Implement `InjectMCPConfig`: write mcp-servers.json to VirtioFS shared directory, run `tart exec <name> -- openclaw mcp reload`
157+
- [ ] Implement `Status`: run `tart list`, parse VM state and IP
158+
- [ ] Implement `Stop`: run `tart stop`
159+
- [ ] Implement `Runtime()`: return `RuntimeMacOS`
160+
- [ ] Write tests for each ContainerManager method with mock CommandRunner
161+
- [ ] Run tests: `go test ./internal/container/ -v -timeout 30s`
162+
163+
### Task 4: CA cert injection for macOS VM
164+
165+
**Files:**
166+
- Modify: `internal/container/tart.go`
167+
- Modify: `internal/container/tart_test.go`
168+
169+
- [ ] Add `InjectCACert(ctx context.Context, hostCertPath, guestCertDir string) error` to the `ContainerManager` interface. Implement no-op on `DockerManager` (Docker handles CA via compose volumes). Implement on `AppleManager` using `update-ca-certificates`. Implement on `TartManager` using `security add-trusted-cert`.
170+
- [ ] TartManager.InjectCACert: copy CA cert to VirtioFS shared volume, run `tart exec <name> -- security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /Volumes/ca/sluice-ca.crt`
171+
- [ ] Set `SSL_CERT_FILE`, `REQUESTS_CA_BUNDLE`, `NODE_EXTRA_CA_CERTS` env vars as fallback for tools that don't use Keychain
172+
- [ ] Call `containerMgr.InjectCACert()` after VM/container startup in main.go (works for all backends via interface)
173+
- [ ] Write tests for cert injection command generation
174+
- [ ] Run tests: `go test ./internal/container/ -v -timeout 30s`
175+
176+
### Task 5: Network routing for macOS VM
177+
178+
**Files:**
179+
- Modify: `internal/container/tart.go`
180+
- Modify: `internal/container/network.go` (refactor DefaultBridgeInterface to accept generic IP getter)
181+
- Create: `scripts/macos-vm-setup.sh`
182+
183+
- [ ] After VM starts, get its IP via `tart ip <name>`
184+
- [ ] Refactor `DefaultBridgeInterface` to accept a function `func() (string, error)` for getting VM IP instead of requiring `*AppleCLI`. This allows both AppleManager and TartManager to use the same routing code.
185+
- [ ] Call `NetworkRouter.SetupNetworkRouting(ctx, vmIP, bridgeIface, tunGateway)` with correct signature (4 params, not 2)
186+
- [ ] On shutdown, call `TeardownNetworkRouting()` to clean up pf rules
187+
- [ ] Create `scripts/macos-vm-setup.sh` that starts tun2proxy on the host, enables IP forwarding, and documents the required setup
188+
- [ ] Log warning if tun2proxy is not running on host (check if TUN device exists)
189+
- [ ] Write tests for routing setup with mock VM IP
190+
- [ ] Run tests: `go test ./internal/container/ -v -timeout 30s`
191+
192+
### Task 6: Wire TartManager into main.go startup
193+
194+
**Files:**
195+
- Modify: `cmd/sluice/main.go`
196+
- Modify: `cmd/sluice/main_test.go`
197+
198+
- [ ] When `--runtime macos`: create `TartManager`. Startup sequence: (1) `tart list --format json` to check if VM exists, (2) if not, `tart clone <--vm-image> <--container-name>` (warn user this may take minutes for macOS images), (3) `tart run` in background goroutine, (4) wait for VM IP via `tart ip`, (5) set up pf routing, (6) inject CA cert via `InjectCACert`.
199+
- [ ] Pass `TartManager` as `ContainerManager` to Telegram commands, API server, MCP auto-injection
200+
- [ ] On shutdown: stop VM, tear down pf rules
201+
- [ ] Write tests for macos runtime startup path (mock tart CLI)
202+
- [ ] Run tests: `go test ./cmd/sluice/ -v -timeout 30s`
203+
204+
### Task 7: Verify acceptance criteria
205+
206+
- [ ] Verify `--runtime macos` starts a macOS VM via tart
207+
- [ ] Verify VM gets its own IP and pf rules are applied
208+
- [ ] Verify traffic from macOS VM routes through tun2proxy -> sluice SOCKS5
209+
- [ ] Verify CA cert is trusted by the macOS VM (Keychain-based)
210+
- [ ] Verify credential hot-reload works (phantom files + tart exec)
211+
- [ ] Verify MCP auto-injection works (mcp-servers.json + tart exec)
212+
- [ ] Verify Docker and Apple Container backends still work (no regression)
213+
- [ ] Verify `--runtime macos` requires explicit flag (not auto-detected)
214+
- [ ] Verify Docker and Apple Container regressions: `go test ./internal/container/ -v -timeout 30s`
215+
- [ ] Run full test suite: `go test ./... -v -timeout 60s`
216+
- [ ] Run linter: `go vet ./...`
217+
218+
### Task 8: [Final] Update documentation
219+
220+
- [ ] Update CLAUDE.md: document macOS VM backend, tart CLI, --runtime macos
221+
- [ ] Update CLAUDE.md: correct Apple framework access claims (macOS VM only, not Apple Container)
222+
- [ ] Update CLAUDE.md: add RuntimeMacOS to runtime comparison table
223+
- [ ] Update CONTRIBUTING.md: note tart requirement for macOS VM testing
224+
- [ ] Move this plan to `docs/plans/completed/`
225+
226+
## Post-Completion
227+
228+
**Manual verification (requires macOS with tart installed):**
229+
- Clone a macOS base image and run OpenClaw in it
230+
- Verify Apple framework access (open Messages.app, create a Reminder)
231+
- Verify sluice governs all network traffic from the VM
232+
- Verify credential injection and MCP auto-injection work
233+
- Test pf rule cleanup on sluice shutdown
234+
235+
**Future considerations:**
236+
- Snapshot and restore macOS VMs for fast startup
237+
- Pre-built OpenClaw macOS OCI images in a registry
238+
- Multiple macOS VMs (agent pool) within EULA limits
239+
- Host-to-VM communication via virtio-vsock for low-latency MCP

0 commit comments

Comments
 (0)