|
| 1 | +# Agent Farm Setup Guide |
| 2 | + |
| 3 | +Run multiple AI agents in parallel on shared infrastructure. Each agent gets an isolated workspace with its own containers, filesystem, and resource limits — managed automatically via the devbox MCP server. |
| 4 | + |
| 5 | +## How It Works |
| 6 | + |
| 7 | +``` |
| 8 | +Agent 1 ──stdio──▶ devbox mcp serve ──▶ Workspace (dev1) |
| 9 | +Agent 2 ──stdio──▶ devbox mcp serve ──▶ Workspace (dev1) |
| 10 | +Agent 3 ──stdio──▶ devbox mcp serve ──▶ Workspace (dev2) |
| 11 | +Agent N ──stdio──▶ devbox mcp serve ──▶ Workspace (devN) |
| 12 | +``` |
| 13 | + |
| 14 | +Each agent connects to its own `devbox mcp serve` process over stdio. On registration, devbox: |
| 15 | + |
| 16 | +1. Auto-selects the least-loaded server from the pool |
| 17 | +2. Creates an isolated workspace with resource limits |
| 18 | +3. Returns the agent ID and workspace name |
| 19 | +4. Enforces workspace isolation — agents cannot access each other's workspaces |
| 20 | +5. Auto-cleans the workspace when the agent disconnects |
| 21 | + |
| 22 | +## Prerequisites |
| 23 | + |
| 24 | +- devbox installed and at least one server added to the pool |
| 25 | +- Servers accessible via SSH with Docker installed |
| 26 | +- Tailscale configured on all servers |
| 27 | + |
| 28 | +## Step 1: Configure Server Pool |
| 29 | + |
| 30 | +Add servers to the pool. devbox automatically distributes agents across them: |
| 31 | + |
| 32 | +```bash |
| 33 | +devbox server add dev1 100.64.0.1 |
| 34 | +devbox server add dev2 100.64.0.2 |
| 35 | +devbox server add dev3 100.64.0.3 |
| 36 | +``` |
| 37 | + |
| 38 | +Verify all servers are healthy: |
| 39 | + |
| 40 | +```bash |
| 41 | +devbox doctor --server dev1 |
| 42 | +devbox doctor --server dev2 |
| 43 | +devbox doctor --server dev3 |
| 44 | +``` |
| 45 | + |
| 46 | +## Step 2: Plan Resources |
| 47 | + |
| 48 | +Each agent workspace consumes CPU and memory. Plan capacity based on your workload: |
| 49 | + |
| 50 | +| Server Spec | Agents @ 1 CPU, 1GB | Agents @ 2 CPU, 2GB | Agents @ 4 CPU, 4GB | |
| 51 | +|-------------|---------------------|---------------------|---------------------| |
| 52 | +| 4 CPU, 16GB | 4 | 2 | 1 | |
| 53 | +| 8 CPU, 32GB | 8 | 4 | 2 | |
| 54 | +| 16 CPU, 64GB | 16 | 8 | 4 | |
| 55 | +| 32 CPU, 128GB | 32 | 16 | 8 | |
| 56 | + |
| 57 | +Leave 10-20% headroom for the host OS and Docker overhead. |
| 58 | + |
| 59 | +## Step 3: Register Agents |
| 60 | + |
| 61 | +Each agent connects via its own MCP session and registers: |
| 62 | + |
| 63 | +``` |
| 64 | +→ devbox_agent_register |
| 65 | + { |
| 66 | + "name": "coder", |
| 67 | + "capabilities": ["code", "test"], |
| 68 | + "cpus": 1.0, |
| 69 | + "memory": "2g" |
| 70 | + } |
| 71 | +
|
| 72 | +← { |
| 73 | + "agent_id": "agent-coder-a1b2c3d4", |
| 74 | + "workspace": "agent-coder-a1b2c3d4", |
| 75 | + "server": "dev1" |
| 76 | + } |
| 77 | +``` |
| 78 | + |
| 79 | +The agent now has a running workspace on `dev1` with 1 CPU and 2GB memory. |
| 80 | + |
| 81 | +## Step 4: Agent Workflow |
| 82 | + |
| 83 | +A typical agent session: |
| 84 | + |
| 85 | +``` |
| 86 | +1. devbox_agent_register → get workspace |
| 87 | +2. devbox_workspace_exec → clone repo, install deps |
| 88 | +3. devbox_workspace_exec → run tasks (code, test, build) |
| 89 | +4. devbox_snapshot_create → checkpoint before risky ops (optional) |
| 90 | +5. devbox_workspace_exec → more work |
| 91 | +6. disconnect → workspace auto-destroyed |
| 92 | +``` |
| 93 | + |
| 94 | +### Execute Commands |
| 95 | + |
| 96 | +``` |
| 97 | +→ devbox_workspace_exec |
| 98 | + { |
| 99 | + "name": "agent-coder-a1b2c3d4", |
| 100 | + "command": "cd /workspaces && git clone https://github.com/org/repo.git && cd repo && go test ./..." |
| 101 | + } |
| 102 | +
|
| 103 | +← { |
| 104 | + "stdout": "ok\tall tests passed\n", |
| 105 | + "stderr": "", |
| 106 | + "exit_code": 0 |
| 107 | + } |
| 108 | +``` |
| 109 | + |
| 110 | +### Check Resources |
| 111 | + |
| 112 | +``` |
| 113 | +→ devbox_metrics |
| 114 | + { "workspace": "agent-coder-a1b2c3d4" } |
| 115 | +
|
| 116 | +← { |
| 117 | + "cpu_percent": 45.2, |
| 118 | + "mem_usage": 1073741824, |
| 119 | + "mem_limit": 2147483648, |
| 120 | + "disk_usage": 5368709120 |
| 121 | + } |
| 122 | +``` |
| 123 | + |
| 124 | +## Workspace Isolation |
| 125 | + |
| 126 | +Agents can only interact with their own workspace: |
| 127 | + |
| 128 | +- `devbox_workspace_exec` with another agent's workspace returns `FORBIDDEN` |
| 129 | +- `devbox_workspace_destroy` with another agent's workspace returns `FORBIDDEN` |
| 130 | +- `devbox_workspace_list` returns all workspaces (read-only, no isolation needed) |
| 131 | +- `devbox_workspace_create` is blocked for registered agents — use `devbox_agent_register` instead |
| 132 | + |
| 133 | +Non-agent MCP sessions (no `devbox_agent_register` call) bypass isolation and have full access — this is the single-user mode for direct integration. |
| 134 | + |
| 135 | +## Crash Recovery |
| 136 | + |
| 137 | +If an agent process crashes (SIGKILL, power loss), the workspace may be orphaned. devbox handles this automatically: |
| 138 | + |
| 139 | +- **PID tracking**: Each agent's OS process ID is recorded in `~/.devbox/agents.json` |
| 140 | +- **Stale pruning**: `devbox_agent_list` checks PIDs and cleans up dead sessions |
| 141 | +- **Manual cleanup**: Use `devbox destroy <workspace-name>` to remove orphaned workspaces |
| 142 | + |
| 143 | +No intervention is needed for graceful disconnects — the `devbox mcp serve` process cleans up on stdin EOF. |
| 144 | + |
| 145 | +## Multi-Server Scaling |
| 146 | + |
| 147 | +When no server is specified during agent registration, devbox selects the server with the most available resources (least-loaded selector). This distributes agents across the pool automatically. |
| 148 | + |
| 149 | +To check current distribution: |
| 150 | + |
| 151 | +``` |
| 152 | +→ devbox_server_list |
| 153 | +
|
| 154 | +← [ |
| 155 | + {"name": "dev1", "status": "online", "resources": {"available_score": 0.35}}, |
| 156 | + {"name": "dev2", "status": "online", "resources": {"available_score": 0.72}}, |
| 157 | + {"name": "dev3", "status": "online", "resources": {"available_score": 0.90}} |
| 158 | + ] |
| 159 | +``` |
| 160 | + |
| 161 | +The next agent registration would be placed on `dev3` (highest available score). |
| 162 | + |
| 163 | +## Monitoring |
| 164 | + |
| 165 | +### List Active Agents |
| 166 | + |
| 167 | +``` |
| 168 | +→ devbox_agent_list |
| 169 | +
|
| 170 | +← [ |
| 171 | + {"id": "agent-coder-a1b2c3d4", "name": "coder", "workspace": "agent-coder-a1b2c3d4", "server": "dev1"}, |
| 172 | + {"id": "agent-tester-e5f6g7h8", "name": "tester", "workspace": "agent-tester-e5f6g7h8", "server": "dev2"} |
| 173 | + ] |
| 174 | +``` |
| 175 | + |
| 176 | +### CLI Monitoring |
| 177 | + |
| 178 | +```bash |
| 179 | +# List all agent workspaces |
| 180 | +devbox list | grep "agent-" |
| 181 | + |
| 182 | +# Resource usage across all agents |
| 183 | +devbox stats |
| 184 | + |
| 185 | +# Interactive dashboard |
| 186 | +devbox tui |
| 187 | +``` |
| 188 | + |
| 189 | +## Claude Desktop Integration |
| 190 | + |
| 191 | +Add devbox to Claude Desktop's MCP configuration: |
| 192 | + |
| 193 | +```json |
| 194 | +{ |
| 195 | + "mcpServers": { |
| 196 | + "devbox": { |
| 197 | + "command": "devbox", |
| 198 | + "args": ["mcp", "serve"] |
| 199 | + } |
| 200 | + } |
| 201 | +} |
| 202 | +``` |
| 203 | + |
| 204 | +Claude can then create workspaces, execute commands, and manage environments through natural language. See [`examples/claude-desktop-config.json`](../examples/claude-desktop-config.json) for a complete configuration. |
| 205 | + |
| 206 | +## Best Practices |
| 207 | + |
| 208 | +1. **Always set resource limits** — prevent runaway agents from consuming all server resources. Default is 1 CPU, 1GB memory. |
| 209 | +2. **Use meaningful agent names** — names appear in `devbox_agent_list` and help identify workloads. |
| 210 | +3. **Snapshot before risky operations** — use `devbox_snapshot_create` to checkpoint state before destructive actions. |
| 211 | +4. **Monitor fleet utilization** — check `devbox_server_list` regularly to ensure servers aren't overloaded. |
| 212 | +5. **Scale horizontally** — add more servers to the pool rather than increasing per-server density. |
| 213 | +6. **Plan for crashes** — `devbox_agent_list` auto-prunes stale sessions, but monitor for orphaned workspaces during heavy usage. |
| 214 | + |
| 215 | +## Further Reading |
| 216 | + |
| 217 | +- [MCP Server Reference](MCP.md) — all tools, parameters, responses, and error codes |
| 218 | +- [CLI Quick Start](QUICKSTART.md) — getting started with devbox |
| 219 | +- [Configuration Reference](CONFIG.md) — all `devbox.yaml` fields |
0 commit comments