Skip to content

Commit 42165d7

Browse files
lpcoxCopilot
andauthored
fix(gvisor): skip iptables and route MCP gateway via NO_PROXY (#6401)
* fix(gvisor): skip iptables and route MCP gateway via NO_PROXY gVisor's isolated userspace netstack is not governed by the host-netns iptables DNAT/RETURN rules that awf-iptables-init installs, so egress (including the MCP gateway bypass at 172.30.0.1) silently failed under gVisor: MCP requests were routed through Squid and rejected with 403, leaving the agent unable to emit safe outputs. Add a usesIptables runtime capability (false for gvisor/sbx) that: - skips the awf-iptables-init container - sets AWF_SKIP_IPTABLES_INIT=1 so the entrypoint skips the init handshake - adds the network gateway + host.docker.internal to NO_PROXY so proxy-aware MCP clients reach the gateway directly Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: be040aef-5830-42b1-a871-4cade09f4585 * fix(gvisor): alias runsc runtime and clarify egress docs - container-runtime: canonicalize raw runsc OCI name to gVisor capabilities via an alias table; add isGvisorRuntime() and use it for the Claude/Bun check. - compose-generator.test: assert gVisor (networkIsolation:false) omits iptables-init, keeps the agent, and sets AWF_SKIP_IPTABLES_INIT (gvisor+runsc). - docs/gvisor-integration: distinguish Squid-routed internet egress from the direct MCP/host-gateway NO_PROXY exception; clarify the enforced boundary is topology (strict) / host DOCKER-USER default-deny (legacy), not proxy env. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: be040aef-5830-42b1-a871-4cade09f4585 --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent db44ecd commit 42165d7

10 files changed

Lines changed: 306 additions & 29 deletions

containers/agent/entrypoint.sh

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -139,8 +139,11 @@ wait_for_iptables() {
139139
#
140140
# In network-isolation (topology) mode there is no iptables-init container —
141141
# egress is enforced by Docker network topology — so skip the handshake.
142-
if [ "${AWF_NETWORK_ISOLATION:-}" = "1" ]; then
143-
echo "[entrypoint] Network-isolation mode: skipping iptables init container wait"
142+
# Likewise for runtimes whose network stack can't be governed by host-netns
143+
# iptables (e.g. gVisor's isolated netstack): AWF_SKIP_IPTABLES_INIT is set and
144+
# egress relies on the HTTP_PROXY/HTTPS_PROXY env vars instead.
145+
if [ "${AWF_NETWORK_ISOLATION:-}" = "1" ] || [ "${AWF_SKIP_IPTABLES_INIT:-}" = "1" ]; then
146+
echo "[entrypoint] iptables-init skipped (proxy-based egress): skipping init container wait"
144147
else
145148
echo "[entrypoint] Waiting for iptables initialization from init container..."
146149
INIT_TIMEOUT=300 # 300 * 0.1s = 30 seconds

docs/gvisor-integration.md

Lines changed: 55 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -99,8 +99,8 @@ gVisor is registered with `executionModel: 'compose'`:
9999

100100
```ts
101101
const RUNTIME_REGISTRY = {
102-
gvisor: { executionModel: 'compose', dockerRuntime: 'runsc', needsStaticDns: true },
103-
sbx: { executionModel: 'microvm', dockerRuntime: undefined, needsStaticDns: false },
102+
gvisor: { executionModel: 'compose', dockerRuntime: 'runsc', needsStaticDns: true, usesIptables: false },
103+
sbx: { executionModel: 'microvm', dockerRuntime: undefined, needsStaticDns: false, usesIptables: false },
104104
};
105105
```
106106

@@ -204,14 +204,51 @@ every static hostname into `/host/etc/hosts`. A new netstack-based runtime must
204204
account for both files.
205205
:::
206206

207-
### iptables DNAT must work inside the sandbox
207+
### gVisor skips iptables entirely (proxy-based egress)
208+
209+
gVisor's userspace netstack is **isolated from the host network namespace**, so
210+
the host-netns iptables DNAT/RETURN rules that `awf-iptables-init` installs
211+
(port 80/443 → Squid, plus the NAT `RETURN` bypass for the MCP gateway) never
212+
govern the sandbox's traffic. AWF therefore **does not run the iptables-init
213+
container for gVisor at all** — the runtime is registered with
214+
`usesIptables: false` in `container-runtime.ts`, which:
215+
216+
- skips `assembleIptablesInitService()` (`optional-services.ts`), and
217+
- sets `AWF_SKIP_IPTABLES_INIT=1` so `entrypoint.sh` doesn't wait for the (absent)
218+
init-container ready-file handshake.
219+
220+
**Internet egress** for proxy-aware clients is routed through Squid via the
221+
`HTTP_PROXY`/`HTTPS_PROXY` env vars. There is one deliberate exception: because
222+
the iptables NAT bypass for the MCP gateway is gone, `proxy-environment.ts` adds
223+
the network gateway IP (e.g. `172.30.0.1`) and `host.docker.internal` to
224+
`NO_PROXY` for non-iptables runtimes, so proxy-aware MCP clients (rmcp) connect
225+
to the gateway **directly** instead of being routed through Squid and rejected
226+
with `403 ERR_ACCESS_DENIED`.
227+
228+
Note that `NO_PROXY`/`HTTP_PROXY` are **client-side routing hints, not a security
229+
boundary** — the agent controls its own environment and could reach the gateway
230+
with or without them. The enforced egress boundary is external to the agent and
231+
runtime-independent:
232+
233+
- **Strict/default mode** (`networkIsolation: true`): the agent sits on an
234+
internal Docker network with no route off-host except through the dual-homed
235+
Squid; the network topology is the boundary.
236+
- **Legacy mode** (`networkIsolation: false`): host-level iptables in the
237+
`DOCKER-USER` (FORWARD) chain default-denies container→internet traffic
238+
(`host-iptables-rules.ts`) — this is what produces "No route to host" and is
239+
unaffected by which runtime the agent uses. The container-level iptables that
240+
gVisor skips was only ever a defense-in-depth DNAT fallback, never the primary
241+
boundary.
242+
243+
:::caution Proxy-unaware tools under gVisor
244+
With no iptables DNAT fallback, tools that ignore `HTTP_PROXY`/`HTTPS_PROXY`
245+
(e.g. a raw `/dev/tcp` connection) have no route to external hosts and will fail
246+
with "No route to host". Egress under gVisor requires proxy-aware clients.
247+
:::
208248

209-
AWF's defense-in-depth relies on iptables DNAT (port 80/443 → Squid:3128) applied
210-
inside the agent's network namespace. gVisor must support those rules for that
211-
fallback to hold. `.github/workflows/test-gvisor-compat.yml` is a **manual,
212-
non-gating diagnostic probe** that exercises iptables DNAT and proxy reachability
213-
inside a `runsc` sandbox; it is useful evidence, but not an enforced guarantee in
214-
CI.
249+
`.github/workflows/test-gvisor-compat.yml` remains a **manual, non-gating
250+
diagnostic probe** for iptables/proxy reachability inside a `runsc` sandbox; it
251+
documents the historical behavior but is not part of the enforced egress path.
215252

216253
### Runtime-specific compatibility shims
217254

@@ -265,6 +302,7 @@ myruntime: {
265302
executionModel: 'compose',
266303
dockerRuntime: 'my-oci-runtime', // the name registered in daemon.json
267304
needsStaticDns: false, // true if its netstack can't reach 127.0.0.11
305+
usesIptables: true, // false if host-netns iptables can't govern its traffic
268306
},
269307
```
270308

@@ -282,11 +320,14 @@ Set `needsStaticDns: true` only if the runtime cannot reach Docker's embedded DN
282320
`/etc/hosts` via `extra_hosts` *and* the chrooted `/host/etc/hosts`) and ensure
283321
topology peers are patched into both.
284322

285-
### 3. Confirm the network fallback works
323+
### 3. Confirm the network egress model
286324

287-
AWF's iptables DNAT-to-Squid path must function inside the runtime's network
288-
namespace. Add a compat check modeled on `test-gvisor-compat.yml` before relying
289-
on it.
325+
If the runtime shares the host network namespace (`usesIptables: true`), AWF's
326+
iptables DNAT-to-Squid path must function inside it — add a compat check modeled
327+
on `test-gvisor-compat.yml`. If the runtime has an isolated netstack
328+
(`usesIptables: false`, like gVisor), AWF skips iptables entirely and relies on
329+
the `HTTP_PROXY`/`HTTPS_PROXY` env vars plus a `NO_PROXY` entry for the MCP
330+
gateway; ensure the agent's tools are all proxy-aware.
290331

291332
### 4. Ensure the runtime is installed
292333

@@ -302,7 +343,7 @@ viable and may be faster.
302343
| --- | --- | --- |
303344
| Host-kernel / syscall isolation | ✅ owns it (Sentry) | selects the runtime |
304345
| In-sandbox network stack | ✅ netstack | works around its DNS limits |
305-
| Domain egress ACL || ✅ Squid + iptables DNAT |
346+
| Domain egress ACL || ✅ Squid (via `HTTP_PROXY`/`HTTPS_PROXY`; no iptables under gVisor) |
306347
| Credential injection || ✅ api-proxy (`COPILOT_*`) |
307348
| Chroot + capability drop || ✅ entrypoint / capsh |
308349
| Lifecycle | OCI runtime under compose |`docker compose` + `docker wait` |

src/compose-generator.test.ts

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -321,6 +321,45 @@ describe('generateDockerCompose', () => {
321321
});
322322
});
323323

324+
describe('gVisor runtime (non-iptables compose agent)', () => {
325+
it('omits iptables-init but keeps the compose agent when networkIsolation is false', () => {
326+
const config = {
327+
...mockConfig,
328+
containerRuntime: 'gvisor',
329+
networkIsolation: false,
330+
};
331+
const result = generateDockerCompose(config, mockNetworkConfig);
332+
333+
expect(result.services.agent).toBeDefined();
334+
expect(result.services['iptables-init']).toBeUndefined();
335+
});
336+
337+
it('sets AWF_SKIP_IPTABLES_INIT (not AWF_NETWORK_ISOLATION) in the agent environment', () => {
338+
const config = {
339+
...mockConfig,
340+
containerRuntime: 'gvisor',
341+
networkIsolation: false,
342+
};
343+
const result = generateDockerCompose(config, mockNetworkConfig);
344+
345+
expect(result.services.agent.environment?.AWF_SKIP_IPTABLES_INIT).toBe('1');
346+
expect(result.services.agent.environment?.AWF_NETWORK_ISOLATION).toBeUndefined();
347+
});
348+
349+
it('treats the raw runsc runtime name the same as gvisor', () => {
350+
const config = {
351+
...mockConfig,
352+
containerRuntime: 'runsc',
353+
networkIsolation: false,
354+
};
355+
const result = generateDockerCompose(config, mockNetworkConfig);
356+
357+
expect(result.services.agent).toBeDefined();
358+
expect(result.services['iptables-init']).toBeUndefined();
359+
expect(result.services.agent.environment?.AWF_SKIP_IPTABLES_INIT).toBe('1');
360+
});
361+
});
362+
324363
describe('microVM runtime (sbx)', () => {
325364
it('omits compose agent and agent-only helper services', () => {
326365
const config = {

src/container-runtime.test.ts

Lines changed: 50 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { resolveDockerRuntime, runtimeNeedsStaticDns, runtimeUsesComposeAgent } from './container-runtime';
1+
import { resolveDockerRuntime, runtimeNeedsStaticDns, runtimeUsesComposeAgent, runtimeUsesIptables, isGvisorRuntime } from './container-runtime';
22
import { sanitizeEnvForSbx } from './sbx-manager';
33

44
describe('container-runtime', () => {
@@ -13,9 +13,12 @@ describe('container-runtime', () => {
1313

1414
it('passes through unknown runtime names unchanged', () => {
1515
expect(resolveDockerRuntime('kata')).toBe('kata');
16-
expect(resolveDockerRuntime('runsc')).toBe('runsc');
1716
expect(resolveDockerRuntime('custom-runtime')).toBe('custom-runtime');
1817
});
18+
19+
it('resolves the runsc alias to gVisor (docker runtime runsc)', () => {
20+
expect(resolveDockerRuntime('runsc')).toBe('runsc');
21+
});
1922
});
2023

2124
describe('runtimeNeedsStaticDns', () => {
@@ -29,7 +32,10 @@ describe('container-runtime', () => {
2932

3033
it('returns false for unknown runtimes', () => {
3134
expect(runtimeNeedsStaticDns('kata')).toBe(false);
32-
expect(runtimeNeedsStaticDns('runsc')).toBe(false);
35+
});
36+
37+
it('returns true for the runsc alias (same as gvisor)', () => {
38+
expect(runtimeNeedsStaticDns('runsc')).toBe(true);
3339
});
3440

3541
it('returns false for undefined/empty', () => {
@@ -38,8 +44,30 @@ describe('container-runtime', () => {
3844
});
3945
});
4046

41-
describe('runtimeUsesComposeAgent', () => {
42-
it('returns true when no runtime is configured', () => {
47+
describe('runtimeUsesIptables', () => {
48+
it('returns false for gvisor (isolated netstack)', () => {
49+
expect(runtimeUsesIptables('gvisor')).toBe(false);
50+
});
51+
52+
it('returns false for sbx (microVM manages own egress)', () => {
53+
expect(runtimeUsesIptables('sbx')).toBe(false);
54+
});
55+
56+
it('returns true for unknown runtimes (share host netns)', () => {
57+
expect(runtimeUsesIptables('kata')).toBe(true);
58+
});
59+
60+
it('returns false for the runsc alias (same as gvisor)', () => {
61+
expect(runtimeUsesIptables('runsc')).toBe(false);
62+
});
63+
64+
it('returns true for undefined/empty (default runc)', () => {
65+
expect(runtimeUsesIptables(undefined)).toBe(true);
66+
expect(runtimeUsesIptables('')).toBe(true);
67+
});
68+
});
69+
70+
describe('runtimeUsesComposeAgent', () => { it('returns true when no runtime is configured', () => {
4371
expect(runtimeUsesComposeAgent(undefined)).toBe(true);
4472
});
4573

@@ -56,6 +84,23 @@ describe('container-runtime', () => {
5684
expect(runtimeUsesComposeAgent('runsc')).toBe(true);
5785
});
5886
});
87+
88+
describe('isGvisorRuntime', () => {
89+
it('returns true for the gvisor friendly name', () => {
90+
expect(isGvisorRuntime('gvisor')).toBe(true);
91+
});
92+
93+
it('returns true for the raw runsc runtime name', () => {
94+
expect(isGvisorRuntime('runsc')).toBe(true);
95+
});
96+
97+
it('returns false for other/undefined runtimes', () => {
98+
expect(isGvisorRuntime('sbx')).toBe(false);
99+
expect(isGvisorRuntime('kata')).toBe(false);
100+
expect(isGvisorRuntime(undefined)).toBe(false);
101+
expect(isGvisorRuntime('')).toBe(false);
102+
});
103+
});
59104
});
60105

61106
describe('sanitizeEnvForSbx', () => {

src/container-runtime.ts

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,21 @@ interface RuntimeCapabilities {
7474
* @see https://github.com/google/gvisor/issues/7469
7575
*/
7676
readonly needsStaticDns: boolean;
77+
78+
/**
79+
* When `true`, AWF sets up egress control via the `awf-iptables-init` container,
80+
* which applies host-netns iptables DNAT/RETURN rules inside the agent's network
81+
* namespace (redirect port 80/443 → Squid, and bypass Squid for the MCP gateway).
82+
*
83+
* When `false`, those rules cannot govern the agent's traffic — e.g. gVisor's
84+
* userspace netstack is isolated from the host network namespace, so the
85+
* iptables-init container is skipped entirely and egress relies solely on the
86+
* `HTTP_PROXY`/`HTTPS_PROXY` env vars plus `NO_PROXY` (the MCP gateway is reached
87+
* directly instead of via an iptables bypass).
88+
*
89+
* @see https://github.com/google/gvisor/issues/7469
90+
*/
91+
readonly usesIptables: boolean;
7792
}
7893

7994
/**
@@ -86,15 +101,46 @@ const RUNTIME_REGISTRY: Readonly<Record<string, RuntimeCapabilities>> = {
86101
executionModel: 'compose',
87102
dockerRuntime: 'runsc',
88103
needsStaticDns: true,
104+
// gVisor's isolated netstack can't be governed by host-netns iptables rules,
105+
// so skip the iptables-init container and route egress via proxy env vars.
106+
usesIptables: false,
89107
},
90108
// Future: Docker sbx microVM backend
91109
sbx: {
92110
executionModel: 'microvm',
93111
dockerRuntime: undefined,
94112
needsStaticDns: false, // sbx manages its own DNS
113+
usesIptables: false, // microVM manages its own network egress
95114
},
96115
};
97116

117+
/**
118+
* Aliases for runtime names that should resolve to the same capabilities as a
119+
* canonical registry entry. This lets users pass either the friendly name
120+
* (`gvisor`) or the raw Docker OCI runtime name (`runsc`) and get identical
121+
* behavior — without an alias, `runsc` would fall through to the unknown-runtime
122+
* defaults (iptables-capable), reintroducing the gVisor egress bug.
123+
*/
124+
const RUNTIME_ALIASES: Readonly<Record<string, string>> = {
125+
runsc: 'gvisor',
126+
};
127+
128+
/**
129+
* Canonicalizes a user-facing runtime name to its registry key, following the
130+
* alias table. Unknown names are returned unchanged.
131+
*/
132+
function canonicalRuntime(runtime: string): string {
133+
return RUNTIME_ALIASES[runtime] ?? runtime;
134+
}
135+
136+
/**
137+
* Looks up the capabilities for a runtime name, following aliases. Returns
138+
* `undefined` for unknown names (raw Docker runtime identifiers / default runc).
139+
*/
140+
function lookupCapabilities(runtime: string): RuntimeCapabilities | undefined {
141+
return RUNTIME_REGISTRY[canonicalRuntime(runtime)];
142+
}
143+
98144
// ─── Public API ──────────────────────────────────────────────────────────────
99145

100146
/**
@@ -105,7 +151,7 @@ const RUNTIME_REGISTRY: Readonly<Record<string, RuntimeCapabilities>> = {
105151
* runtime field.
106152
*/
107153
export function resolveDockerRuntime(runtime: string): string | undefined {
108-
const entry = RUNTIME_REGISTRY[runtime];
154+
const entry = lookupCapabilities(runtime);
109155
if (entry) return entry.dockerRuntime;
110156
// Unknown name — pass through as a raw Docker runtime identifier
111157
return runtime;
@@ -121,7 +167,24 @@ export function resolveDockerRuntime(runtime: string): string | undefined {
121167
*/
122168
export function runtimeNeedsStaticDns(runtime: string | undefined): boolean {
123169
if (!runtime) return false;
124-
return RUNTIME_REGISTRY[runtime]?.needsStaticDns ?? false;
170+
return lookupCapabilities(runtime)?.needsStaticDns ?? false;
171+
}
172+
173+
/**
174+
* Returns `true` when AWF should set up egress control for the runtime via the
175+
* `awf-iptables-init` container (host-netns iptables DNAT/RETURN rules).
176+
*
177+
* Returns `false` for runtimes whose network stack cannot be governed by those
178+
* rules (e.g. gVisor's isolated netstack, or microVM backends that manage their
179+
* own egress). For these, AWF skips the iptables-init container and relies on
180+
* proxy env vars + `NO_PROXY`.
181+
*
182+
* Defaults to `true` for unknown/undefined runtimes (raw Docker runtime names
183+
* and the default runc), which share the host network namespace.
184+
*/
185+
export function runtimeUsesIptables(runtime: string | undefined): boolean {
186+
if (!runtime) return true;
187+
return lookupCapabilities(runtime)?.usesIptables ?? true;
125188
}
126189

127190
/**
@@ -136,7 +199,18 @@ export function runtimeNeedsStaticDns(runtime: string | undefined): boolean {
136199
*/
137200
export function runtimeUsesComposeAgent(runtime: string | undefined): boolean {
138201
if (!runtime) return true;
139-
const entry = RUNTIME_REGISTRY[runtime];
202+
const entry = lookupCapabilities(runtime);
140203
if (!entry) return true; // unknown runtime → assume compose
141204
return entry.executionModel === 'compose';
142205
}
206+
207+
/**
208+
* Returns `true` when the configured runtime is gVisor, accepting either the
209+
* friendly name (`gvisor`) or the raw Docker OCI runtime name (`runsc`). Use
210+
* this instead of a direct `=== 'gvisor'` comparison so both spellings are
211+
* handled consistently.
212+
*/
213+
export function isGvisorRuntime(runtime: string | undefined): boolean {
214+
if (!runtime) return false;
215+
return canonicalRuntime(runtime) === 'gvisor';
216+
}

0 commit comments

Comments
 (0)