feat(v1): execution-time network isolation for Docker (Linux+macOS) with allow/block egress lists#2024
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c93f2ae8f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc73789990
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 98426856a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8604a47e87
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
8604a47 to
384ace5
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 384ace51dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b5c4d47a6c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9e9a2d9dd2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
xeophon
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c30a8375b1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
DockerConfig.network_access=False now gives containers full internet through setup, then removes it right before the agent starts (Runtime.prepare_execution, called from Rollout after tool/user servers are up). Afterwards the container reaches only host services (interception, host MCP servers) — no sidecars, no proxies. Linux: one shared, persistent --internal network (verifiers-offline) that every offline rollout joins; the cut is 'docker network disconnect bridge', leaving only the bridge gateway (the host) reachable. The shared interception pool additionally binds that gateway IP (extra_host), so multiplexing is fully preserved — no per-rollout networks, servers, or subnet pressure. macOS (Docker Desktop/OrbStack): internal networks have no route to the host, so a ~100ms one-shot helper (docker run --rm --network container:X --cap-add NET_ADMIN alpine) pins a /32 route to the host NAT IP, drops the default route and blackholes DNS; URLs keep using host.docker.internal via an /etc/hosts entry. The harness container never holds NET_ADMIN, so the agent can't undo the cut (NET_ADMIN/NET_RAW dropped, no-new-privileges, IPv6 off). MCP: colocated servers work unchanged; task-scoped and shared host servers bind the same gateway via MCP_HOST; external URLs, non-colocated tool runtime containers, and colocated user sims are rejected with clear errors. A post-cut probe fails loudly when the host firewall (e.g. ufw) drops container-to-host traffic instead of hanging the agent. Runtime.prepare_execution + host_service_host + network_isolated are runtime-agnostic seams: Prime sandbox allow/block lists can plug in later by implementing prepare_execution in runtimes/prime.py only.
- verifiers-offline now sets com.docker.network.bridge.enable_icc=false: concurrent offline rollouts can't reach each other on the shared network (the gateway/host is unaffected — that traffic is INPUT, not FORWARD). - serve_tools rewrites configured loopback MCP endpoints (ToolsetConfig.url and shared external URLs) through host_url when the harness is isolated; post-cut, loopback inside the container is the container itself, not the host.
DockerConfig gains allow/block host patterns (fnmatch wildcards, apex included), mirroring the upcoming Prime sandbox semantics: broad access (network_access=True, no block list) is unchanged; network_access=False cuts to host-only as before, now plus the allow list; a block list subtracts in either mode. Enforcement: the L3 cut is unchanged (direct egress impossible). When filtering is configured, a shared host-side egress proxy (runtimes/egress.py: CONNECT for HTTPS, absolute-form for plain HTTP, one per ruleset) is the only way out and enforces block-wins, then default-allow (broad) or default-deny (allow list). Proxy env is injected per-command only after the cut, so setup keeps full unfiltered access; NO_PROXY keeps framework traffic direct, and the framework host is always proxy-allowed for clients that ignore it. prepare_execution validates routes against the same matcher; the cut is idempotent. Review fixes: reused verifiers-offline networks are validated (foreign non-internal refused, stale ours-and-empty recreated); configured loopback MCP URLs are rejected on Linux (a foreign server can't be rebound to the gateway) while macOS keeps the host.docker.internal rewrite; the macOS cut now also removes the connected bridge-subnet route so peer containers are unreachable. Verified live on Linux (VPS, Docker 29) and macOS (OrbStack): setup unfiltered, allowlist 200 / denied 000, direct bypass blocked, interception via NO_PROXY, peer blocked on both platforms, idempotent cut, network validation cases.
…probe robustness - Environment._offline_bind and shared_tools used only network_access=False to decide isolation, but DockerRuntime.network_isolated is also true with a non-empty block list — block-only evals on Linux cut containers without binding the gateway on the interception pool/shared MCP servers. The predicate now lives on DockerConfig and is used everywhere. - host_matcher strips trailing dots on both sides: https://example.com./ can no longer bypass a block on example.com. - prepare_execution sets _cut before the post-cut probe, so a tunneled (external) model route is probed through the egress proxy — the way the harness will reach it — instead of failing on a direct connection. - The wget probe branch accepts GNU wget's exit 8 (server error response = reachable): GNU quiet mode prints no HTTP/ line even on a healthy 404; busybox prints it even with -q.
c30a837 to
cb0f2ae
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cb0f2aec02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 54534f1fe9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 66db575. Configure here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6e3fc17bdc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| writer.write(b"HTTP/1.1 403 Forbidden\r\nContent-Length: 0\r\n\r\n") | ||
| await writer.drain() | ||
| return | ||
| upstream_reader, upstream_writer = await asyncio.open_connection(host, port) |
There was a problem hiding this comment.
Fresh evidence relative to the earlier loopback finding is that the new check only applies ip_address(hostname) to the literal host string before this open_connection call. In filtered default-allow mode (network_access=True with any block) an agent can request resolver aliases such as 2130706433, 0177.0.0.1, or a DNS name that resolves to 127.0.0.1; the policy permits the non-loopback-looking hostname and then the proxy dials the Verifiers host loopback, exposing services that are supposed to be reserved for framework routes. Resolve/canonicalize the destination address and reject loopback results before connecting.
Useful? React with 👍 / 👎.
| ```toml | ||
| [harness.runtime] | ||
| type = "docker" | ||
| network_access = false |
There was a problem hiding this comment.
this is redunant with allow/block, no? should we just have allow = ["*"] as the default?
There was a problem hiding this comment.
and then allow and block should be mutually exclusive
|
|
||
| The names of these tools are set by the respective harness. Consult the relevant documentation for the given harness for the relevant name(s). Some harnesses do not offer support to disable tools. | ||
|
|
||
| ## Docker network policies |
There was a problem hiding this comment.
should we generalize this section so that we can reuse the same once prime sandbox as support for this?
| if ( | ||
| isinstance(cfg.runtime, DockerConfig) | ||
| and cfg.runtime.network_isolated | ||
| and not (colocated and harness_runtime is not None) | ||
| ): | ||
| raise ToolsetError( | ||
| "Docker network policies are supported on the harness runtime; " | ||
| f"server {server.server_name!r} must be colocated or use an " | ||
| "unrestricted Docker runtime" | ||
| ) |
There was a problem hiding this comment.
why is this true? in my mind:
- if the server has own runtime, can configure allow/block as want
- if server is colocated, it will adopt the harness runtimes' setting
| and colocated | ||
| and for_host | ||
| ): | ||
| raise UserError( |
There was a problem hiding this comment.
ouf ye this is ultra ugly. defo a smell with how we do user sim rn (e.g. that we hop from agent -> host -> user and back) but it is what it is for now
There was a problem hiding this comment.
if this reusable across runtimes? if not, can we either put this into docker.py or make docker a submodule runtimes/docker/ with 2 files?
2 participants
Overview
Adds execution-time URL allowlists and blocklists to v1 Docker harness runtimes on Linux and macOS.
Trusted setup is always online: image startup, task and harness setup, package installation, and MCP/user-server provisioning all happen before restrictions are applied. Immediately before the agent starts, Verifiers installs the configured policy and keeps it active through finalization and scoring.
Filtering is handled by an HTTP(S) proxy in the Verifiers process. There is no proxy sidecar, persistent helper container, public interception tunnel, or per-rollout Docker network.
Configuration
network_access = true, noblockentriesnetwork_access = true, withblockentriesnetwork_access = falseallowdestinationsInterception and MCP URLs are framework routes and always remain reachable. User
blockrules take precedence over userallowrules; framework routes take precedence over both.Rules may be bare host patterns or URL origins. A scheme or port narrows a match, URL paths are ignored, trailing dots are normalized, and
*.example.comincludes the apex. CONNECT uses port 443 unless an explicit HTTPS origin inallowor the framework routes names another port.Host-loopback destinations are reserved for framework routes and cannot be exposed by the default-allow posture or a user
allowrule.Harbor task policies
Harbor tasksets resolve the pinned Harbor schema's effective agent policy before selecting a runtime. An explicit
[agent].network_modeoverrides the[environment]baseline, and legacy[environment].allow_internetvalues are normalized by Harbor.publicno-networknetwork_access = falseallowlistnetwork_access = falseand adds Harbor'sallowed_hoststoallowTask-level restrictions and evaluator policy compose: evaluator
allowentries remain available, evaluatorblockentries still win, and a public Harbor task never widens a restricted evaluator runtime. Restricted Harbor tasks require Docker so interception and MCP routes can remain available under the framework-aware policy. The effective agent policy stays active through shared verifier scoring.Enforcement
Filtered containers start on the normal Docker bridge with a permissive proxy policy so setup can use the network.
Runtime.prepare_execution()then swaps in the rollout's framework-aware policy and removes direct egress beforeHarness.run().host.docker.internal. At the execution boundary, only the route and TCP port needed for that proxy remain usable.The helpers exit after installing the listener or network rules, so the harness is the only persistent container. Direct HTTP, raw TCP, DNS, peer-container, and arbitrary host-port access cannot bypass the proxy policy.
Interception and MCP
The rollout passes its interception endpoint and resolved MCP URLs to
prepare_execution()after all servers are provisioned. Colocated MCP servers stay on containerlocalhost; host-local, shared, and external MCP endpoints use the in-process proxy without requiring a public tunnel.Network policy belongs to the harness Docker runtime. A separately filtered Docker tool-server runtime is rejected in favor of a colocated or unrestricted server runtime. Colocated user simulators are unsupported for filtered harnesses because they are driven by the host.
Documentation and lifecycle
Debug execution applies the same network boundary after task setup. GSM8K prepares its verifier environment during online setup so scoring remains available after the cut.
The Docker sandbox guide, Harbor guide, and bundled environment reference document configuration, Harbor precedence, rule matching, lifecycle timing, platform-specific enforcement, and the relationship to Docker Sandboxes network governance.
Note
Add execution-time network isolation for Docker with allow/block egress lists
network_access,allow, andblockfields toDockerConfig(docker.py); when isolation is enabled, the container starts on a bridge network and routes are cut viaprepare_execution()just before agent work begins.TaskDatagainsnetwork_accessandnetwork_allowfields (task.py); tasks declaringnetwork_access=Falserequire a Docker runtime and inject their allowlist into it.NetworkModeandallowed_hostsinto task network policy (taskset.py).localhostfor in-container reachability.prepare_execution()time; background servers must be started before this point or they will lose connectivity. The isolation mechanism differs between Linux (in-container netns listener) and macOS (host-side proxy withhost.docker.internal).Macroscope summarized 6e3fc17.
Note
High Risk
Changes core Docker networking, route manipulation, and egress policy for agent execution; misconfiguration or platform edge cases could break rollouts or weaken isolation.
Overview
Adds execution-time network policies for v1 Docker harness runtimes: trusted setup stays online, then
Runtime.prepare_execution()applies filtering that remains active through agent run, finalize, and scoring.DockerConfiggainsnetwork_access,allow, andblock; isolated containers use bridge networking, an in-processEgressProxy(NetworkPolicyrule matching), HTTP(S) env proxying, and one-shot route/iptablescuts (Linux vs macOS host-gateway paths).TaskDataaddsnetwork_access/network_allow, merged inresolve_runtime_config(restrictions compose; public tasks cannot widen evaluator policy).Rollout and debug call
prepare_executionwith interception and MCP URLs after provisioning. MCP launch rejects separately filtered Docker tool runtimes unless colocated; adjusts colocated URLs under isolation. Harbor mapsNetworkMode/allowed_hostsinto task data. GSM8K pre-provisions its verifier uv env insetupbefore the cut.Docs cover config, Harbor mapping, and enforcement semantics.
Reviewed by Cursor Bugbot for commit 6e3fc17. Bugbot is set up for automated code reviews on this repo. Configure here.