feat(compile): adopt AWF strict network isolation (#1534)

* feat(compile): adopt AWF strict network isolation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94c43f6f-d9b6-40ab-99c3-3f9d05a1d72b

* fix(mcp): allow bridge-hosted MCPG requests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94c43f6f-d9b6-40ab-99c3-3f9d05a1d72b

* fix(test): call routed noop tool by name

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94c43f6f-d9b6-40ab-99c3-3f9d05a1d72b

* fix(compile): bypass Squid for topology MCPG

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94c43f6f-d9b6-40ab-99c3-3f9d05a1d72b

* refactor(mcp): run SafeOutputs as stdio container

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94c43f6f-d9b6-40ab-99c3-3f9d05a1d72b

* fix(test): match SafeOutputs runtime glibc floor

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94c43f6f-d9b6-40ab-99c3-3f9d05a1d72b

Author: jamesadevine

.gitattributes

@@ -3,6 +3,7 @@
 # Install scripts must always be LF (executed on Linux/macOS via curl|sh).
 scripts/install/*.sh text eol=lf
 scripts/install/*.ps1 text eol=lf
+tests/awf-copilot-safeoutputs/*.sh text eol=lf
 # Agency plugin is a Linux/macOS-consumed artifact (and release-please rewrites
 # it on Linux CI) — keep every text file LF.
 agency/plugins/** text eol=lf

.github/workflows/copilot-cli-safeoutputs.yml

@@ -15,7 +15,10 @@ on:
 jobs:
   contract:
     if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
-    runs-on: ubuntu-latest
+    # Match the Linux release build floor: the mounted binary runs inside the
+    # Ubuntu 22-based AWF agent image.
+    runs-on: ubuntu-22.04
+    timeout-minutes: 20
     permissions:
       contents: read
       copilot-requests: write
@@ -36,18 +39,33 @@ jobs:
           rustc --version
           cargo --version
 
-      - name: Resolve compiler-pinned Copilot CLI version
-        id: copilot-version
+      - name: Build SafeOutputs runtime and resolve source-pinned versions
+        id: versions
         run: |
           set -euo pipefail
-          version="$(cargo run --quiet -- catalog --kind versions --json | jq -r '.versions.copilot_cli')"
-          test -n "$version" && test "$version" != "null"
-          echo "version=${version}" >> "$GITHUB_OUTPUT"
+          cargo build --quiet --bin ado-aw
+
+          read_version() {
+            local constant="$1"
+            local source="$2"
+            awk -F'"' -v prefix="pub const ${constant}: &str = " \
+              'index($0, prefix) == 1 { print $2; exit }' "$source"
+          }
+
+          while IFS=: read -r key constant source; do
+            version="$(read_version "$constant" "$source")"
+            test -n "$version" && test "$version" != "null"
+            echo "${key}=${version}" >> "$GITHUB_OUTPUT"
+          done <<'VERSIONS'
+          copilot_cli:COPILOT_CLI_VERSION:src/engine.rs
+          awf:AWF_VERSION:src/compile/common.rs
+          mcpg:MCPG_VERSION:src/compile/common.rs
+          VERSIONS
 
       - name: Install compiler-pinned GitHub Copilot CLI
         run: |
           set -euo pipefail
-          version="${{ steps.copilot-version.outputs.version }}"
+          version="${{ steps.versions.outputs.copilot_cli }}"
           base_url="https://github.com/github/copilot-cli/releases/download/v${version}"
           temp_dir="$(mktemp -d)"
           trap 'rm -rf "$temp_dir"' EXIT
@@ -67,10 +85,35 @@ jobs:
           echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
           "$RUNNER_TEMP/bin/copilot" --version
 
-      - name: Run live Copilot CLI + SafeOutputs contract test
+      - name: Install compiler-pinned AWF
+        run: |
+          set -euo pipefail
+          version="${{ steps.versions.outputs.awf }}"
+          base_url="https://github.com/github/gh-aw-firewall/releases/download/v${version}"
+          temp_dir="$(mktemp -d)"
+          trap 'rm -rf "$temp_dir"' EXIT
+          curl -fsSL --retry 3 --retry-delay 5 \
+            -o "$temp_dir/awf-linux-x64" \
+            "$base_url/awf-linux-x64"
+          curl -fsSL --retry 3 --retry-delay 5 \
+            -o "$temp_dir/checksums.txt" \
+            "$base_url/checksums.txt"
+          expected="$(awk '$2 == "awf-linux-x64" { print $1; exit }' "$temp_dir/checksums.txt" | tr 'A-F' 'a-f')"
+          actual="$(sha256sum "$temp_dir/awf-linux-x64" | awk '{ print $1 }' | tr 'A-F' 'a-f')"
+          test -n "$expected"
+          test "$expected" = "$actual"
+          install -m 0755 "$temp_dir/awf-linux-x64" "$RUNNER_TEMP/bin/awf"
+          "$RUNNER_TEMP/bin/awf" --version
+
+      - name: Run handwritten AWF + Copilot + SafeOutputs contract
         env:
           ADO_AW_COPILOT_CLI_ARTIFACT_DIR: ${{ runner.temp }}/copilot-cli-safeoutputs
-        run: cargo test --test copilot_cli_safeoutputs_tests real_copilot_cli_noop_contract -- --ignored --nocapture
+          ADO_AW_BIN: ${{ github.workspace }}/target/debug/ado-aw
+          AWF_BIN: ${{ runner.temp }}/bin/awf
+          COPILOT_BIN: ${{ runner.temp }}/bin/copilot
+          AWF_VERSION: ${{ steps.versions.outputs.awf }}
+          MCPG_VERSION: ${{ steps.versions.outputs.mcpg }}
+        run: bash tests/awf-copilot-safeoutputs/run.sh
 
       - name: Upload contract-test artifacts on failure
         if: failure()

AGENTS.md

@@ -439,7 +439,9 @@ Following the gh-aw security model:
    environment via AWF — see [`docs/network.md`](docs/network.md). **Scope
    note:** AWF's L7 allowlist wraps *only* the agent's copilot command
    (`awf … --allow-domains … -- '<engine_run>'` in
-   `src/compile/agentic_pipeline.rs::run_agent_step`). All other ADO steps —
+   `src/compile/agentic_pipeline.rs::run_agent_step`). AWF runs rootlessly in
+   strict topology mode; the Agent reaches trusted MCPG through
+   `--topology-attach awmg-mcpg`, not through host access. All other ADO steps —
    binary/bundle downloads, `docker pull`, ACR/NuGet auth (including the
    `supply-chain:` mirror fetches) — run *outside* the sandbox with the build
    agent pool's normal network, so they do **not** need entries in the AWF

docs/ado-script.md

@@ -70,7 +70,7 @@ pipeline** as runtime helpers. Today it produces thirteen bundles:
   [`engine.md`](engine.md#github-app-backed-copilot-engine-auth).
 - `prepare-pr-base.js` — create-pull-request base-ref preparer that runs when
   `create-pull-request` is configured, in **both** the Agent job (before the
-  Copilot invocation, so the host-side SafeOutputs MCP server can compute a diff
+  Copilot invocation, so the containerized SafeOutputs MCP server can compute a diff
   base — issue #1413) **and** the SafeOutputs job (before `ado-aw execute`, so
   the Stage 3 executor's `git worktree add` resolves `origin/<target>`; each ADO
   job has an isolated checkout, so the ref must be re-fetched in the job that

docs/audit.md

@@ -54,8 +54,7 @@ URL-encoded project segments are decoded before the ADO context is resolved. `t=
 │   │   └── otel.jsonl                # Copilot OTel (when emitted)
 │   └── logs/
 │       ├── firewall/                 # AWF Squid proxy logs
-│       ├── mcpg/                     # MCP Gateway logs
-│       ├── safeoutputs.log           # SafeOutputs HTTP server log
+│       ├── mcpg/                     # MCP Gateway logs (includes the SafeOutputs stdio child's stdout/stderr)
 │       └── agent-output.txt          # Filtered agent stdout
 ├── analyzed_outputs[_<BuildId>]/     # Downloaded artifact (Detection stage)
 │   ├── threat-analysis.json          # Aggregate verdict + reasons

docs/cli.md

@@ -23,10 +23,11 @@ Global flags (apply to all subcommands): `--verbose, -v` (enable info-level logg
   - `<pipeline>` - Path to the pipeline YAML file to verify
   - The source markdown path is auto-detected from the `@ado-aw` header in the pipeline file
   - Useful for CI checks to ensure pipelines are regenerated after source changes
-- `mcp <output_directory> <bounding_directory>` - Run SafeOutputs as a stdio MCP server
+- `mcp <output_directory> <bounding_directory>` - Run SafeOutputs as a stdio MCP server. **This is what compiled pipelines use**: MCPG spawns it as a hardened, network-isolated sibling container entrypoint (see [`docs/mcpg.md`](mcpg.md)).
   - `--enabled-tools <name>` - Restrict available tools to those named (repeatable)
 - `mcp-author` - Run the author-facing stdio MCP server for IDE/Copilot Chat integrations. See [`mcp-author.md`](mcp-author.md) for the full tool surface and trust model.
-- `mcp-http <output_directory> <bounding_directory>` - Run SafeOutputs as an HTTP MCP server (for MCPG integration)
+- `mcp-http <output_directory> <bounding_directory>` - Run SafeOutputs as an HTTP MCP server (for direct/local MCPG integration or manual testing). **Optional and not compiler-emitted** — compiled pipelines no longer start this server; see [`docs/local-development.md`](local-development.md).
+  - `--bind-address <ip>` - Listener IP address (default: `127.0.0.1`)
   - `--port <port>` - Port to listen on (default: 8100)
   - `--api-key <key>` - API key for authentication (auto-generated if not provided)
   - `--enabled-tools <name>` - Restrict available tools to those named (repeatable)

docs/engine.md

@@ -221,18 +221,23 @@ step and the run could observe an expired token.
 
 #### Credential isolation (api-proxy sidecar)
 
-When a provider credential is configured (`base-url` + `token`/`api-key`), the
-compiler automatically enables the AWF **api-proxy sidecar**
-(`--enable-api-proxy`) on the agent step and pre-pulls its container image. With
-the sidecar active:
+AWF 0.27.32+ **always** enables its api-proxy sidecar (the old
+`--enable-api-proxy` flag is deprecated and no longer needed), and the
+compiler always pre-pulls the `api-proxy` container image alongside `squid`
+and `agent` for both the Agent and Detection jobs, regardless of whether a
+provider credential is configured. When a provider credential *is* configured
+(`base-url` + `token`/`api-key`), the compiler additionally passes the
+credential env keys as AWF `--exclude-env` flags on the agent step. With the
+sidecar active:
 
 - The **real** credential is read by the AWF host process and held inside the
   proxy container; the agent container receives only a placeholder value and a
   proxy URL. The proxy strips the client's auth header and injects the real
   credential on the outbound request, so the secret never reaches the Copilot
   CLI process or the agent sandbox.
-- The credential keys are additionally passed as AWF `--exclude-env` flags so the
-  raw value is never copied into the agent via `--env-all` (defense-in-depth).
+- The `--exclude-env` flags are defense-in-depth: they keep the raw credential
+  value out of the agent via `--env-all` passthrough, on top of the sidecar's
+  placeholder override.
 
 This isolation applies to **both** the Agent stage and the Detection
 (threat-analysis) stage: the detection Copilot run inherits the same

docs/local-development.md

@@ -6,8 +6,15 @@
 # Local Development Guide
 
 This guide explains how to run an agentic pipeline locally for development and
-testing. The workflow mirrors what the compiled Azure DevOps pipeline does, but
-each step is run manually on your machine.
+testing using the optional `ado-aw mcp-http` command, which remains supported
+for direct/local use. **This is not what compiled pipelines run.** Since the
+containerized SafeOutputs architecture landed, compiled pipelines spawn
+SafeOutputs as a hardened stdio child container through MCPG (`ado-aw mcp`,
+`--network none`, non-root, read-only rootfs) — there is no host-side HTTP
+server, bridge-gateway resolution, or `host.docker.internal` mapping in CI.
+See [`docs/mcpg.md`](mcpg.md) for that shape. The manual `mcp-http` workflow
+below is a simpler, still-supported way to exercise an agent's prompt + MCP
+wiring by hand without a full pipeline replica.
 
 ## Prerequisites
 
@@ -21,7 +28,8 @@ each step is run manually on your machine.
 A pipeline execution has three stages:
 
 1. **SafeOutputs MCP server** — receives tool calls from the agent and writes
-   them as NDJSON records
+   them as NDJSON records. Locally this guide uses `ado-aw mcp-http`; compiled
+   pipelines use `ado-aw mcp` as a stdio child spawned by MCPG instead.
 2. **Agent execution** — Copilot CLI runs with a prompt and MCP config,
    interacting with SafeOutputs (and optionally other MCPs via MCPG)
 3. **Safe output execution** — processes the NDJSON records and makes real ADO
@@ -36,15 +44,23 @@ export WORK_DIR=$(mktemp -d)
 echo "Working directory: $WORK_DIR"
 ```
 
-### 2. Start the SafeOutputs HTTP server
+### 2. Start the SafeOutputs HTTP server (local-only; optional `mcp-http`)
+
+This step is specific to the local manual workflow. Resolve the Docker bridge
+gateway address so a locally-run MCPG container (started outside AWF) can
+reach the host-side server:
 
 ```bash
+# Resolve the Docker bridge gateway (same address MCPG uses to reach the host)
+export SAFE_OUTPUTS_BIND_ADDRESS=$(docker network inspect bridge | jq -er '.[0].IPAM.Config[0].Gateway')
+
 # Pick a port and generate an API key
 export SO_PORT=8100
 export SO_API_KEY=$(openssl rand -hex 32)
 
-# Start in the background
+# Start in the background, bound only to the bridge gateway address
 cargo run -- mcp-http \
+  --bind-address "$SAFE_OUTPUTS_BIND_ADDRESS" \
   --port "$SO_PORT" \
   --api-key "$SO_API_KEY" \
   "$WORK_DIR" \
@@ -54,7 +70,7 @@ export SO_PID=$!
 echo "SafeOutputs PID: $SO_PID"
 
 # Wait for health check
-until curl -sf "http://127.0.0.1:$SO_PORT/health" > /dev/null 2>&1; do
+until curl -sf "http://$SAFE_OUTPUTS_BIND_ADDRESS:$SO_PORT/health" > /dev/null 2>&1; do
   sleep 1
 done
 echo "SafeOutputs ready"
@@ -69,8 +85,12 @@ Skip this step if your agent only uses SafeOutputs (no `mcp-servers:` or
 export MCPG_PORT=8080
 export MCPG_API_KEY=$(openssl rand -hex 32)
 
-# Generate MCPG config — adapt the JSON to your agent's mcp-servers front matter.
-# See the compiled pipeline's mcpg-config.json for the expected format.
+# Generate MCPG config for this manual local flow. Note: compiled pipelines
+# wire "safeoutputs" as a hardened stdio child container (`type: "stdio"`,
+# container: the pinned AWF `agent` image, entrypoint: ado-aw mcp ...; see
+# docs/mcpg.md), not the HTTP backend shown below -- the HTTP shape here is
+# specific to this local/manual workflow, which runs SafeOutputs as a plain
+# host process for simplicity.
 cat > "$WORK_DIR/mcpg-config.json" <<EOF
 {
   "mcpServers": {
@@ -90,12 +110,19 @@ cat > "$WORK_DIR/mcpg-config.json" <<EOF
 }
 EOF
 
-# Start MCPG container (macOS/Windows — use host.docker.internal)
-docker run -i --rm --name ado-aw-mcpg \
-  -p "$MCPG_PORT:$MCPG_PORT" \
+# Start MCPG on Docker's default bridge network, published to localhost.
+# The bridge network + docker-socket + port-mapping shape below matches what
+# compiled pipelines do for the MCPG container itself; only the SafeOutputs
+# backend type differs (HTTP here vs. stdio child container in CI).
+# The local Copilot process runs on the host, so the config above advertises
+# 127.0.0.1; compiled AWF pipelines advertise awmg-mcpg instead.
+docker run -i --rm --name awmg-mcpg \
+  --network bridge \
+  -p "127.0.0.1:$MCPG_PORT:$MCPG_PORT" \
+  --add-host "host.docker.internal:$SAFE_OUTPUTS_BIND_ADDRESS" \
   -v /var/run/docker.sock:/var/run/docker.sock \
   --entrypoint /app/awmg \
-  ghcr.io/github/gh-aw-mcpg:v0.3.12 \
+  ghcr.io/github/gh-aw-mcpg:v0.4.1 \
   --routed --listen "0.0.0.0:$MCPG_PORT" --config-stdin \
   < "$WORK_DIR/mcpg-config.json" \
   > "$WORK_DIR/gateway-output.json" 2>"$WORK_DIR/mcpg-stderr.log" &
@@ -118,7 +145,7 @@ cat > "$WORK_DIR/mcp-config.json" <<EOF
   "mcpServers": {
     "safeoutputs": {
       "type": "http",
-      "url": "http://127.0.0.1:$SO_PORT/mcp",
+      "url": "http://$SAFE_OUTPUTS_BIND_ADDRESS:$SO_PORT/mcp",
       "headers": {
         "Authorization": "Bearer $SO_API_KEY"
       },
@@ -186,7 +213,7 @@ cargo run -- execute \
 kill "$SO_PID" 2>/dev/null
 
 # Stop MCPG (if started)
-docker stop ado-aw-mcpg 2>/dev/null
+docker stop awmg-mcpg 2>/dev/null
 
 echo "Done. Output files in: $WORK_DIR"
 ```

docs/mcp.md

@@ -99,4 +99,5 @@ network:
 2. **Containerization**: Stdio MCP servers run as isolated Docker containers (per MCPG spec §3.2.1)
 3. **Environment Isolation**: MCP containers are spawned by MCPG with only the configured environment variables
 4. **MCPG Gateway**: All MCP traffic flows through the MCP Gateway which enforces tool-level filtering
-5. **Network Isolation**: MCP containers run within the same AWF-isolated network. Users must explicitly allow external domains via `network.allowed`
+5. **Trusted egress**: MCPG and the stdio/HTTP backends it spawns from `mcp-servers:` front matter are trusted infrastructure that runs outside the agent's Squid-enforced allowlist — they have direct network egress and are not subject to `network.allowed`/`network.blocked`. Only the Copilot agent process itself is confined to the AWF sandbox and its domain allowlist; see [`docs/mcpg.md`](mcpg.md) and [`docs/network.md`](network.md) for the topology.
+6. **SafeOutputs is further hardened**: unlike arbitrary `mcp-servers:` entries, the compiler-owned `safeoutputs` MCPG backend is not a user-configurable trusted-egress container — it is a dedicated stdio child spawned by MCPG from the pinned AWF `agent` image with `--network none`, `--cap-drop ALL`, a read-only rootfs, and the host ADO runner's non-root UID/GID. It has no network access at all, trusted or otherwise; see [`docs/mcpg.md`](mcpg.md).