Commit 1cddfb3
feat: swap to using aw-mcpg (#19)
* refactor: remove custom MCP firewall, metadata, and built-in MCP concept
Remove the custom MCP firewall proxy (src/mcp_firewall.rs), embedded tool
metadata (src/mcp_metadata.rs, mcp-metadata.json), and all firewall tests.
The copilot CLI no longer has built-in MCPs, so:
- Replace BUILTIN_MCPS constant with is_custom_mcp() helper
- Remove --disable-builtin-mcps, --disable-mcp-server, --mcp from copilot params
- Remove mcp-firewall CLI subcommand
- Simplify create wizard to MCP-level selection (no tool-level metadata)
- Update 1ES compiler to use is_custom_mcp() check
- Remove terminal_size dependency (only used by deleted MCP tool selector)
All MCPs are now handled through the MCP Gateway (MCPG) instead of
the custom firewall proxy.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add MCP Gateway (MCPG) support with SafeOutputs HTTP server
Add infrastructure for the gh-aw-mcpg gateway integration:
- Add MCPG_VERSION and MCPG_IMAGE constants in common.rs
- Add {{ mcpg_version }} and {{ mcpg_image }} template markers
- Replace generate_firewall_config() with generate_mcpg_config() that
produces MCPG-compatible JSON (mcpServers + gateway sections)
- SafeOutputs always included as HTTP backend via host.docker.internal
- Custom MCPs (with command:) become stdio servers in MCPG config
- Add host.docker.internal to CORE_ALLOWED_HOSTS for AWF container
to reach host-side MCPG
- Add mcp-http subcommand: serves SafeOutputs over HTTP using rmcp's
StreamableHttpService with axum, API key auth, and health endpoint
- Add axum and rmcp transport-streamable-http-server dependencies
- Remove terminal_size dependency (no longer used)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: rewrite base.yml template for MCPG integration
Replace the legacy MCP firewall pipeline steps with MCPG architecture:
- Replace 'Prepare MCP firewall config' with 'Prepare MCPG config'
- Replace stdio MCP config (safeoutputs + mcp-firewall) with HTTP
config pointing copilot to MCPG via host.docker.internal
- Add 'Start SafeOutputs HTTP server' step (background process on host)
- Add 'Start MCP Gateway (MCPG)' step (Docker container on host network)
- Add 'Stop MCPG and SafeOutputs' cleanup step (condition: always)
- Add --enable-host-access to AWF invocation for container-to-host access
- Pre-pull MCPG Docker image alongside AWF images
- Update step display names and comments
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test: update compiler tests for MCPG migration
- Add assertions for MCPG Docker image reference in compiled output
- Add assertions for host.docker.internal and --enable-host-access
- Add assertions verifying no legacy mcp-firewall references
- Add template structure checks for mcpg_config, mcpg_image, mcpg_version markers
- Verify template no longer contains mcp-firewall-config or MCP_FIREWALL_EOF
- Update fixture test to not require built-in MCP assertions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: update AGENTS.md for MCPG migration
- Update architecture file tree (remove mcp_firewall.rs, mcp_metadata.rs, mcp-metadata.json)
- Replace {{ firewall_config }} marker docs with {{ mcpg_config }}
- Add {{ mcpg_version }} and {{ mcpg_image }} marker documentation
- Update {{ agency_params }} to remove MCP-related flags
- Replace mcp-firewall CLI docs with mcp-http subcommand
- Remove 'Built-in MCP Servers' section (no built-in MCPs exist)
- Simplify MCP Configuration to custom servers only
- Add host.docker.internal to allowed domains table
- Replace entire MCP Firewall section with MCP Gateway (MCPG) docs
- Update standalone target description for MCPG
- Update front matter example to remove built-in MCP references
- Add gh-aw-mcpg and gh-aw-firewall to References
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address review feedback on MCPG integration
- Always call generate_mcpg_config() regardless of mcp_servers being
empty — safeoutputs must always be present in the MCPG config
- Pre-initialize SafeOutputs outside the StreamableHttpService factory
closure to avoid block_on() panic on a Tokio worker thread
- Add failure guards to readiness wait loops in base.yml: both
SafeOutputs and MCPG now fail the pipeline step explicitly if they
don't become ready within 30 seconds
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: harden security per PR review feedback
- Move host.docker.internal from CORE_ALLOWED_HOSTS to standalone
compiler's generate_allowed_domains — only pipelines using MCPG
need host access from the AWF container
- Replace weak time-based API key fallback with /dev/urandom (32
bytes); time-based value retained only as last-resort if urandom
is unavailable
- Remove SAFE_OUTPUTS_API_KEY println to avoid leaking the secret
to log files; the pipeline already knows the key it generated
- Add security comment explaining why Docker socket mount is
required for MCPG (spawns stdio MCP servers as sibling containers)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: improve code quality per PR review feedback
- Extract MCPG_PORT constant alongside MCPG_VERSION/MCPG_IMAGE to
avoid hardcoded port 80 in generated config
- Propagate MCPG config serialization error with ? instead of
silently falling back to broken JSON missing the gateway section
- Omit empty args array from MCPG config for cleaner output
(consistent with existing env/tools handling)
- Add warning in 1ES compiler when non-custom MCPs fall back to
convention-based service connection names that may not exist
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address round 3 PR review feedback
- Guard reserved 'safeoutputs' name in generate_mcpg_config to prevent
user-defined MCPs from overwriting the safe outputs HTTP backend
- Log MCPG config template before API key substitution to avoid leaking
MCP_GATEWAY_API_KEY to pipeline logs (ADO secret masking only applies
in subsequent steps, not the current step's output)
- Clarify allowed_hosts.rs comment: host.docker.internal is always added
for standalone pipelines (not conditionally)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: harden SafeOutputs HTTP server security
- Use subtle::ConstantTimeEq for API key comparison to prevent
timing side-channel attacks from a compromised AWF container
- Fail loudly (panic) when /dev/urandom is unavailable instead of
silently generating a weak time-based key
- Add doc comment on SafeOutputs Clone confirming concurrent safety:
only contains immutable PathBuf fields, file I/O opens fresh
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: improve MCPG reliability and diagnostics
- Pre-cleanup stale mcpg container before docker run to prevent
retry failures when a previous run was interrupted (OOM/SIGKILL
leaves container behind despite --rm)
- Differentiate duplicate warning messages for MCPs without commands:
options-but-no-command vs boolean-enabled (helps users understand
migration path from removed built-in MCPs)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: improve SafeOutputs HTTP server robustness
- Replace panic! with anyhow bail via .context() for /dev/urandom
failure — panics in async contexts crash the Tokio task instead of
propagating cleanly through the Result chain
- Bind to 127.0.0.1 instead of 0.0.0.0 — MCPG runs with --network
host so localhost is sufficient, reducing exposure on shared agents
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: correct MCPG networking for Linux host mode
- SafeOutputs URL in MCPG config changed from host.docker.internal
to localhost — MCPG runs with --network host on Linux where
host.docker.internal is not auto-injected in host network mode
- Gateway domain remains host.docker.internal (used by AWF container
which runs in bridge network mode with --enable-host-access)
- Add docker rm -f pre-cleanup before MCPG container start to handle
interrupted retry scenarios
- Differentiate MCP warning messages for options-without-command vs
boolean-enabled formats
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: add MCPG client auth and generate API key early
MCPG enforces authentication on incoming client requests. The agent's
mcp-config.json now includes an Authorization header with the gateway
API key. To support this, the MCP_GATEWAY_API_KEY is generated in the
'Prepare MCPG config' step (as an ADO secret variable) instead of the
'Start MCP Gateway' step, making it available when writing both the
MCPG server config and the agent's client config.
Also updates MCPG startup step to reference the ADO variable via
$(MCP_GATEWAY_API_KEY) instead of the shell-local variable.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: minor code quality improvements from review
- Replace Response::builder().unwrap() with IntoResponse tuple for
the 401 response in auth middleware (avoids unwrap on user path)
- Case-insensitive safeoutputs name reservation check to prevent
collision via 'SafeOutputs' or 'SAFEOUTPUTS' variants
- Fix typo: 'required' → 'require' in user-visible warning message
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: inline MCPG image URL in template
The MCPG image name (ghcr.io/github/gh-aw-mcpg) is a static string that
belongs in the template, not compiled into the binary. Only the version
number needs to be a compiled-in constant for template replacement.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: pipeline template and docs fixes for SafeOutputs/MCPG
- Add mkdir -p for staging/logs before SafeOutputs starts (prevents
nohup redirect failure on every pipeline run)
- Fix AGENTS.md SafeOutputs URL: host.docker.internal -> localhost
(MCPG runs with --network host, so localhost is correct)
- Remove unused MCPG_PID pipeline variable (cleanup uses docker stop
by container name)
- Add unit test for safeoutputs reserved-name collision guard
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add tests for mcpg
* Merge main
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 4ba6706 commit 1cddfb3
18 files changed
Lines changed: 1919 additions & 5818 deletions
Large diffs are not rendered by default.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
17 | 21 | | |
18 | | - | |
| 22 | + | |
19 | 23 | | |
20 | 24 | | |
21 | 25 | | |
| |||
24 | 28 | | |
25 | 29 | | |
26 | 30 | | |
| 31 | + | |
| 32 | + | |
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | | - | |
8 | 8 | | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| |||
303 | 304 | | |
304 | 305 | | |
305 | 306 | | |
306 | | - | |
307 | | - | |
308 | | - | |
309 | | - | |
310 | 307 | | |
311 | 308 | | |
312 | 309 | | |
| |||
340 | 337 | | |
341 | 338 | | |
342 | 339 | | |
343 | | - | |
344 | | - | |
345 | | - | |
346 | | - | |
347 | 340 | | |
348 | 341 | | |
349 | 342 | | |
| |||
506 | 499 | | |
507 | 500 | | |
508 | 501 | | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
509 | 510 | | |
510 | 511 | | |
511 | 512 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
123 | | - | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
124 | 127 | | |
125 | 128 | | |
126 | 129 | | |
127 | | - | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
128 | 134 | | |
129 | 135 | | |
130 | | - | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
131 | 140 | | |
132 | 141 | | |
133 | 142 | | |
134 | | - | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
135 | 147 | | |
136 | 148 | | |
137 | 149 | | |
| |||
222 | 234 | | |
223 | 235 | | |
224 | 236 | | |
225 | | - | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
226 | 241 | | |
227 | 242 | | |
228 | 243 | | |
229 | 244 | | |
230 | 245 | | |
231 | 246 | | |
232 | | - | |
| 247 | + | |
233 | 248 | | |
234 | 249 | | |
235 | | - | |
| 250 | + | |
236 | 251 | | |
237 | 252 | | |
238 | 253 | | |
239 | | - | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
240 | 266 | | |
241 | 267 | | |
242 | | - | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
243 | 278 | | |
244 | 279 | | |
245 | 280 | | |
| |||
0 commit comments