Skip to content

Commit 11b01ec

Browse files
fix(run): base64-encode PAT for ADO MCP and pass prompts via @file (#270)
* feat: generate mcp-config.json from MCPG runtime output instead of compile time Replaces the compile-time generate_mcp_client_config() approach with runtime conversion of MCPG's actual gateway output, matching gh-aw's convert_gateway_config_copilot.cjs pattern. Changes: - Remove generate_mcp_client_config() and {{ mcp_client_config }} template marker - MCPG stdout is now captured to gateway-output.json - After health check, poll until gateway output contains valid JSON - Use jq to transform URLs (127.0.0.1 -> host.docker.internal) and add tools: [*] - Apply same changes to both standalone and 1ES templates - Update AGENTS.md documentation This ensures the Copilot CLI config reflects MCPG's actual runtime state rather than a compile-time prediction, fixing agents not seeing configured MCPs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: create /tmp/gh-aw/mcp-logs before redirecting MCPG stderr The stderr redirect to /tmp/gh-aw/mcp-logs/stderr.log failed because the directory didn't exist yet. Add it to the mkdir -p call alongside the gateway output directory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: stream MCPG stderr to pipeline console via tee Use process substitution to tee stderr to both the log file and the pipeline console, giving real-time visibility into MCPG startup logs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: copy MCPG logs to published artifact directory Copy /tmp/gh-aw/mcp-logs/ into staging/logs/mcpg/ so MCPG gateway logs, stderr output, and gateway-output.json are available in the published pipeline artifact for post-run debugging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: pre-install Azure DevOps MCP to avoid MCPG startup timeout MCPG has a 30s startup timeout for stdio MCP containers. The previous approach used npx -y @azure-devops/mcp which downloads the package at container launch time, regularly exceeding this timeout. The AzureDevOpsExtension now emits a prepare_steps() that builds a local Docker image (ado-mcp-cached:latest) with @azure-devops/mcp pre-installed via npm install -g. The MCPG config references this cached image, so the container starts instantly without any network download. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add node ecosystem domains for Azure DevOps MCP container The ADO MCP container runs via npx which needs npm registry access to resolve @azure-devops/mcp. AWF's host-level iptables rules block outbound traffic from MCPG-spawned containers unless the domains are allowlisted. Add 'node' ecosystem identifier to AzureDevOpsExtension::required_hosts() so npm/node domains are included in AWF's allowed domains list. Also reverts the cached image approach (pre-install via docker commit) as allowing network access is the correct fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add MCPG debug logging and backend probe step Two changes to surface MCPG failures early: 1. Pass DEBUG=* to the MCPG container to enable full debug logging on stderr (streamed live via tee). This surfaces backend launch details, connection errors, and timeout diagnostics. 2. Add a 'Verify MCP backends' step after MCPG starts that probes each configured backend with a tools/list call. This forces MCPG's lazy initialization and catches failures (e.g., container timeout, network blocked) before the agent runs. Failures are surfaced as ADO pipeline warnings. Relates to #254 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: capture HTTP status and body in MCP backend probe The probe was using curl -sf which swallows error responses (exit 22 on 4xx). Now captures HTTP status code and response body separately so we can see what MCPG actually returns on failure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: pass MCPG API key via env mapping in probe step Secret variables set via ##vso[task.setvariable;issecret=true] must be explicitly mapped via env: to be available in bash steps. The probe was using \ macro syntax in the script body, but bash interprets \ as command substitution before ADO can expand it. Pass the key via env: mapping as MCPG_API_KEY and reference it as \ in the script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use raw API key for MCPG auth (not Bearer scheme) MCPG spec 7.1 requires the API key directly in the Authorization header, not as a Bearer token. The probe was sending 'Authorization: Bearer <key>' but MCPG expects 'Authorization: <key>'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: perform MCP initialize handshake before tools/list probe MCP requires an initialize handshake before any other method. The probe now sends initialize first, captures the Mcp-Session-Id from the response header, then sends tools/list with that session ID. Also parses SSE data lines to extract the tool count. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add --network host to ADO MCP container for AWF compatibility AWF's DOCKER-USER iptables rules block outbound traffic from containers on Docker's default bridge network. The ADO MCP container (spawned by MCPG as a sibling container) was unable to reach dev.azure.com, causing MCP error -32001 (request timed out) on every tool call. Adding --network host via the args field bypasses the FORWARD chain rules, matching gh-aw's approach for its built-in agentic-workflows MCP server (see mcp_config_builtin.go). SafeOutputs was unaffected because it runs as an HTTP backend on localhost — no FORWARD chain involved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: guard MCPG debug diagnostics behind --debug-pipeline flag Move debug/diagnostic pipeline additions behind a compile-time flag (--debug-pipeline, debug builds only) matching the --skip-integrity pattern: - MCPG DEBUG=* env var and stderr tee → {{ mcpg_debug_flags }} marker - MCP backend probe step → {{ verify_mcp_backends }} marker When --debug-pipeline is not passed (the default), both markers resolve to empty strings and the generated pipeline contains no debug diagnostics. MCPG log artifact collection is kept always-on as it is low-cost and useful in production. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add DEBUG=* env to ADO MCP container for diagnostics The ADO MCP container still times out on API calls despite --network host. Adding DEBUG=* enables verbose logging from the @azure-devops/mcp npm package to surface the root cause (auth issues, DNS, proxy, etc.). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: add integration tests for --skip-integrity and --debug-pipeline Add 12 new compiler integration tests covering: - --skip-integrity: omits integrity step, produces valid YAML (both targets) - --debug-pipeline: includes DEBUG env, stderr tee, probe step, produces valid YAML (both targets) - Combined flags work together - Default (no flags) excludes debug content and includes integrity step - Probe step indentation is correct for both standalone (8sp) and 1ES (18sp) Also fix replacement content indentation: replace_with_indent adds the marker's indent to continuation lines, so replacement content must have zero base indent. Also refactor compile_fixture into compile_fixture_with_flags with atomic counter for unique temp dirs to prevent parallel test collisions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: resolve debug step replacement ordering and add marker test Debug replacements must run BEFORE extra_replacements so that {{ mcpg_port }} inside the probe step content gets resolved by the subsequent extra_replacements pass. Previously, debug replacements ran after extra_replacements, leaving {{ mcpg_port }} unresolved. Add test_debug_pipeline_no_unresolved_markers to catch this class of ordering bugs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: fix docker run line continuation and simplify debug markers Fix bash line continuation in the MCPG docker run command: - mcpg_debug_flags emits backslash when disabled (maintains line continuation) and -e DEBUG env flag when enabled - Stderr tee is always-on (baked into template), not debug-gated - Fix replacement ordering: debug before extra_replacements so mcpg_port in probe step content gets resolved - Drop mcpg_stderr_redirect marker (back to 2 markers total) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: generic pipeline variable injection for MCP extensions Replace the ADO-specific special case in generate_mcpg_docker_env with a generic mechanism driven by the CompilerExtension trait. New trait method: required_pipeline_vars() -> Vec<PipelineEnvMapping> - Extensions declare which container env vars map to pipeline variables - AzureDevOpsExtension maps AZURE_DEVOPS_EXT_PAT -> SC_READ_TOKEN The compiler generates the full chain generically: 1. env: block on MCPG step (ADO secret -> bash var) 2. -e flag on MCPG docker run (bash var -> MCPG process env) 3. MCPG config keeps empty string (MCPG passthrough to child) ADO secrets require env: mapping because Azure DevOps does not expand secret variables via macro syntax in script bodies. Also updates azure-devops-mcp-agent fixture to use first-class tools.azure-devops instead of manual mcp-servers configuration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use correct auth for @azure-devops/mcp (envvar + ADO_MCP_AUTH_TOKEN) The @azure-devops/mcp npm package does NOT use AZURE_DEVOPS_EXT_PAT (that's for the az devops CLI extension). It accepts auth type via CLI arg (-a) and reads the token from a specific env var per type. For pipeline use with bearer tokens from az account get-access-token: - Add '-a envvar' to entrypoint args (selects envvar auth mode) - Map ADO_MCP_AUTH_TOKEN (not AZURE_DEVOPS_EXT_PAT) to SC_READ_TOKEN - The ADO MCP reads the bearer token from ADO_MCP_AUTH_TOKEN Auth types in @azure-devops/mcp (src/auth.ts): - 'pat': reads PERSONAL_ACCESS_TOKEN (base64 PAT) - 'envvar': reads ADO_MCP_AUTH_TOKEN (bearer token) ← we use this - 'azcli'/'env': uses DefaultAzureCredential - 'interactive': OAuth browser flow (default, breaks stdio) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(run): fix MCPG local development issues - Use dynamic high port for MCPG instead of hardcoded port 80 (requires admin privileges on most systems) - Platform-aware Docker networking: --network host on Linux, -p port mapping on Windows/macOS where Docker Desktop uses a VM - Rewrite SafeOutputs URL to host.docker.internal on Windows/macOS so MCPG container can reach host services - Derive MCP client config from MCPG runtime gateway output instead of compile-time prediction (matches pipeline behavior) - Add AdoAuthMode enum to AzureDevOpsExtension: Bearer for pipelines (ADO_MCP_AUTH_TOKEN), Pat for local dev (PERSONAL_ACCESS_TOKEN) - Remove stale generate_mcp_client_config re-export (removed on branch) - Update --pat help text to describe MCPG token plumbing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(run): add MCPG and Copilot CLI debug capabilities to local run mode - Capture MCPG stderr: inherit to terminal in debug mode (-d), redirect to stderr.log in normal mode (previously discarded via Stdio::null) - Pass DEBUG=* env to MCPG container when -d flag is set, enabling full namespace-based debug logging ([LAUNCHER] output, [stderr] from spawned containers, protocol diagnostics) - Remove DEBUG=* from ADO MCP server config env — MCPG forwards server env to child containers where it corrupts JSON-RPC stdio - Surface MCPG log file locations after startup with tail -f tip - Dump MCPG diagnostic logs (stderr.log, mcp-gateway.log, per-server logs) on startup failure for actionable error messages - Probe MCP backends in debug mode: send initialize + tools/list handshakes to each backend after gateway ready, catching broken MCPs before the agent runs - Pass --log-level debug and --log-dir to Copilot CLI when -d flag is set for verbose agent logging Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(run): base64-encode PAT for ADO MCP and pass prompts via @file - Encode raw PAT as base64(:<PAT>) for PERSONAL_ACCESS_TOKEN env var, matching the microsoft/azure-devops-mcp -a pat auth contract - Replace hardcoded debug prompt with @filepath reference to the agent-prompt.md file already written to disk - Update standalone and 1ES pipeline templates to use --prompt @/tmp/awf-tools/agent-prompt.md instead of --prompt \\\\, avoiding shell expansion issues Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address PR review feedback - Add missing #[test] attribute on test_debug_pipeline_probe_step_indentation_standalone - Fix pat doc comment: PERSONAL_ACCESS_TOKEN, not ADO_MCP_AUTH_TOKEN - Prevent dangling env: null on MCPG step when no extensions need pipeline vars — generate_mcpg_step_env now emits the full env: block or empty string Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: remove stray list_projects.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: cleanup duplicate comment, mcpg log dir, and extension API - Remove duplicate step 12 comment in compile_pipeline (common.rs) - Use output_dir for mcpg-logs in both debug and non-debug modes, preventing stray directories in the user's working directory - Collapse collect_extensions_with_ado_auth one-liner wrapper into collect_extensions_with_auth, eliminating the intermediate collect_extensions_with_options private function Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent d20ae25 commit 11b01ec

12 files changed

Lines changed: 1290 additions & 410 deletions

File tree

AGENTS.md

Lines changed: 23 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -739,6 +739,18 @@ Generates the "Verify pipeline integrity" pipeline step that downloads the relea
739739

740740
When the compiler is built with `--skip-integrity` (debug builds only), this placeholder is replaced with an empty string and the integrity step is omitted from the generated pipeline.
741741

742+
## {{ mcpg_debug_flags }}
743+
744+
Generates MCPG debug environment flags for the Docker run command. When `--debug-pipeline` is passed (debug builds only), this inserts `-e DEBUG="*"` to enable verbose MCPG logging.
745+
746+
When `--debug-pipeline` is not passed, this placeholder is replaced with a bare `\` to maintain bash line continuation.
747+
748+
## {{ verify_mcp_backends }}
749+
750+
Generates a pipeline step that probes each configured MCPG backend with an MCP initialize + tools/list handshake. This forces MCPG's lazy initialization and catches failures (e.g., container timeout, network blocked) before the agent runs, surfacing them as ADO pipeline warnings.
751+
752+
When `--debug-pipeline` is not passed (the default), this placeholder is replaced with an empty string.
753+
742754
## {{ pr_trigger }}
743755

744756
Generates PR trigger configuration. When a schedule or pipeline trigger is configured, this generates `pr: none` to disable PR triggers. Otherwise, it generates an empty string, allowing the default PR trigger behavior.
@@ -795,41 +807,18 @@ If no passthrough env vars are needed, this marker is replaced with an empty str
795807

796808
## {{ mcp_client_config }}
797809

798-
Should be replaced with the Copilot CLI `mcp-config.json` content, generated at compile time from the MCPG server configuration. This follows gh-aw's pattern where `convert_gateway_config_copilot.cjs` produces per-server routed URLs.
810+
**Removed.** The Copilot CLI `mcp-config.json` is no longer generated at compile time. Instead, it is derived at **pipeline runtime** from MCPG's actual gateway output, matching gh-aw's `convert_gateway_config_copilot.cjs` pattern.
799811

800-
MCPG runs in routed mode by default, exposing each backend at `/mcp/{serverID}`. The generated JSON lists one entry per MCPG-managed server with:
801-
- `type: "http"` — Copilot CLI HTTP transport
802-
- `url` — routed endpoint (`http://host.docker.internal:{port}/mcp/{name}`)
803-
- `headers` — Bearer auth with the gateway API key (ADO variable `$(MCP_GATEWAY_API_KEY)`)
804-
- `tools: ["*"]` — allow all tools (Copilot CLI requirement)
812+
The "Start MCP Gateway (MCPG)" pipeline step:
813+
1. Redirects MCPG's stdout to `gateway-output.json`
814+
2. Waits for the health check and for valid JSON output
815+
3. Transforms the output with a Python script that:
816+
- Rewrites URLs from `127.0.0.1` → `host.docker.internal` (AWF container loopback vs host)
817+
- Ensures `tools: ["*"]` on each server entry (Copilot CLI requirement)
818+
- Preserves all other fields (headers, type, etc.)
819+
4. Writes the result to `/tmp/awf-tools/mcp-config.json` and `$HOME/.copilot/mcp-config.json`
805820

806-
**Variable expansion note:** The `$(MCP_GATEWAY_API_KEY)` token in the generated JSON uses ADO macro syntax. Although the JSON is written to disk via a quoted bash heredoc (`<< 'EOF'`), which prevents bash `$(...)` command substitution, ADO macro expansion processes the entire script body *before* bash executes it. The real API key value is therefore substituted by ADO at pipeline runtime, and the file on disk contains the actual secret — Copilot CLI does not need to perform any variable expansion.
807-
808-
Server names are validated for URL path safety (no `/`, `#`, `?`, `%`, or spaces). Server entries are sorted alphabetically for deterministic output.
809-
810-
Example output:
811-
```json
812-
{
813-
"mcpServers": {
814-
"azure-devops": {
815-
"type": "http",
816-
"url": "http://host.docker.internal:80/mcp/azure-devops",
817-
"headers": {
818-
"Authorization": "Bearer $(MCP_GATEWAY_API_KEY)"
819-
},
820-
"tools": ["*"]
821-
},
822-
"safeoutputs": {
823-
"type": "http",
824-
"url": "http://host.docker.internal:80/mcp/safeoutputs",
825-
"headers": {
826-
"Authorization": "Bearer $(MCP_GATEWAY_API_KEY)"
827-
},
828-
"tools": ["*"]
829-
}
830-
}
831-
}
832-
```
821+
This ensures the Copilot CLI config reflects MCPG's actual runtime state rather than a compile-time prediction.
833822

834823
## {{ allowed_domains }}
835824

@@ -986,6 +975,7 @@ Global flags (apply to all subcommands): `--verbose, -v` (enable info-level logg
986975
- `compile [<path>]` - Compile a markdown file to Azure DevOps pipeline YAML. If no path is given, auto-discovers and recompiles all detected agentic pipelines in the current directory.
987976
- `--output, -o <path>` - Optional output path for generated YAML (only valid when a path is provided)
988977
- `--skip-integrity` - *(debug builds only)* Omit the "Verify pipeline integrity" step from the generated pipeline. Useful during local development when the compiled output won't match a released compiler version. This flag is not available in release builds.
978+
- `--debug-pipeline` - *(debug builds only)* Include MCPG debug diagnostics in the generated pipeline: `DEBUG=*` environment variable for verbose MCPG logging, stderr streaming to log files, and a "Verify MCP backends" step that probes each backend with MCP initialize + tools/list before the agent runs. This flag is not available in release builds.
989979
- `check <pipeline>` - Verify that a compiled pipeline matches its source markdown
990980
- `<pipeline>` - Path to the pipeline YAML file to verify
991981
- The source markdown path is auto-detected from the `@ado-aw` header in the pipeline file

0 commit comments

Comments
 (0)