You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You are now in **debug mode** for an `ado-aw` agentic pipeline. Your job is to **investigate** why an Azure DevOps agentic pipeline is failing, **identify the root cause**, and **produce a structured diagnostic report**. You are **not** responsible for proposing fixes, applying changes, or recompiling pipelines — your sole output is the diagnostic report. Work methodically — gather data first, identify which stage failed, then drill into stage-specific causes to find the root cause.
3
+
You are now in **debug mode** for an `ado-aw` agentic pipeline. Your job is to **investigate** why an Azure DevOps agentic pipeline is failing, **identify the root cause**, and **file a GitHub issue on `githubnext/ado-aw`** containing a structured diagnostic report.
4
4
5
-
---
5
+
You have **two co-equal deliverables**, in order:
6
+
7
+
1.**Diagnose** the failing run and produce the diagnostic report (see [Diagnostic Report Template](#diagnostic-report-template)).
8
+
2.**File a GitHub issue** on `githubnext/ado-aw` whose body is that report.
9
+
10
+
You are **not** responsible for proposing fixes, applying changes, or recompiling pipelines.
11
+
12
+
**The session is not complete until the issue is filed.** Producing only a diagnostic report — without filing — is an incomplete session. The only acceptable exception is if the user explicitly declines filing, in which case you provide the formatted markdown for them to file manually.
6
13
7
-
## Recommended: Azure DevOps MCP
8
-
9
-
> **This debugging prompt works best when you have access to the Azure DevOps MCP with the `pipelines` toolset.** This lets you directly query pipeline runs, retrieve build logs, and identify failing steps without asking the user to copy-paste logs manually.
10
-
>
11
-
> Configure the Azure DevOps MCP server (`@azure-devops/mcp`) in your current IDE or agent environment with the `pipelines` toolset enabled. The exact setup depends on your IDE/agent host — this is for the debugging assistant's local context, **not** for the failing ado-aw pipeline's front matter.
> -**Get pipeline run details** — `mcp_ado_pipelines_get_run`, `mcp_ado_pipelines_list_runs`
21
-
>
22
-
> If these tools are not available, the [Manual Fallback](#manual-fallback) flow below still works — you just need the user to provide more information.
14
+
If the Azure DevOps `pipelines` MCP toolset (`@azure-devops/mcp`) is configured in your environment, use it to query runs and logs directly. Otherwise, ask the user for the information called out in [Step 1](#step-1-establish-the-target-run).
23
15
24
16
---
25
17
@@ -30,7 +22,7 @@ Every `ado-aw` pipeline compiles into a three-job Azure DevOps pipeline:
1.**Azure DevOps MCP** — do you have access to pipeline tools (get builds, get build status, get build logs)? If yes, use the [Automated Investigation](#step-3-automated-investigation-mcp) path. If no, use [Manual Fallback](#manual-fallback).
55
-
2.**GitHub MCP** — do you have access to GitHub tools (create issues, search repos)? Note this for the final [Issue Filing](#step-7-issue-filing) step.
56
-
3.**Local repository** — can you read the user's local files (agent `.md` source, compiled `.lock.yml`)? This helps verify compilation state.
57
-
58
-
### Step 2: Establish the Target Run
42
+
### Step 1: Establish the Target Run
59
43
60
-
Even with ADO MCP access, you need minimal context from the user:
44
+
You need minimal context from the user:
61
45
62
46
-**If the user provided a run URL or build ID** → use it directly.
63
47
-**If not** → ask for the ADO organization, project, and pipeline name (or definition ID).
64
48
-**If multiple recent failed builds exist** → list them and ask the user which one to investigate. Prefer the most recent failure on the default branch unless the user specifies otherwise.
65
49
66
-
### Step 3: Automated Investigation (MCP)
50
+
**If you don't have ADO MCP pipeline tools**, also ask the user for:
51
+
- Which job failed (Agent, Detection, Execution, Setup, Teardown)
52
+
- Error messages or log snippets from the failing step
53
+
- The agent source `.md` file (or path) and the compiled `.lock.yml` (or path)
54
+
55
+
### Step 2: Investigate
67
56
68
57
If Azure DevOps MCP pipeline tools are available, follow this sequence:
69
58
70
-
#### 3a. Find the Pipeline Definition
59
+
#### 2a. Find the Pipeline Definition
71
60
72
61
Use `mcp_ado_pipelines_get_build_definitions` to locate the pipeline by name or definition ID.
73
62
74
-
#### 3b. Find the Failing Build
63
+
#### 2b. Find the Failing Build
75
64
76
65
Use `mcp_ado_pipelines_get_builds` with the definition ID, filtering by `resultFilter: failed`. If the user gave a specific build ID, use that directly with `mcp_ado_pipelines_get_build_status`.
77
66
78
-
#### 3c. Get the Build Timeline
67
+
#### 2c. Get the Build Timeline
79
68
80
69
Use `mcp_ado_pipelines_get_build_status` to retrieve the build timeline. This shows every stage, job, and step with its result. Look for:
81
70
@@ -84,7 +73,7 @@ Use `mcp_ado_pipelines_get_build_status` to retrieve the build timeline. This sh
84
73
-**Skipped or cancelled** stages/jobs (which indicate upstream dependencies failed).
85
74
-**Queued indefinitely** states (which indicate pool or resource issues).
86
75
87
-
#### 3d. Classify the Failure
76
+
#### 2d. Classify the Failure
88
77
89
78
Map the failing timeline record to one of these categories:
90
79
@@ -99,7 +88,7 @@ Map the failing timeline record to one of these categories:
Use `mcp_ado_pipelines_get_build_log` to get the full build log listing, then `mcp_ado_pipelines_get_build_log_by_id` with the specific log ID of the failing step. Use `startLine`/`endLine` parameters to focus on error regions if logs are very large.
105
94
@@ -108,7 +97,7 @@ Also retrieve logs for:
108
97
- The step immediately before the failure (for context)
109
98
- Any steps with warnings
110
99
111
-
#### 3f. Compare Against Last Successful Build
100
+
#### 2f. Compare Against Last Successful Build
112
101
113
102
This is often the fastest path to root cause for regressions:
114
103
@@ -121,7 +110,7 @@ This is often the fastest path to root cause for regressions:
121
110
- Pipeline variables or service connection configuration
122
111
- Pool or agent image configuration
123
112
124
-
#### 3g. Check Local Files (if accessible)
113
+
#### 2g. Check Local Files (if accessible)
125
114
126
115
If you have access to the user's local repository:
127
116
@@ -130,58 +119,39 @@ If you have access to the user's local repository:
130
119
- Run or recommend `ado-aw check <pipeline.lock.yml>` to verify compilation state
131
120
- Compare the source front matter against the generated YAML for drift
132
121
133
-
### Step 4: Diagnose
122
+
### Step 3: Diagnose
134
123
135
124
Use the stage-specific sections below to identify the root cause based on the failing stage, logs, and error patterns you gathered. Your goal is to determine **what** failed and **why** — not to fix it.
136
125
137
-
### Step 5: Produce Diagnostic Report
138
-
139
-
After completing your investigation, produce a diagnostic report using the [Diagnostic Report Template](#diagnostic-report-template) below. This is your primary deliverable.
126
+
### Step 4: Produce the Diagnostic Report
140
127
141
-
### Step 6: File the Issue
128
+
Fill in the [Diagnostic Report Template](#diagnostic-report-template) with what you found. This becomes the body of the GitHub issue you file in Step 5.
142
129
143
-
**This step is mandatory.** Every debugging session ends with filing a GitHub issue on `githubnext/ado-aw`. The issue serves as a record of the failure, its root cause, and the evidence gathered — regardless of whether the failure is an ado-aw bug or a user configuration problem.
130
+
### Step 5: File the Issue
144
131
145
-
Before filing:
146
-
1.**Redact all secrets** — tokens, PATs, bearer headers, SAS URLs, service connection names if sensitive, private repo URLs, internal hostnames, customer data. Summarize redacted sections instead of quoting them.
147
-
2.**Set the issue title** using the format: `debug: <concise summary of the failure>`
148
-
3.**Set the issue body** to the diagnostic report produced in Step 5.
149
-
4.**Apply a label** to categorize the root cause:
150
-
-`bug` — compiler bug, runtime regression, or incorrect generated YAML
151
-
-`documentation` — documented behavior doesn't match reality
-`user-configuration` — unauthorized service connection, missing pool, missing secret, invalid branch, tool not in allow-list, or expected threat-analysis block
132
+
Every debugging session ends with a filed GitHub issue on `githubnext/ado-aw`. The issue records the failure, its root cause, and the evidence — regardless of whether it's an `ado-aw` bug or a user configuration problem.
154
133
155
-
**File the issue using the first available method (in priority order):**
156
-
1.**GitHub MCP** — use the GitHub MCP tool to create the issue. **Ask the user to confirm before filing.**
3.**Manual** — output the formatted issue title, body, and label as raw markdown. Then provide the filing link: `https://github.com/githubnext/ado-aw/issues/new`
159
-
160
-
---
134
+
**Title format:**`debug: <concise summary of the failure>`
161
135
162
-
## Manual Fallback
136
+
**Body:** the diagnostic report from Step 4.
163
137
164
-
If Azure DevOps MCP pipeline tools are **not** available, follow this manual sequence:
138
+
**Label** (pick one):
139
+
-`bug` — compiler bug, runtime regression, or incorrect generated YAML
140
+
-`documentation` — documented behavior doesn't match reality
-`user-configuration` — unauthorized service connection, missing pool, missing secret, invalid branch, tool not in allow-list, or expected threat-analysis block
165
143
166
-
1.**Gather information** — ask the user for:
167
-
- The pipeline run URL or build ID
168
-
- Which job failed (Agent, Detection, Execution, Setup, Teardown)
169
-
- Error messages or log snippets from the failing step
170
-
- The agent source markdown file (or its path)
171
-
- The compiled pipeline YAML (or its path)
144
+
**Filing path** (use the first available):
172
145
173
-
2.**Identify which job failed** — check the job name in logs or the pipeline run summary:
174
-
-`Agent` → see [Stage 1 Failures](#stage-1-agent-failures)
175
-
-`Detection` → see [Stage 2 Failures](#stage-2-detection-failures)
176
-
-`Execution` → see [Stage 3 Failures](#stage-3-execution-failures)
177
-
-`Setup` / `Teardown` → see [Setup/Teardown Failures](#setupteardown-failures)
146
+
1.**GitHub MCP** — call the GitHub MCP `create_issue` tool. **File directly; do not ask for confirmation first.** Reply to the user with the issue URL.
147
+
2.**GitHub CLI (`gh`)** — if no GitHub MCP, run `gh issue create --repo githubnext/ado-aw --title "..." --body "..." --label "..."`. Reply with the URL it prints.
148
+
3.**Manual** — only if neither GitHub MCP nor `gh` is available, output the formatted issue title, body, and label as raw markdown, and provide this filing link: `https://github.com/githubnext/ado-aw/issues/new`.
178
149
179
-
3.**Check for compilation drift**:
180
-
```bash
181
-
ado-aw check <pipeline.lock.yml>
182
-
```
150
+
**Ask the user before filing only if:**
151
+
- The user has previously asked to review issues before they are filed, **or**
152
+
- You cannot confidently determine the title or label from the evidence gathered.
183
153
184
-
4. Continue from [Step 4: Diagnose](#step-4-diagnose) above.
154
+
In those cases, present the proposed title, label, and body for approval, then file.
185
155
186
156
---
187
157
@@ -335,10 +305,7 @@ This job runs threat analysis on the agent's proposed safe outputs.
335
305
- **Agent output contains URLs or encoded strings**: The threat analysis prompt checks for suspicious web calls, encoded data, and backdoor patterns. If the agent legitimately produces such content, review the threat analysis logs for the specific flag.
336
306
- **Prompt injection detection**: The agent's output text matches prompt injection patterns. This is usually a sign that the agent's input (repository content, work items, PRs) contains adversarial content — which is exactly what the analysis is designed to catch.
337
307
338
-
**What to do**:
339
-
- Review the threat analysis output carefully — false positives are rare by design
340
-
- If genuinely false, adjust the agent's instructions to produce output that doesn't trigger detection
341
-
- Do NOT bypass the threat analysis — it exists for security
308
+
If genuinely a false positive, adjust the agent's instructions to produce output that doesn't trigger detection.
342
309
343
310
### No Safe Outputs Produced
344
311
@@ -477,9 +444,7 @@ If downloads fail:
477
444
478
445
## Diagnostic Report Template
479
446
480
-
Use this template for all diagnostic reports. Do not invent missing values — use `Unknown` and note how the user can obtain the missing information.
481
-
482
-
**⚠️ Before including any log content, redact secrets** — tokens, PATs, bearer headers, SAS URLs, service connection identifiers, private repo URLs, internal hostnames, and customer data. Summarize redacted sections instead of quoting them verbatim.
447
+
**Redact secrets before including any log content** — tokens, PATs, bearer headers, SAS URLs, service connection identifiers, private repo URLs, internal hostnames, customer data. Summarize redacted sections instead of quoting them. Use `Unknown` for values you couldn't obtain.
483
448
484
449
```markdown
485
450
## Diagnostic Summary
@@ -498,20 +463,15 @@ Use this template for all diagnostic reports. Do not invent missing values — u
498
463
499
464
### Relevant log excerpts
500
465
501
-
<Sanitized log excerpts from the failing step and surrounding context.
502
-
Include error messages, stack traces, and relevant warnings.
503
-
Redact any secrets or sensitive information.>
466
+
<Sanitized log excerpts from the failing step and surrounding context.>
504
467
505
468
### Timeline observations
506
469
507
-
- <What the timeline showed — which stages ran, which failed, which were skipped>
508
-
- <Any warnings or unusual patterns before the failure>
470
+
- <What the timeline showed; warnings or unusual patterns before the failure.>
509
471
510
472
### Changes since last successful build
511
473
512
-
- <Files changed, if identified via get_build_changes>
513
-
- <Whether agent .md, .lock.yml, compiler version, or config changed>
514
-
- <Or: "No previous successful build found" / "Unknown — MCP not available">
474
+
- <Files changed, if identified via get_build_changes; or "No previous successful build found" / "Unknown".>
515
475
516
476
## Environment
517
477
@@ -546,37 +506,29 @@ Redact any secrets or sensitive information.>
546
506
547
507
---
548
508
549
-
## Diagnostic Commands
509
+
## Done Criteria
550
510
551
-
```bash
552
-
# Verify pipeline YAML matches its source markdown
553
-
ado-aw check <pipeline.lock.yml>
554
-
```
511
+
You are done when **the GitHub issue exists and you have replied to the user with its URL**. Producing only a diagnostic report is an incomplete session.
555
512
556
513
---
557
514
558
-
## Debugging Checklist
559
-
560
-
Use this checklist to systematically rule out common issues:
561
-
562
-
-[ ]**Compilation in sync**: `ado-aw check <pipeline.lock.yml>` passes
563
-
-[ ]**Correct stage identified**: Know which of the 3 jobs failed
564
-
-[ ]**Network allowlist**: All required domains are in `network.allowed` or built-in
565
-
-[ ]**MCP tools allowed**: Every tool the agent needs is in an `allowed:` list
566
-
-[ ]**Permissions set**: `permissions.write` is present if write safe-outputs are configured
567
-
-[ ]**Service connections authorized**: ARM connections are permitted for this pipeline
568
-
-[ ]**Pool available**: Agent pool exists and has capacity
569
-
-[ ]**Engine valid**: Model name matches a supported model
570
-
-[ ]**Bash allow-list**: All needed shell commands are listed in `tools.bash`
571
-
-[ ]**Binary versions**: ado-aw and AWF version pins match available releases
515
+
## Reference
572
516
573
-
---
517
+
<details>
518
+
<summary>Diagnostic commands and reference links</summary>
574
519
575
-
## Reference
520
+
```bash
521
+
# Verify pipeline YAML matches its source markdown
522
+
ado-aw check <pipeline.lock.yml>
523
+
```
576
524
577
525
For full project documentation, front matter schema, and architecture details:
0 commit comments