Commit bf34af3
feat(audit): add
Single-run audit: download a build's artifacts, run every analyzer
(firewall, MCP gateway, OTel, safe outputs, detection verdict, build
timeline, missing tools/data/noops), and emit a Markdown or JSON
report. ADO-side counterpart to `gh aw audit`.
New module tree under `src/audit/`:
- `model.rs` — `AuditData` (drift-compatible with gh-aw's top-level
contract; adds ADO-specific `detection_analysis`,
`safe_output_execution`, `rejected_safe_outputs` sections).
- `url.rs` — parses bare IDs, dev.azure.com URLs, legacy
visualstudio.com URLs, and on-prem Azure DevOps Server URLs (with
optional `&j=`/`&t=`/`&s=` job/step anchors).
- `cache.rs` — CLI-version-keyed `run-summary.json` with atomic writes.
- `analyzers/{firewall,policy,mcp,otel,safe_outputs,detection,missing,jobs}.rs`
— eight defensive NDJSON/REST analyzers.
- `findings.rs` — eight heuristic rules emitting severity-rated
findings + recommendations.
- `render/{console,json}.rs` — two renderers; JSON shape is the
public contract.
- `cli.rs` — orchestration: URL parse → auth → metadata fetch →
artifact download → analyzers → findings → cache → render.
Unified rejection trace: when the aggregate `THREAT_DETECTION_RESULT`
has any threat flag set, every proposal lands in
`not_processed_due_to_aggregate_gate` carrying the aggregate
`reasons[]`, exactly one severity-`high` `KeyFinding` is emitted, and a
`rejected_safe_outputs` rollup appears at the top level.
Pipeline-side runtime additions (so an `ado-aw audit` of an existing
build has the data it needs):
- `src/data/*-base.yml` (via `AdoAwMarkerExtension`): emits
`staging/aw_info.json` at runtime with engine, model, agent name,
source path, target, compiler version, and ADO build context.
- `src/execute.rs`: writes a per-item `safe-outputs-executed.ndjson`
in `<output-dir>` so the audit can show the proposed → detection →
executed trace.
CLI surface:
ado-aw audit <build-id-or-url>
-o, --output <dir> # default ./logs
--json
--org / --project / --pat
--artifacts <agent,detection,safe-outputs>
--no-cache
New dependencies: `zip` (artifact unpack), `wiremock` (dev only —
integration test mock server).
Tests: 80 new audit unit tests + 3 integration tests against a fake
ADO REST server (happy path, permission-denied, cache hit) using a
thin `ADO_AW_TEST_ORG_URL` test seam. 1740 total tests pass.
Docs: new `docs/audit.md`; updates to `docs/cli.md`, `README.md`,
`AGENTS.md` index, and `prompts/debug-ado-agentic-workflow.md` (Step 1
first-move + new Step 2a-prime + `AuditData` reference + jq-diff
fallback).
Out of scope (explicit follow-ups): diff mode, cross-run trends,
`--parse` log.md/firewall.md, job/step-anchored audit, MCP-exposed
audit, per-item detection verdict (upstream coordination with gh-aw),
partial-approval gating, AWF policy-manifest plumbing, AWF
token-usage.jsonl, `audit-manifest.json` build inventory.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>ado-aw audit <build-id-or-url> command1 parent 9c500e3 commit bf34af3
33 files changed
Lines changed: 10183 additions & 191 deletions
File tree
- docs
- prompts
- src
- ado
- audit
- analyzers
- render
- compile/extensions
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
227 | 227 | | |
228 | 228 | | |
229 | 229 | | |
230 | | - | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
231 | 234 | | |
232 | 235 | | |
233 | 236 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| 42 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
496 | 496 | | |
497 | 497 | | |
498 | 498 | | |
| 499 | + | |
| 500 | + | |
499 | 501 | | |
500 | 502 | | |
501 | 503 | | |
| |||
513 | 515 | | |
514 | 516 | | |
515 | 517 | | |
| 518 | + | |
516 | 519 | | |
517 | 520 | | |
518 | 521 | | |
| |||
0 commit comments