Skip to content

Commit 0b6ec30

Browse files
authored
Improve active learning density and add prerequisites to audit reference side quest (#1784)
1 parent 7ce0449 commit 0b6ec30

1 file changed

Lines changed: 67 additions & 4 deletions

File tree

workshop/side-quest-25-01-audit-reference.md

Lines changed: 67 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@
44

55
> _A detailed companion to [Audit and Monitor Your Agentic Workflows](25-audit-and-observability.md). Use this side quest when you want to understand the full contents of an audit report or dig into individual artifact files._
66
7+
## 📋 Before You Start
8+
9+
- You completed [Audit and Monitor Your Agentic Workflows](25-audit-and-observability.md) and have at least one workflow run ID to work with.
10+
- `gh aw` is installed and authenticated (see [Install the gh-aw CLI Extension](06-install-gh-aw.md)).
11+
712
## [gh aw audit](https://github.github.com/gh-aw/reference/audit/#gh-aw-audit) report anatomy
813

914
`gh aw audit` generates a Markdown report that covers:
@@ -28,9 +33,9 @@ The `agent` artifact — downloaded by both `gh aw logs --artifacts all` and `gh
2833
| `sandbox/firewall/audit/` | Domain-level network access log (raw data) |
2934
| `agent_usage.json` | Token usage for the agent turn |
3035

31-
### Parsed log files (--parse)
36+
### Readable log files
3237

33-
When you run `gh aw audit <run-id> --parse`, two readable files are written alongside the raw artifacts:
38+
The audit report is accompanied by readable files written alongside the raw artifacts:
3439

3540
- `log.md` — the full agent conversation formatted as Markdown
3641
- `firewall.md` — a formatted summary of outbound network access (allowed and blocked domains)
@@ -55,12 +60,70 @@ network:
5560
5661
Share the allowed-domains list from a successful run with your enterprise security team as a ready-made firewall allowlist.
5762
63+
## Try it yourself
64+
65+
### Run an audit on a recent run
66+
67+
Open the **Actions** tab in your repository, click a completed workflow run, and copy the run ID from the URL (the number after `/runs/`). Then run:
68+
69+
```bash
70+
gh aw audit <run-id>
71+
```
72+
73+
Sample output:
74+
75+
```text
76+
## Audit Report
77+
78+
**Workflow:** daily-status
79+
**Trigger:** schedule
80+
**Engine:** copilot
81+
**Model:** gpt-4o
82+
83+
| Metric | Value |
84+
|---|---|
85+
| Agent AIC | 42 |
86+
| ⌖ AIC | 3 |
87+
| MCP calls | 7 |
88+
| Threat verdict | none |
89+
```
90+
91+
- [ ] I found a run ID from the Actions tab
92+
- [ ] The report shows the workflow name, trigger, and model
93+
- [ ] The ⌖ AIC figure appears separately from Agent AIC
94+
- [ ] The threat verdict shows `none` (or I noted what was flagged)
95+
96+
### Explore MCP tool calls
97+
98+
Download the artifacts for a run, then open the `mcp-logs/` directory. Each file corresponds to one MCP server and lists every tool call the agent made.
99+
100+
```bash
101+
gh aw logs <your-workflow-id> --artifacts all
102+
```
103+
104+
Browse the log files in `.github/aw/logs/<run-id>/mcp-logs/`.
105+
106+
- [ ] I found the `mcp-logs/` directory in the downloaded artifacts
107+
- [ ] I identified at least one tool call and noted the tool name
108+
- [ ] I wrote one sentence describing what the agent was trying to accomplish
109+
- [ ] I checked `agent_usage.json` for the total token count
110+
111+
### Inspect the firewall records
112+
113+
The raw domain-level network access logs live in `sandbox/firewall/audit/` inside the agent artifact. Scan them to confirm your workflow only contacted expected domains.
114+
115+
- [ ] I opened `sandbox/firewall/audit/` in the downloaded artifacts
116+
- [ ] I identified at least one domain the workflow accessed
117+
- [ ] If any domains were blocked, I know to add them to `network.allow` in the workflow frontmatter
118+
58119
## ✅ Checkpoint
59120

60-
- [ ] You can identify the five files inside the agent artifact and what each contains
121+
- [ ] You can identify each file inside the agent artifact and what it contains
61122
- [ ] You understand what ⌖ AIC represents and how it differs from agent AIC
62-
- [ ] You can use `firewall.md` to identify blocked domains and add them to `network.allow`
123+
- [ ] You can find blocked domains in the firewall audit records and add them to `network.allow`
63124
- [ ] You know what the threat detection verdict checks for
125+
- [ ] You ran `gh aw audit` on a real run and reviewed the generated report
126+
- [ ] You explored `mcp-logs/` to identify tool calls from a completed run
64127

65128
<!-- journey: all -->
66129
Return to [Audit and Monitor Your Agentic Workflows](25-audit-and-observability.md).

0 commit comments

Comments
 (0)