You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: workshop/side-quest-25-01-audit-reference.md
+67-4Lines changed: 67 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,6 +4,11 @@
4
4
5
5
> _A detailed companion to [Audit and Monitor Your Agentic Workflows](25-audit-and-observability.md). Use this side quest when you want to understand the full contents of an audit report or dig into individual artifact files._
6
6
7
+
## 📋 Before You Start
8
+
9
+
- You completed [Audit and Monitor Your Agentic Workflows](25-audit-and-observability.md) and have at least one workflow run ID to work with.
10
+
-`gh aw` is installed and authenticated (see [Install the gh-aw CLI Extension](06-install-gh-aw.md)).
11
+
7
12
## [gh aw audit](https://github.github.com/gh-aw/reference/audit/#gh-aw-audit) report anatomy
8
13
9
14
`gh aw audit` generates a Markdown report that covers:
@@ -28,9 +33,9 @@ The `agent` artifact — downloaded by both `gh aw logs --artifacts all` and `gh
|`agent_usage.json`| Token usage for the agent turn |
30
35
31
-
### Parsed log files (--parse)
36
+
### Readable log files
32
37
33
-
When you run `gh aw audit <run-id> --parse`, two readable files are written alongside the raw artifacts:
38
+
The audit report is accompanied by readable files written alongside the raw artifacts:
34
39
35
40
-`log.md` — the full agent conversation formatted as Markdown
36
41
-`firewall.md` — a formatted summary of outbound network access (allowed and blocked domains)
@@ -55,12 +60,70 @@ network:
55
60
56
61
Share the allowed-domains list from a successful run with your enterprise security team as a ready-made firewall allowlist.
57
62
63
+
## Try it yourself
64
+
65
+
### Run an audit on a recent run
66
+
67
+
Open the **Actions** tab in your repository, click a completed workflow run, and copy the run ID from the URL (the number after `/runs/`). Then run:
68
+
69
+
```bash
70
+
gh aw audit <run-id>
71
+
```
72
+
73
+
Sample output:
74
+
75
+
```text
76
+
## Audit Report
77
+
78
+
**Workflow:** daily-status
79
+
**Trigger:** schedule
80
+
**Engine:** copilot
81
+
**Model:** gpt-4o
82
+
83
+
| Metric | Value |
84
+
|---|---|
85
+
| Agent AIC | 42 |
86
+
| ⌖ AIC | 3 |
87
+
| MCP calls | 7 |
88
+
| Threat verdict | none |
89
+
```
90
+
91
+
- [ ] I found a run ID from the Actions tab
92
+
- [ ] The report shows the workflow name, trigger, and model
93
+
- [ ] The ⌖ AIC figure appears separately from Agent AIC
94
+
- [ ] The threat verdict shows `none` (or I noted what was flagged)
95
+
96
+
### Explore MCP tool calls
97
+
98
+
Download the artifacts for a run, then open the `mcp-logs/` directory. Each file corresponds to one MCP server and lists every tool call the agent made.
99
+
100
+
```bash
101
+
gh aw logs <your-workflow-id> --artifacts all
102
+
```
103
+
104
+
Browse the log files in `.github/aw/logs/<run-id>/mcp-logs/`.
105
+
106
+
- [ ] I found the `mcp-logs/` directory in the downloaded artifacts
107
+
- [ ] I identified at least one tool call and noted the tool name
108
+
- [ ] I wrote one sentence describing what the agent was trying to accomplish
109
+
- [ ] I checked `agent_usage.json` for the total token count
110
+
111
+
### Inspect the firewall records
112
+
113
+
The raw domain-level network access logs live in `sandbox/firewall/audit/` inside the agent artifact. Scan them to confirm your workflow only contacted expected domains.
114
+
115
+
- [ ] I opened `sandbox/firewall/audit/` in the downloaded artifacts
116
+
- [ ] I identified at least one domain the workflow accessed
117
+
- [ ] If any domains were blocked, I know to add them to `network.allow` in the workflow frontmatter
118
+
58
119
## ✅ Checkpoint
59
120
60
-
- [ ] You can identify the five files inside the agent artifact and what each contains
121
+
- [ ] You can identify each file inside the agent artifact and what it contains
61
122
- [ ] You understand what ⌖ AIC represents and how it differs from agent AIC
62
-
- [ ] You can use `firewall.md` to identify blocked domains and add them to `network.allow`
123
+
- [ ] You can find blocked domains in the firewall audit records and add them to `network.allow`
63
124
- [ ] You know what the threat detection verdict checks for
125
+
- [ ] You ran `gh aw audit` on a real run and reviewed the generated report
126
+
- [ ] You explored `mcp-logs/` to identify tool calls from a completed run
64
127
65
128
<!-- journey: all -->
66
129
Return to [Audit and Monitor Your Agentic Workflows](25-audit-and-observability.md).
0 commit comments