[observability] Observability Coverage Report - 2026-06-17

[github-actions[bot]]

Executive Summary

I reviewed a capped sample of 20 workflow runs from the last 7 days. Firewall observability is the weak spot: 18 sampled runs had firewall enabled, but only 11 included the Squid access.log, so 7 firewall-enabled failures are missing the log needed for network debugging. MCP telemetry is present in every sampled run via rpc-messages.jsonl; 11 files contain usable JSON-RPC traffic and 9 are zero-length placeholders from early-abort runs.

No run in the sample was missing both MCP telemetry files, so there are no MCP critical gaps. The main action item is to restore firewall log emission on failure paths.

Key Alerts and Anomalies

Caution

Critical issues:

• access.log is missing from 7 firewall-enabled runs: §27653712017, §27653712023, §27653712058, §27653712062, §27653929599, §27653928816, §27653927895.

Warning

Warnings:

• 9 runs produced zero-length rpc-messages.jsonl files. They are not critical because the file exists, but they are not useful for debugging: §27653712017, §27653712023, §27653712029, §27653712034, §27653712058, §27653712062, §27653929599, §27653928816, §27653927895.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	18	11	61.1%	Critical
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	20	20	100.0%	Warning

Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
PR Sous Chef	27655733554	Present	207	68	139	Healthy
Smoke Copilot - AOAI (apikey)	27655298919	Present	765	324	441	Healthy
Smoke Copilot	27655298190	Present	239	62	177	Healthy
Smoke Claude	27655260785	Present	207	73	134	Healthy
Daily BYOK Ollama Test	27653556622	Present	54	22	32	Healthy
Auto-Triage Issues	27653494889	Present	25	9	16	Healthy
Design Decision Gate 🏗️	27653544507	Present	28	13	15	Healthy
Test Quality Sentinel	27653544511	Present	251	46	205	Healthy
Matt Pocock Skills Reviewer	27653544516	Present	183	43	140	Healthy
PR Code Quality Reviewer	27653544520	Present	407	97	310	Healthy
PR Sous Chef	27653957697	Present	201	28	173	Healthy

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Link
Smoke Codex	27653712017	2026-06-16	§27653712017
Changeset Generator	27653712023	2026-06-16	§27653712023
Smoke Claude	27653712058	2026-06-16	§27653712058
Agent Container Smoke Test	27653712062	2026-06-16	§27653712062
Smoke Copilot - AOAI (Entra)	27653929599	2026-06-16	§27653929599
Smoke Copilot - AOAI (apikey)	27653928816	2026-06-16	§27653928816
Smoke Copilot	27653927895	2026-06-16	§27653927895

MCP-Enabled Runs

All sampled MCP runs used the canonical fallback rpc-messages.jsonl; no gateway.jsonl file was present in this sample.

Workflow	Run ID	Telemetry Source	Entries	Tool Calls	Errors	Status
PR Sous Chef	27655733554	rpc-messages.jsonl	10	4	0	Healthy
Smoke Copilot - AOAI (apikey)	27655298919	rpc-messages.jsonl	34	13	0	Healthy
Smoke Copilot	27655298190	rpc-messages.jsonl	54	23	0	Healthy
Smoke Claude	27655260785	rpc-messages.jsonl	48	20	0	Healthy
Daily BYOK Ollama Test	27653556622	rpc-messages.jsonl	4	0	0	Healthy
Auto-Triage Issues	27653494889	rpc-messages.jsonl	4	1	0	Healthy
Design Decision Gate 🏗️	27653544507	rpc-messages.jsonl	4	1	0	Healthy
Test Quality Sentinel	27653544511	rpc-messages.jsonl	6	2	0	Healthy
Matt Pocock Skills Reviewer	27653544516	rpc-messages.jsonl	12	4	0	Healthy
PR Code Quality Reviewer	27653544520	rpc-messages.jsonl	10	4	0	Healthy
PR Sous Chef	27653957697	rpc-messages.jsonl	4	1	0	Healthy
Smoke Codex	27653712017	rpc-messages.jsonl	0	0	0	Warning
Changeset Generator	27653712023	rpc-messages.jsonl	0	0	0	Warning
Smoke Gemini	27653712029	rpc-messages.jsonl	0	0	0	Warning
Smoke Antigravity	27653712034	rpc-messages.jsonl	0	0	0	Warning
Smoke Claude	27653712058	rpc-messages.jsonl	0	0	0	Warning
Agent Container Smoke Test	27653712062	rpc-messages.jsonl	0	0	0	Warning
Smoke Copilot - AOAI (Entra)	27653929599	rpc-messages.jsonl	0	0	0	Warning
Smoke Copilot - AOAI (apikey)	27653928816	rpc-messages.jsonl	0	0	0	Warning
Smoke Copilot	27653927895	rpc-messages.jsonl	0	0	0	Warning

Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 2,567
• Allowed requests: 785
• Blocked requests: 1,782 (69.4%)
• Most accessed domains: api.githubcopilot.com, gh-aw-foundry.openai.azure.com, api.anthropic.com, github.githubassets.com, api.openai.com

MCP Log Quality

• Telemetry source: rpc-messages.jsonl fallback
• Total JSONL entries analyzed: 190 across the non-empty files
• Unique MCP servers used: 6
• Tool calls: 73
• Top tool-call servers: safeoutputs (56), serena (11), mcpscripts (4), tavily (1), agenticworkflows (1)
• Error rate: 0%
• Average response time: N/A from rpc-messages.jsonl

Healthy Runs Summary

• 11 sampled runs had both firewall logs and usable MCP telemetry.
• 9 sampled runs need early-exit telemetry fixes because both access.log and the RPC payload content were missing.

Recommended Actions

1. Make firewall log emission durable on failure paths so access.log is flushed even when the run aborts early.
2. Add an explicit startup/teardown heartbeat for MCP telemetry so zero-length rpc-messages.jsonl files are distinguishable from successful toolless runs.
3. Keep the fallback RPC capture path in place until gateway.jsonl is consistently available across all MCP-enabled workflows.

References:

• §27653712017
• §27653494889
• §27655298919

Report generated automatically by the Daily Observability Report workflow
Analysis window: Last 7 days | Runs analyzed: 20

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• api.github.com
• github.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.github.com"
    - "github.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · ◷

• expires on Jun 17, 2026, 4:16 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #39928.