Skip to content

Improve agentic workflow resilience, tool surface, and gh-aw v0.69.3 alignment #2013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pethers merged 6 commits into from
Apr 26, 2026
Merged

Improve agentic workflow resilience, tool surface, and gh-aw v0.69.3 alignment #2013

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5ccd889
feat(agentic-workflows): improve resilience, full tool surface, and g…
Copilot Apr 26, 2026
6c18c96
Merge branch 'main' into copilot/improve-agentic-workflows
pethers Apr 26, 2026
e9d2eba
docs(prompts): clarify web-fetch (kebab-case YAML) vs web_fetch (snak…
Copilot Apr 26, 2026
d38cd8f
docs: address review feedback on keepalive wording, containers egress…
Copilot Apr 26, 2026
1cec920
workflows: replace `containers` ecosystem with explicit Docker Hub ho…
Copilot Apr 26, 2026
a8e1dc5
prompts/07-commit-and-pr: clarify 14-day cache-memory window is targe…
Copilot Apr 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions .github/prompts/02-mcp-access.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,21 @@ Authoritative per-workflow surface: the `mcp-servers:` + `tools:` blocks in that

## Servers & tool naming

News workflows declare three data MCP servers + the built-in `github` toolset (via `tools.github.toolsets: [all]`) + `bash` + `agentic-workflows`.
News workflows declare three data MCP servers + the built-in `github` toolset (via `tools.github.toolsets: [all]`) + `bash` + `edit` + `web-fetch` (frontmatter key; agent calls it as `web_fetch`) + `agentic-workflows` + `cache-memory` (resilience).

| Server | Transport | Declared in | Tool-name style | Example tools |
|--------|-----------|-------------|-----------------|---------------|
> **Naming convention reminder**: gh-aw frontmatter keys use **kebab-case** (`tools.web-fetch:`, `tools.cache-memory:`, `tools.agentic-workflows:`); the **runtime tool names** the agent invokes use **snake_case** (`web_fetch`, `cache_memory`, …). The same split applies to safe outputs (`safe-outputs.create-pull-request:` in YAML → `safeoutputs___create_pull_request` at call time).

| Server / tool | Transport | Declared in | Tool-name style | Example tools |
|---------------|-----------|-------------|-----------------|---------------|
| `riksdag-regering` | HTTP (Render) | workflow `mcp-servers:` | `snake_case` | `get_sync_status`, `search_dokument`, `get_voteringar`, `get_dokument_innehall` |
| `scb` | container (`@jarib/pxweb-mcp`) | workflow `mcp-servers:` | `snake_case` | `search_tables`, `get_table_info`, `query_table` |
| `world-bank` | container (`worldbank-mcp`) | workflow `mcp-servers:` | `kebab-case` | `get-economic-data` *(legacy — economic context has migrated to IMF CLI; keep for WGI governance / environment / social residue only)*, `get-country-info`, `search-indicators` |
| `github` | HTTP (Copilot MCP) | workflow `tools.github` | standard | full GitHub MCP toolset |
| `bash` | local helper | workflow `tools.bash` | standard | shell execution (**also hosts the IMF CLI — see § IMF CLI below**) |
| `safeoutputs` | runner | always available | `snake_case` | `safeoutputs___create_pull_request`, `safeoutputs___noop`, `safeoutputs___dispatch_workflow` |
| `scb` | container (`@jarib/pxweb-mcp`, `node:25-alpine`) | workflow `mcp-servers:` | `snake_case` | `search_tables`, `get_table_info`, `query_table` |
| `world-bank` | container (`worldbank-mcp`, `node:25-alpine`) | workflow `mcp-servers:` | `kebab-case` | `get-economic-data` *(legacy — economic context has migrated to IMF CLI; keep for WGI governance / environment / social residue only)*, `get-country-info`, `search-indicators` |
| `github` | HTTP (Copilot MCP) | workflow `tools.github` (`toolsets: [all]`) | standard | full GitHub MCP toolset (issues, PRs, repos, code-search, actions, releases, discussions, …) |
| `bash` | local helper | workflow `tools.bash: true` | standard | shell execution (**also hosts the IMF CLI — see § IMF CLI below**) |
| `edit` | local helper | workflow `tools.edit:` | standard | filesystem edits inside `$GITHUB_WORKSPACE` |
| `web-fetch` | local helper | workflow `tools.web-fetch:` | standard | HTTP fetch for non-MCP public sources (e.g. `www.statskontoret.se`, `riksdagsmonitor.com`) — domain-filtered through the AWF firewall. **Agent calls this as `web_fetch`** (snake_case runtime name) |
| `cache-memory` | GitHub Actions cache | workflow `tools.cache-memory:` | (filesystem) | persistent file storage at `/tmp/gh-aw/cache-memory/` keyed by `news-${workflow}-${article_date}` (14-day retention). Survives across runs, restores from previous run on cache miss → **resilience for failed-PR retries**. See [`07-commit-and-pr.md` §Cache-memory recovery](07-commit-and-pr.md). |
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the cache-memory row, the phrase “restores from previous run on cache miss” is misleading: a cache miss means nothing is restored. If the intent is that restore-keys can fall back to older keys, reword to something like “restores from the most recent prior cache via restore-keys when the exact key isn’t found” to avoid confusing operators/agents.

Suggested change
| `cache-memory` | GitHub Actions cache | workflow `tools.cache-memory:` | (filesystem) | persistent file storage at `/tmp/gh-aw/cache-memory/` keyed by `news-${workflow}-${article_date}` (14-day retention). Survives across runs, restores from previous run on cache miss → **resilience for failed-PR retries**. See [`07-commit-and-pr.md` §Cache-memory recovery](07-commit-and-pr.md). |
| `cache-memory` | GitHub Actions cache | workflow `tools.cache-memory:` | (filesystem) | persistent file storage at `/tmp/gh-aw/cache-memory/` keyed by `news-${workflow}-${article_date}` (14-day retention). Survives across runs and can restore the most recent prior cache via `restore-keys` when the exact key is not found → **resilience for failed-PR retries**. See [`07-commit-and-pr.md` §Cache-memory recovery](07-commit-and-pr.md). |

Copilot uses AI. Check for mistakes.
| `safeoutputs` | runner (Streamable HTTP) | always available | `snake_case` | `safeoutputs___create_pull_request`, `safeoutputs___noop`, `safeoutputs___dispatch_workflow`, `safeoutputs___add_comment`, `safeoutputs___missing_data`, `safeoutputs___missing_tool`, `safeoutputs___report_incomplete` |

`filesystem`, `memory`, and `sequential-thinking` are declared in [`.github/copilot-mcp.json`](../copilot-mcp.json) for the **local Copilot / `assign_copilot_to_issue`** channel and are **not** available to news workflows unless the workflow itself declares them under `mcp-servers:`.

Expand Down Expand Up @@ -67,4 +72,16 @@ Run once at workflow start, then proceed — do not loop forever.

## Pre-warm step (CI job, not prompt)

Every news workflow declares a **single** `curl`-based pre-warm step with ≤ 6 retries, ≤ 20 s apart. With `curl --max-time 30`, the worst-case runtime can exceed 4 minutes, so this is a best-effort pre-warm rather than a hard ≤ 2 minute guarantee. If a strict 2 minute cap is required, the workflow's `curl` timeout and/or retry policy must be reduced accordingly. No background pingers. MCP session longevity is maintained via `sandbox.mcp.keepalive-interval: 300`.
Every news workflow declares a **single** `curl`-based pre-warm step with ≤ 6 retries, ≤ 20 s apart. With `curl --max-time 30`, the worst-case runtime can exceed 4 minutes, so this is a best-effort pre-warm rather than a hard ≤ 2 minute guarantee. If a strict 2 minute cap is required, the workflow's `curl` timeout and/or retry policy must be reduced accordingly. No background pingers.

## MCP gateway keepalive (`sandbox.mcp.keepalive-interval`)

Every news workflow sets `sandbox.mcp.keepalive-interval: 300`, which compiles to the gh-aw mcp-gateway's `keepaliveInterval` field. Semantics ([upstream spec](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/mcp-gateway.md)):

| Value | Meaning |
|-------|---------|
| `0` or unset | Gateway default = **1500 s (25 min)** — too slow for 45-min news jobs; the `riksdag-regering` HTTP MCP would idle out before Pass 2 finishes |
| `-1` | Disable keepalive entirely (do not use) |
| **`300`** *(our setting)* | Ping every 5 minutes — keeps `riksdag-regering` (HTTP) and any other HTTP-backed MCPs warm for the entire 45–50 min job. **This is the resilience knob that lets us run 45-50 min sessions reliably.** |

The keepalive pings the **upstream HTTP MCPs through the gateway**. It does **not** keep the local `safeoutputs` Streamable-HTTP idle session alive — that session has its own ~25–30 min idle timeout (Timer C in `00-base-contract.md` and `07-commit-and-pr.md`). Reach `safeoutputs___create_pull_request` by minute 28 (hard 30) regardless of keepalive.
23 changes: 23 additions & 0 deletions .github/prompts/07-commit-and-pr.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,29 @@ Translations for the remaining twelve languages are produced by the dedicated **

5. **Do not** `git push`, `git checkout`, or `git checkout -b` after the call. The safe-outputs runner job publishes the PR; subsequent agent commits are not added.

## Cache-memory recovery (resilience for failed PRs)

Every news workflow declares `tools.cache-memory:` keyed by `news-${{ github.workflow }}-${{ inputs.article_date || 'today' }}` with 14-day retention (see `02-mcp-access.md` §Servers & tool naming). gh-aw automatically restores the cache from the previous run on each invocation — analysis artifacts under `/tmp/gh-aw/cache-memory/` survive across failed runs and can be reused on the next attempt.

**On every run, immediately after MCP pre-warm:**

1. Check whether `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` exists with prior analysis artifacts (Family A/B/C/D `.md` files). If so, this is a **retry of a failed run**. Copy them into `analysis/daily/$ARTICLE_DATE/$SUBFOLDER/` *before* re-running the analysis pipeline so Pass 2 builds on Pass 1 work that previous runs already paid for.
2. After a successful Pass 1 (or after the analysis gate passes), copy the produced `.md` artifacts back to `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` so the next run can recover them if `safeoutputs___create_pull_request` fails or the run is killed by Timer A/B/C.
3. The cache is **automatically saved** by gh-aw at job end — the agent does **not** call any safe-output tool to persist it. Just write to `/tmp/gh-aw/cache-memory/`.
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section states cache-memory artifacts “survive across failed runs” and are “automatically saved” at job end. In the compiled workflows, saving back to the GitHub cache is performed in a separate update_cache_memory job that only runs when needs.agent.result == 'success', so runs where the agent job fails won’t persist new cache-memory contents for the next retry. Please clarify the wording to match this behavior (e.g., recovery is reliable for PR-publication failures after a successful agent run, but not for agent-job failures/timeouts).

Suggested change
Every news workflow declares `tools.cache-memory:` keyed by `news-${{ github.workflow }}-${{ inputs.article_date || 'today' }}` with 14-day retention (see `02-mcp-access.md` §Servers & tool naming). gh-aw automatically restores the cache from the previous run on each invocation — analysis artifacts under `/tmp/gh-aw/cache-memory/` survive across failed runs and can be reused on the next attempt.
**On every run, immediately after MCP pre-warm:**
1. Check whether `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` exists with prior analysis artifacts (Family A/B/C/D `.md` files). If so, this is a **retry of a failed run**. Copy them into `analysis/daily/$ARTICLE_DATE/$SUBFOLDER/` *before* re-running the analysis pipeline so Pass 2 builds on Pass 1 work that previous runs already paid for.
2. After a successful Pass 1 (or after the analysis gate passes), copy the produced `.md` artifacts back to `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` so the next run can recover them if `safeoutputs___create_pull_request` fails or the run is killed by Timer A/B/C.
3. The cache is **automatically saved** by gh-aw at job end — the agent does **not** call any safe-output tool to persist it. Just write to `/tmp/gh-aw/cache-memory/`.
Every news workflow declares `tools.cache-memory:` keyed by `news-${{ github.workflow }}-${{ inputs.article_date || 'today' }}` with 14-day retention (see `02-mcp-access.md` §Servers & tool naming). gh-aw automatically restores the cache from the last successfully persisted run on each invocation. Analysis artifacts under `/tmp/gh-aw/cache-memory/` can therefore be reused on the next attempt when a previous run reached the cache-update stage, but newly generated cache-memory content from an agent job that fails or times out is **not** guaranteed to persist for the next retry.
**On every run, immediately after MCP pre-warm:**
1. Check whether `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` exists with prior analysis artifacts (Family A/B/C/D `.md` files). If so, treat this as a **retry with recoverable prior work**. Copy them into `analysis/daily/$ARTICLE_DATE/$SUBFOLDER/` *before* re-running the analysis pipeline so Pass 2 builds on Pass 1 work that a previous successful agent run already produced.
2. After a successful Pass 1 (or after the analysis gate passes), copy the produced `.md` artifacts back to `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` so they are available for persistence if the workflow later fails during PR publication or another post-agent stage.
3. The agent does **not** call any safe-output tool to persist cache-memory; it only writes to `/tmp/gh-aw/cache-memory/`. In compiled workflows, the updated cache is saved for the next run by a separate cache-update step/job that runs only after a **successful agent job**, so recovery is reliable for post-agent failures (for example PR-publication problems) but not for agent-job failures/timeouts.

Copilot uses AI. Check for mistakes.

Cache-memory is **not** a substitute for committing real files on disk under `analysis/daily/`. It is a recovery mechanism for the next run, not a deliverable.

## PR creation resilience (`fallback-as-issue`, `if-no-changes`)

Every news workflow's `safe-outputs.create-pull-request:` block sets two explicit resilience flags:

| Flag | Value | Effect |
|------|-------|--------|
| `fallback-as-issue` | `true` *(explicit, also the gh-aw default)* | If org settings disable "Allow GitHub Actions to create and approve pull requests", the safe-outputs runner falls back to creating an **issue with branch link** instead of failing. The agent's commit is still pushed; only the PR-creation step degrades. |
| `if-no-changes` | `warn` | If the agent commits but the patch is empty (e.g. all artifacts already exist for this date with `force_generation=false`), the runner emits a warning instead of failing the workflow. Combined with the run-mode selection in `03-data-download.md`, this prevents spurious red runs on duplicate-date dispatches. |

Neither flag changes the agent's behaviour — both are runner-side resilience knobs. The agent still calls `safeoutputs___create_pull_request` exactly once. See [upstream `create-pull-request` reference](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/safe-outputs-pull-requests.md) for the full schema.

## Canonical PR body template

```markdown
Expand Down
18 changes: 18 additions & 0 deletions .github/workflows/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,24 @@ Each agentic workflow is a **pair**: an authored `.md` source + a compiled `.loc
8. `../prompts/07-commit-and-pr.md` — stage → commit → exactly one `create_pull_request`
9. *(Tier-C workflows only)* `../prompts/ext/tier-c-aggregation.md` — 14-artifact gate, period multipliers

### Common tool surface (every `news-*.md`)

Every news workflow declares the **same** tool & runtime surface for parity, resilience, and full gh-aw v0.69.3 capability coverage:

| Field | Value | Purpose |
|-------|-------|---------|
| `runtimes.node.version` | `"25"` | Pinned Node 25 for IMF CLI + render scripts |
| `tools.github.toolsets` | `[all]` | Full GitHub MCP surface (issues, PRs, repos, code-search, actions, releases, discussions, …); see [`github-tools.md`](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/github-tools.md) |
| `tools.bash` / `tools.edit` / `tools.web-fetch` / `tools.agentic-workflows` | enabled | Full local tool surface; `web-fetch` reaches non-MCP public sources (`statskontoret.se`, `riksdagsmonitor.com`) through the AWF firewall |
| `tools.cache-memory` | keyed by `news-${workflow}-${article_date}`, 14-day retention | **Resilience knob** — analysis artifacts persisted at `/tmp/gh-aw/cache-memory/`; restored on the next run if the previous PR failed (see [`07-commit-and-pr.md` §Cache-memory recovery](../prompts/07-commit-and-pr.md)) |
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The table claims tools.cache-memory has “14-day retention”. In the compiled workflow this is implemented via actions/cache, which doesn’t support a fixed per-cache retention period (eviction is best-effort and repo-policy driven). Recommend rewording to avoid a hard guarantee (e.g. “configured with 14-day artifact retention / best-effort cache persistence”).

Suggested change
| `tools.cache-memory` | keyed by `news-${workflow}-${article_date}`, 14-day retention | **Resilience knob** — analysis artifacts persisted at `/tmp/gh-aw/cache-memory/`; restored on the next run if the previous PR failed (see [`07-commit-and-pr.md` §Cache-memory recovery](../prompts/07-commit-and-pr.md)) |
| `tools.cache-memory` | keyed by `news-${workflow}-${article_date}`; best-effort cache persistence aligned with a 14-day recovery window | **Resilience knob** — analysis artifacts persisted at `/tmp/gh-aw/cache-memory/`; may be restored on the next run if the previous PR failed and the cache entry is still available (see [`07-commit-and-pr.md` §Cache-memory recovery](../prompts/07-commit-and-pr.md)) |

Copilot uses AI. Check for mistakes.
| `tools.playwright` | enabled in `news-evening-analysis` + `news-realtime-monitor` only | Live HTML validation for tier-C aggregation runs |
| `features.mcp-gateway` | `true` | Routes all MCP traffic through the gh-aw mcp-gateway (single audit point) |
| `sandbox.mcp.keepalive-interval` | `300` (5 min) | Compiles to gateway `keepaliveInterval`; overrides upstream default `1500 s (25 min)` so HTTP MCPs (`riksdag-regering`) stay warm for the full 45–50 min job (see [`02-mcp-access.md` §MCP gateway keepalive](../prompts/02-mcp-access.md)) |
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The keepalive description mentions enabling “45–50 min sessions” / “full 45–50 min job”, but these workflows are configured with timeout-minutes: 45. Suggest adjusting this wording to avoid implying runs can exceed the configured job timeout (e.g., “full 45‑minute job budget”).

Suggested change
| `sandbox.mcp.keepalive-interval` | `300` (5 min) | Compiles to gateway `keepaliveInterval`; overrides upstream default `1500 s (25 min)` so HTTP MCPs (`riksdag-regering`) stay warm for the full 45–50 min job (see [`02-mcp-access.md` §MCP gateway keepalive](../prompts/02-mcp-access.md)) |
| `sandbox.mcp.keepalive-interval` | `300` (5 min) | Compiles to gateway `keepaliveInterval`; overrides upstream default `1500 s (25 min)` so HTTP MCPs (`riksdag-regering`) stay warm for the full 45-minute job budget (see [`02-mcp-access.md` §MCP gateway keepalive](../prompts/02-mcp-access.md)) |

Copilot uses AI. Check for mistakes.
| `safe-outputs.create-pull-request.fallback-as-issue` | `true` (explicit) | If org disables Actions PR creation, fall back to an issue + branch link instead of failing |
| `safe-outputs.create-pull-request.if-no-changes` | `warn` | Empty patches emit a warning instead of failing the run (e.g. duplicate-date dispatches) |
| `network.allowed` | `node`, `containers`, `github`, `defaults` + IMF/SCB/Riksdag/Statskontoret/site domains | Ecosystem identifiers preferred per upstream `network.md`; `containers` covers `node:25-alpine` images for SCB + World Bank MCPs |
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including the containers ecosystem identifier materially broadens outbound egress (compiled locks now allow multiple container registries like *.docker.io, ghcr.io, quay.io, etc.). Since this is a security-relevant expansion, it would help to explicitly document the expected/required registry set (and/or why broad wildcards are acceptable here) so reviewers can validate it against the project’s egress/allowlist policy.

Suggested change
| `network.allowed` | `node`, `containers`, `github`, `defaults` + IMF/SCB/Riksdag/Statskontoret/site domains | Ecosystem identifiers preferred per upstream `network.md`; `containers` covers `node:25-alpine` images for SCB + World Bank MCPs |
| `network.allowed` | `node`, `containers`, `github`, `defaults` + IMF/SCB/Riksdag/Statskontoret/site domains | Ecosystem identifiers preferred per upstream `network.md`; `containers` is required only for the MCP container images (`node:25-alpine`) used by the SCB and World Bank servers. Reviewers should expect Docker Hub resolution for these pulls (`docker.io`, `registry-1.docker.io`, `auth.docker.io`, and `production.cloudflare.docker.com`). Upstream ecosystem expansion can cause compiled locks to include broader container-registry patterns; in this repo that broader capability is accepted only because current workflows are intended to pull Docker Hub-hosted `node:25-alpine` images. Any switch to `ghcr.io`, `quay.io`, or other registries must be explicitly documented and reviewed against the egress allowlist policy before merge. |

Copilot uses AI. Check for mistakes.
| `permissions` | `contents: read`, `issues: read`, `pull-requests: read`, `actions: read`, `discussions: read`, `security-events: read` | Least-privilege agent token; write capabilities live exclusively in the safe-outputs runner job |

## 🛠️ Automation & Tooling (4)

| File | Trigger | Purpose |
Expand Down
Loading
Loading