Improve agentic workflow resilience, tool surface, and gh-aw v0.69.3 alignment

.github/prompts/02-mcp-access.md

@@ -4,16 +4,21 @@ Authoritative per-workflow surface: the `mcp-servers:` + `tools:` blocks in that
 
 ## Servers & tool naming
 
-News workflows declare three data MCP servers + the built-in `github` toolset (via `tools.github.toolsets: [all]`) + `bash` + `agentic-workflows`.
+News workflows declare three data MCP servers + the built-in `github` toolset (via `tools.github.toolsets: [all]`) + `bash` + `edit` + `web-fetch` (frontmatter key; agent calls it as `web_fetch`) + `agentic-workflows` + `cache-memory` (resilience).
 
-| Server | Transport | Declared in | Tool-name style | Example tools |
-|--------|-----------|-------------|-----------------|---------------|
+> **Naming convention reminder**: gh-aw frontmatter keys use **kebab-case** (`tools.web-fetch:`, `tools.cache-memory:`, `tools.agentic-workflows:`); the **runtime tool names** the agent invokes use **snake_case** (`web_fetch`, `cache_memory`, …). The same split applies to safe outputs (`safe-outputs.create-pull-request:` in YAML → `safeoutputs___create_pull_request` at call time).
+
+| Server / tool | Transport | Declared in | Tool-name style | Example tools |
+|---------------|-----------|-------------|-----------------|---------------|
 | `riksdag-regering` | HTTP (Render) | workflow `mcp-servers:` | `snake_case` | `get_sync_status`, `search_dokument`, `get_voteringar`, `get_dokument_innehall` |
-| `scb` | container (`@jarib/pxweb-mcp`) | workflow `mcp-servers:` | `snake_case` | `search_tables`, `get_table_info`, `query_table` |
-| `world-bank` | container (`worldbank-mcp`) | workflow `mcp-servers:` | `kebab-case` | `get-economic-data` *(legacy — economic context has migrated to IMF CLI; keep for WGI governance / environment / social residue only)*, `get-country-info`, `search-indicators` |
-| `github` | HTTP (Copilot MCP) | workflow `tools.github` | standard | full GitHub MCP toolset |
-| `bash` | local helper | workflow `tools.bash` | standard | shell execution (**also hosts the IMF CLI — see § IMF CLI below**) |
-| `safeoutputs` | runner | always available | `snake_case` | `safeoutputs___create_pull_request`, `safeoutputs___noop`, `safeoutputs___dispatch_workflow` |
+| `scb` | container (`@jarib/pxweb-mcp`, `node:25-alpine`) | workflow `mcp-servers:` | `snake_case` | `search_tables`, `get_table_info`, `query_table` |
+| `world-bank` | container (`worldbank-mcp`, `node:25-alpine`) | workflow `mcp-servers:` | `kebab-case` | `get-economic-data` *(legacy — economic context has migrated to IMF CLI; keep for WGI governance / environment / social residue only)*, `get-country-info`, `search-indicators` |
+| `github` | HTTP (Copilot MCP) | workflow `tools.github` (`toolsets: [all]`) | standard | full GitHub MCP toolset (issues, PRs, repos, code-search, actions, releases, discussions, …) |
+| `bash` | local helper | workflow `tools.bash: true` | standard | shell execution (**also hosts the IMF CLI — see § IMF CLI below**) |
+| `edit` | local helper | workflow `tools.edit:` | standard | filesystem edits inside `$GITHUB_WORKSPACE` |
+| `web-fetch` | local helper | workflow `tools.web-fetch:` | standard | HTTP fetch for non-MCP public sources (e.g. `www.statskontoret.se`, `riksdagsmonitor.com`) — domain-filtered through the AWF firewall. **Agent calls this as `web_fetch`** (snake_case runtime name) |
+| `cache-memory` | GitHub Actions cache | workflow `tools.cache-memory:` | (filesystem) | persistent file storage at `/tmp/gh-aw/cache-memory/` keyed by `news-${workflow}-${article_date}` (14-day retention). Survives across runs and can restore the most recent prior cache via `restore-keys` when the exact key is not found → **resilience for failed-PR retries**. See [`07-commit-and-pr.md` §Cache-memory recovery](07-commit-and-pr.md). |
+| `safeoutputs` | runner (Streamable HTTP) | always available | `snake_case` | `safeoutputs___create_pull_request`, `safeoutputs___noop`, `safeoutputs___dispatch_workflow`, `safeoutputs___add_comment`, `safeoutputs___missing_data`, `safeoutputs___missing_tool`, `safeoutputs___report_incomplete` |
 
 `filesystem`, `memory`, and `sequential-thinking` are declared in [`.github/copilot-mcp.json`](../copilot-mcp.json) for the **local Copilot / `assign_copilot_to_issue`** channel and are **not** available to news workflows unless the workflow itself declares them under `mcp-servers:`.
 
@@ -67,4 +72,16 @@ Run once at workflow start, then proceed — do not loop forever.
 
 ## Pre-warm step (CI job, not prompt)
 
-Every news workflow declares a **single** `curl`-based pre-warm step with ≤ 6 retries, ≤ 20 s apart. With `curl --max-time 30`, the worst-case runtime can exceed 4 minutes, so this is a best-effort pre-warm rather than a hard ≤ 2 minute guarantee. If a strict 2 minute cap is required, the workflow's `curl` timeout and/or retry policy must be reduced accordingly. No background pingers. MCP session longevity is maintained via `sandbox.mcp.keepalive-interval: 300`.
+Every news workflow declares a **single** `curl`-based pre-warm step with ≤ 6 retries, ≤ 20 s apart. With `curl --max-time 30`, the worst-case runtime can exceed 4 minutes, so this is a best-effort pre-warm rather than a hard ≤ 2 minute guarantee. If a strict 2 minute cap is required, the workflow's `curl` timeout and/or retry policy must be reduced accordingly. No background pingers.
+
+## MCP gateway keepalive (`sandbox.mcp.keepalive-interval`)
+
+Every news workflow sets `sandbox.mcp.keepalive-interval: 300`, which compiles to the gh-aw mcp-gateway's `keepaliveInterval` field. Semantics ([upstream spec](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/mcp-gateway.md)):
+
+| Value | Meaning |
+|-------|---------|
+| `0` or unset | Gateway default = **1500 s (25 min)** — too slow for 45-min news jobs; the `riksdag-regering` HTTP MCP would idle out before Pass 2 finishes |
+| `-1` | Disable keepalive entirely (do not use) |
+| **`300`** *(our setting)* | Ping every 5 minutes — keeps `riksdag-regering` (HTTP) and any other HTTP-backed MCPs warm for the full 45-minute job budget. **This is the resilience knob that lets us run the full 45-minute sessions reliably.** |
+
+The keepalive pings the **upstream HTTP MCPs through the gateway**. It does **not** keep the local `safeoutputs` Streamable-HTTP idle session alive — that session has its own ~25–30 min idle timeout (Timer C in `00-base-contract.md` and `07-commit-and-pr.md`). Reach `safeoutputs___create_pull_request` by minute 28 (hard 30) regardless of keepalive.

.github/prompts/07-commit-and-pr.md

@@ -52,6 +52,29 @@ Translations for the remaining twelve languages are produced by the dedicated **
 
 5. **Do not** `git push`, `git checkout`, or `git checkout -b` after the call. The safe-outputs runner job publishes the PR; subsequent agent commits are not added.
 
+## Cache-memory recovery (resilience for failed PRs)
+
+Every news workflow declares `tools.cache-memory:` keyed by `news-${{ github.workflow }}-${{ inputs.article_date || 'today' }}` with a configured 14-day **target** window (see `02-mcp-access.md` §Servers & tool naming). Treat 14 days as an *intended recovery horizon*, **not** as a strict guarantee that cache-memory will remain available for 14 days: actual availability depends on GitHub Actions cache persistence and eviction policy (best-effort, repo-policy driven), and the 14-day setting primarily affects retained artifacts/related workflow data rather than guaranteeing cache retention. gh-aw automatically attempts to restore cache-memory from the **last successfully persisted run** on each invocation. Analysis artifacts under `/tmp/gh-aw/cache-memory/` can therefore often be reused on the next attempt when a previous run reached the cache-update stage, but newly generated cache-memory content from an agent job that **fails or times out** is **not** guaranteed to persist for the next retry.
+
+**On every run, immediately after MCP pre-warm:**
+
+1. Check whether `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` exists with prior analysis artifacts (Family A/B/C/D `.md` files). If so, treat this as a **retry with recoverable prior work**. Copy them into `analysis/daily/$ARTICLE_DATE/$SUBFOLDER/` *before* re-running the analysis pipeline so Pass 2 builds on Pass 1 work that a previous successful agent run already produced.
+2. After a successful Pass 1 (or after the analysis gate passes), copy the produced `.md` artifacts back to `/tmp/gh-aw/cache-memory/$ARTICLE_DATE/$SUBFOLDER/` so they are available for persistence if the workflow later fails during PR publication or another post-agent stage.
+3. The agent does **not** call any safe-output tool to persist cache-memory; it only writes to `/tmp/gh-aw/cache-memory/`. In compiled workflows, the updated cache is saved for the next run by a separate cache-update step/job that runs **only after a successful agent job** (`needs.agent.result == 'success'`), so recovery is reliable for **post-agent failures** (e.g. PR-publication problems) but **not** for agent-job failures or timeouts.
+
+Cache-memory is **not** a substitute for committing real files on disk under `analysis/daily/`. It is a recovery mechanism for the next run, not a deliverable.
+
+## PR creation resilience (`fallback-as-issue`, `if-no-changes`)
+
+Every news workflow's `safe-outputs.create-pull-request:` block sets two explicit resilience flags:
+
+| Flag | Value | Effect |
+|------|-------|--------|
+| `fallback-as-issue` | `true` *(explicit, also the gh-aw default)* | If org settings disable "Allow GitHub Actions to create and approve pull requests", the safe-outputs runner falls back to creating an **issue with branch link** instead of failing. The agent's commit is still pushed; only the PR-creation step degrades. |
+| `if-no-changes` | `warn` | If the agent commits but the patch is empty (e.g. all artifacts already exist for this date with `force_generation=false`), the runner emits a warning instead of failing the workflow. Combined with the run-mode selection in `03-data-download.md`, this prevents spurious red runs on duplicate-date dispatches. |
+
+Neither flag changes the agent's behaviour — both are runner-side resilience knobs. The agent still calls `safeoutputs___create_pull_request` exactly once. See [upstream `create-pull-request` reference](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/safe-outputs-pull-requests.md) for the full schema.
+
 ## Canonical PR body template
 
 ```markdown

.github/workflows/README.md

@@ -99,6 +99,24 @@ Each agentic workflow is a **pair**: an authored `.md` source + a compiled `.loc
 8. `../prompts/07-commit-and-pr.md` — stage → commit → exactly one `create_pull_request`
 9. *(Tier-C workflows only)* `../prompts/ext/tier-c-aggregation.md` — 14-artifact gate, period multipliers
 
+### Common tool surface (every `news-*.md`)
+
+Every news workflow declares the **same** tool & runtime surface for parity, resilience, and full gh-aw v0.69.3 capability coverage:
+
+| Field | Value | Purpose |
+|-------|-------|---------|
+| `runtimes.node.version` | `"25"` | Pinned Node 25 for IMF CLI + render scripts |
+| `tools.github.toolsets` | `[all]` | Full GitHub MCP surface (issues, PRs, repos, code-search, actions, releases, discussions, …); see [`github-tools.md`](https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/github-tools.md) |
+| `tools.bash` / `tools.edit` / `tools.web-fetch` / `tools.agentic-workflows` | enabled | Full local tool surface; `web-fetch` reaches non-MCP public sources (`statskontoret.se`, `riksdagsmonitor.com`) through the AWF firewall |
+| `tools.cache-memory` | keyed by `news-${workflow}-${article_date}`; best-effort cache persistence aligned with a 14-day recovery window | **Resilience knob** — analysis artifacts persisted at `/tmp/gh-aw/cache-memory/`; may be restored on the next run if the previous PR failed and the cache entry is still available (see [`07-commit-and-pr.md` §Cache-memory recovery](../prompts/07-commit-and-pr.md)) |
+| `tools.playwright` | enabled in `news-evening-analysis` + `news-realtime-monitor` only | Live HTML validation for tier-C aggregation runs |
+| `features.mcp-gateway` | `true` | Routes all MCP traffic through the gh-aw mcp-gateway (single audit point) |
+| `sandbox.mcp.keepalive-interval` | `300` (5 min) | Compiles to gateway `keepaliveInterval`; overrides upstream default `1500 s (25 min)` so HTTP MCPs (`riksdag-regering`) stay warm for the full 45-minute job budget (see [`02-mcp-access.md` §MCP gateway keepalive](../prompts/02-mcp-access.md)) |
+| `safe-outputs.create-pull-request.fallback-as-issue` | `true` (explicit) | If org disables Actions PR creation, fall back to an issue + branch link instead of failing |
+| `safe-outputs.create-pull-request.if-no-changes` | `warn` | Empty patches emit a warning instead of failing the run (e.g. duplicate-date dispatches) |
+| `network.allowed` | `node`, `github`, `defaults` + explicit Docker Hub hosts (`docker.io`, `registry-1.docker.io`, `auth.docker.io`, `production.cloudflare.docker.com`) + IMF/SCB/Riksdag/Statskontoret/site domains | Ecosystem identifiers preferred per upstream `network.md`. The broad `containers` ecosystem (which would also permit `ghcr.io`, `quay.io`, `gcr.io`, `mcr.microsoft.com`, `pkgs.k8s.io`, …) is **deliberately omitted** to keep least-privilege egress; only the minimal Docker Hub hosts actually required to resolve `node:25-alpine` for the SCB and World Bank MCP servers are enumerated. Any future switch to `ghcr.io`, `quay.io`, or other registries must add the specific hosts and be reviewed against the egress allowlist policy before merge. |
+| `permissions` | `contents: read`, `issues: read`, `pull-requests: read`, `actions: read`, `discussions: read`, `security-events: read` | Least-privilege agent token; write capabilities live exclusively in the safe-outputs runner job |
+
 ## 🛠️ Automation & Tooling (4)
 
 | File | Trigger | Purpose |