Merge pull request #2008 from Hack23/copilot/update-agentic-workflows-and-prompts

Fix drift in agentic workflow contract, prompts, and docs

Author: pethers

.github/actions/news-prewarm/action.yml

@@ -0,0 +1,106 @@
+name: 'News workflow pre-warm & pre-flight'
+description: |
+  Shared setup for every `.github/workflows/news-*.md` agentic workflow:
+    1. Set up Node.js 25 (matches `runtimes.node.version` in every workflow).
+    2. `npm ci --prefer-offline --no-audit`.
+    3. Pre-warm the riksdag-regering MCP server (Render.com cold-start mitigation).
+    4. Pre-flight DNS + HTTPS reachability + MCP `tools/list` probe for the
+       upstream services consumed by the news pipeline.
+
+  This action exists to deduplicate ~80 lines of identical YAML across the 11
+  news workflow files. Edit it in one place.
+inputs:
+  node-version:
+    description: 'Node.js version (must match the workflow `runtimes.node.version`).'
+    required: false
+    default: '25'
+  mcp-url:
+    description: 'Riksdag-regering MCP HTTP endpoint to pre-warm.'
+    required: false
+    default: 'https://riksdag-regering-ai.onrender.com/mcp'
+runs:
+  using: 'composite'
+  steps:
+    - name: Setup Node.js
+      uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+      with:
+        node-version: ${{ inputs.node-version }}
+
+    - name: Install dependencies
+      shell: bash
+      run: npm ci --prefer-offline --no-audit
+
+    - name: Pre-warm MCP server (Render.com cold start mitigation)
+      shell: bash
+      env:
+        MCP_URL: ${{ inputs.mcp-url }}
+      run: |
+        echo "🔥 Pre-warming riksdag-regering MCP server via MCP protocol..."
+        WARM=false
+        for i in 1 2 3 4 5 6; do
+          RESP=$(curl -sf --max-time 30 -X POST \
+            -H "Content-Type: application/json" \
+            -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
+            "$MCP_URL" 2>/dev/null) || true
+          if echo "$RESP" | grep -q '"tools"'; then
+            TOOL_COUNT=$(echo "$RESP" | grep -o '"name"' | wc -l)
+            echo "✅ MCP server responded on attempt $i with $TOOL_COUNT tools registered"
+            WARM=true
+            break
+          fi
+          echo "⏳ Attempt $i/6 — server may be cold-starting, waiting 20s..."
+          sleep 20
+        done
+        if [ "$WARM" = "false" ]; then
+          echo "⚠️ MCP server did not respond after 6 attempts — agent will retry via in-prompt health gate"
+        fi
+
+    - name: Pre-flight external endpoint reachability check (runs before MCP Gateway)
+      shell: bash
+      env:
+        MCP_URL: ${{ inputs.mcp-url }}
+      run: |
+        echo "🔍 Network Diagnostics — $(date -u '+%Y-%m-%dT%H:%M:%SZ')"
+        echo "═══════════════════════════════════════════"
+        echo ""
+        echo "📡 DNS Resolution Tests:"
+        for domain in riksdag-regering-ai.onrender.com api.scb.se api.worldbank.org data.riksdagen.se www.riksdagen.se www.regeringen.se www.statskontoret.se statskontoret.se; do
+          if nslookup "$domain" >/dev/null 2>&1; then
+            IP=$(nslookup "$domain" 2>/dev/null | grep -A1 "Name:" | grep "Address:" | head -1 | awk '{print $2}')
+            echo "  ✅ $domain → $IP"
+          else
+            echo "  ❌ $domain — DNS FAILED"
+          fi
+        done
+        echo ""
+        echo "🌐 HTTPS Connectivity Tests:"
+        for url in \
+          "$MCP_URL" \
+          "https://api.scb.se/OV0104/v2beta" \
+          "https://api.worldbank.org/v2/country/SE?format=json" \
+          "https://data.riksdagen.se/dokumentlista/?sok=test&doktyp=bet&utformat=json&a=1" \
+        ; do
+          HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 10 "$url" 2>/dev/null || echo "000")
+          DOMAIN=$(echo "$url" | sed 's|https://||' | cut -d/ -f1)
+          if [ "$HTTP_CODE" -ge 200 ] && [ "$HTTP_CODE" -lt 400 ]; then
+            echo "  ✅ $DOMAIN → HTTP $HTTP_CODE"
+          elif [ "$HTTP_CODE" = "000" ]; then
+            echo "  ❌ $DOMAIN → TIMEOUT/UNREACHABLE"
+          else
+            echo "  ⚠️ $DOMAIN → HTTP $HTTP_CODE"
+          fi
+        done
+        echo ""
+        echo "🔌 MCP Server Tool Count:"
+        TOOL_RESP=$(curl -sf --max-time 15 -X POST \
+          -H "Content-Type: application/json" \
+          -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
+          "$MCP_URL" 2>/dev/null) || TOOL_RESP=""
+        if echo "$TOOL_RESP" | grep -q '"tools"'; then
+          TOOL_COUNT=$(echo "$TOOL_RESP" | grep -o '"name"' | wc -l)
+          echo "  ✅ riksdag-regering MCP: $TOOL_COUNT tools registered"
+        else
+          echo "  ❌ riksdag-regering MCP: No tools response (server may still be starting)"
+        fi
+        echo ""
+        echo "═══════════════════════════════════════════"

.github/agents/intelligence-operative.md

@@ -12,7 +12,7 @@ tools: ["*"]
 2. [`.github/copilot-mcp.json`](/.github/copilot-mcp.json) — MCP servers available
 3. [`README.md`](/README.md) — project mission, features, architecture
 4. [`SECURITY_ARCHITECTURE.md`](/SECURITY_ARCHITECTURE.md) & [`THREAT_MODEL.md`](/THREAT_MODEL.md)
-5. [`.github/skills/`](/.github/skills/) — auto-loaded skills library (92 skills)
+5. [`.github/skills/`](/.github/skills/) — auto-loaded skills library (91 skills)
 
 ---
 

.github/aw/actions-lock.json

@@ -30,6 +30,11 @@
       "version": "v6.3.0",
       "sha": "53b83947a5a98c8d113130e565377fae1a50d02f"
     },
+    "actions/setup-node@v6.4.0": {
+      "repo": "actions/setup-node",
+      "version": "v6.4.0",
+      "sha": "48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e"
+    },
     "actions/upload-artifact@v7": {
       "repo": "actions/upload-artifact",
       "version": "v7",

.github/copilot-instructions.md

@@ -3,17 +3,17 @@
 ## 📋 Repository Context
 
 **Project**: Riksdagsmonitor — Swedish Parliament (Riksdag) monitoring platform
-**Stack**: HTML5, CSS3, TypeScript 6.0.2, Vite 8.0.3, Vitest 4.1.2, Cypress 15.13.0
+**Stack**: HTML5, CSS3, TypeScript 6.0.3, Vite 8.0.10, Vitest 4.1.5, Cypress 15.14.1
 **Runtime**: Node.js 25, ES2025 target, ESNext modules
 **Deploy**: GitHub Pages + AWS S3 dual deployment
 **Languages**: 14-language support (EN, SV, DA, NB, FI, DE, FR, ES, NL, AR, HE, JA, KO, ZH)
 **Security**: ISO 27001:2022, NIST CSF 2.0, CIS Controls v8.1 compliant
 **Organization**: Hack23 AB
 **ISMS**: [Hack23 ISMS-PUBLIC](https://github.com/Hack23/ISMS-PUBLIC)
-**Version**: 0.8.17
+**Version**: 0.8.56
 **Agents**: 24 agent files (14 persona + 9 workflow-specialist + 1 developer-instructions) in `.github/agents/`
 **Skills**: 91 skills in `.github/skills/` (including 13 gh-aw skills)
-**Workflows**: 45 workflow files (21 standard `.yml` + 12 agentic `.md` sources + 12 compiled `.lock.yml`)
+**Workflows**: 43 workflow files (21 standard `.yml` + 11 agentic `.md` sources + 11 compiled `.lock.yml`)
 **MCP Servers**: 8 configured (riksdag-regering, scb, world-bank, github, filesystem, memory, sequential-thinking, playwright)
 
 ## 🎯 Core Rules
@@ -112,7 +112,7 @@ Map every security-relevant control to **ISO 27001:2022 Annex A**, **NIST CSF 2.
 
 ## 🤖 GitHub Agentic Workflows
 
-This repo uses [GitHub Agentic Workflows](https://github.github.com/gh-aw/) (gh-aw v0.69.3, pinned via `gh-aw-actions/setup-cli@v0.69.3`) for AI-powered news generation. 12 agentic workflows in `.github/workflows/` produce daily political intelligence articles with five-layer security:
+This repo uses [GitHub Agentic Workflows](https://github.github.com/gh-aw/) (gh-aw v0.69.3, pinned via `gh-aw-actions/setup-cli@v0.69.3`) for AI-powered news generation. 11 agentic workflows in `.github/workflows/` produce daily political intelligence articles with five-layer security:
 
 1. **Read-only tokens** — Agent gets only read permissions
 2. **Zero secrets in agent** — Write tokens isolated in separate jobs
@@ -289,5 +289,5 @@ tsx scripts/imf-fetch.ts list-indicators
 
 ---
 
-**Last Updated**: 2026-04-24
-**Version**: 3.3
+**Last Updated**: 2026-04-26
+**Version**: 3.4

.github/prompts/00-base-contract.md

@@ -55,11 +55,15 @@ Stage analysis + article.md + news/*.html → Commit → ONE create_pull_request
 
 ## Session keepalive requirement
 
-> ⚠️ **Critical**: The Copilot API creates a server-side session when the agent starts. That session is bound to the `github.token` baked in at step start — it is **never refreshed** mid-run. The session expires at approximately **60 minutes** (gh-aw issue #24920). After expiry, all tool calls and inference requests fail silently. The workflow appears to run but makes zero progress, and **the PR is never created**.
-
-To mitigate MCP idle-connection drops, workflows set `sandbox.mcp.keepalive-interval: 300` (5-minute ping). This keeps MCP connections alive but does **not** refresh the Copilot API token.
-
-**The reliable mitigation is to ensure `safeoutputs___create_pull_request` is called well before the session approaches expiry.** A second, shorter-firing clock — the Safe Outputs HTTP MCP idle session (~25–30 min observed) — now drives the operative deadline. Plan the run so the PR is created **within 22–27 minutes** (hard deadline **30 minutes**) of agent start. See `07-commit-and-pr.md §Deadline enforcement` for the authoritative PR-timing procedure, including the full two-timer explanation; that section supersedes any older ~45-minute guidance that predated the 23-artifact pipeline.
+> ⚠️ **Critical — three timers**: Plan every run for the **shortest** of the three.
+>
+> 1. **Job timeout (45 min)** — every news workflow declares `timeout-minutes: 45`. After 45 min the GitHub Actions runner kills the agent unconditionally.
+> 2. **Copilot API session (~60 min)** — bound to the `github.token` baked in at step start; never refreshed mid-run (gh-aw issue #24920). After expiry every tool call and inference fails silently.
+> 3. **Safe Outputs MCP idle session (~25–30 min observed)** — drops if the agent goes idle toward `safeoutputs___*` for 25+ minutes; every subsequent safe-output call returns `session not found`.
+>
+> The operative deadline is therefore Timer 3. To mitigate MCP-side idle drops, workflows set `sandbox.mcp.keepalive-interval: 300` (5-minute ping). That keeps upstream MCPs alive but does **not** refresh the Copilot session and does **not** keep the safeoutputs HTTP session alive.
+
+**The reliable mitigation is to ensure `safeoutputs___create_pull_request` is called well before the safeoutputs idle session approaches expiry.** Plan the run so the PR is created **within 22–27 minutes** (hard deadline **30 minutes**) of agent start. The remaining 15+ minutes of the 45-min job budget exist solely as a safety margin for the safeoutputs runner to publish the PR — do **not** schedule additional analysis after the PR call. See `07-commit-and-pr.md §Deadline enforcement` for the authoritative PR-timing procedure.
 
 Do not add per-phase checkpoint PRs or repo-memory push steps.
 

.github/prompts/02-mcp-access.md

@@ -6,29 +6,6 @@ Authoritative per-workflow surface: the `mcp-servers:` + `tools:` blocks in that
 
 News workflows declare three data MCP servers + the built-in `github` toolset (via `tools.github.toolsets: [all]`) + `bash` + `agentic-workflows`.
 
-| Server | Transport | Declared in | Tool-name style | Example tools |
-|--------|-----------|-------------|-----------------|---------------|
-| `riksdag-regering` | HTTP (Render) | workflow `mcp-servers:` | `snake_case` | `get_sync_status`, `search_dokument`, `get_voteringar`, `get_dokument_innehall` |
-| `scb` | container (`@jarib/pxweb-mcp`) | workflow `mcp-servers:` | `snake_case` | `search_tables`, `get_table_info`, `query_table` |
-| `world-bank` | container (`worldbank-mcp`) | workflow `mcp-servers:` | `kebab-case` | `get-economic-data`, `get-country-info`, `search-indicators` |
-| `github` | HTTP (Copilot MCP) | workflow `tools.github` | standard | full GitHub MCP toolset |
-| `bash` | local helper | workflow `tools.bash` | standard | shell execution |
-| `safeoutputs` | runner | always available | `snake_case` | `safeoutputs___create_pull_request`, `safeoutputs___noop`, `safeoutputs___dispatch_workflow` |
-
-`filesystem`, `memory`, and `sequential-thinking` are declared in [`.github/copilot-mcp.json`](../copilot-mcp.json) for the **local Copilot / `assign_copilot_to_issue`** channel and are **not** available to news workflows unless the workflow itself declares them under `mcp-servers:`.
-
-`playwright` must be treated separately: in news workflows it is available as the built-in workflow tool `tools.playwright` when that workflow declares it under `tools:` (e.g. `news-evening-analysis`, `news-realtime-monitor`). In that case it is **not** an MCP server, so do **not** infer its availability from `mcp-servers:` alone and do **not** skip Playwright/browser validation steps when `tools.playwright` is present in workflow frontmatter.
-
-Authoritative inventory: [`.github/copilot-mcp.json`](../copilot-mcp.json) for the local Copilot MCP surface, and each workflow's `mcp-servers:` plus `tools:` frontmatter for the actual per-run surface.
-
-# 02 — MCP Access
-
-Authoritative per-workflow surface: the `mcp-servers:` + `tools:` blocks in that workflow's frontmatter. `.github/copilot-mcp.json` is the **local Copilot** surface (used by `assign_copilot_to_issue` / agent files in `.github/agents/`), not by news workflow runs.
-
-## Servers & tool naming
-
-News workflows declare three data MCP servers + the built-in `github` toolset (via `tools.github.toolsets: [all]`) + `bash` + `agentic-workflows`.
-
 | Server | Transport | Declared in | Tool-name style | Example tools |
 |--------|-----------|-------------|-----------------|---------------|
 | `riksdag-regering` | HTTP (Render) | workflow `mcp-servers:` | `snake_case` | `get_sync_status`, `search_dokument`, `get_voteringar`, `get_dokument_innehall` |

.github/prompts/03-data-download.md

@@ -7,33 +7,53 @@ Run this check as the **first action** after MCP pre-warm, before any download:
 ```bash
 ANALYSIS_DIR="analysis/daily/$ARTICLE_DATE/$SUBFOLDER"
 
-# 23 required artifacts (Families A+B+C+D) — every workflow, every run
-REQ=(
-  # Family A — Core Synthesis (9)
-  README.md executive-brief.md synthesis-summary.md significance-scoring.md \
-  classification-results.md swot-analysis.md risk-assessment.md \
-  threat-analysis.md stakeholder-perspectives.md \
-  # Family B — Structural Metadata (2)
-  data-download-manifest.md cross-reference-map.md \
-  # Family C — Strategic Extensions (5)
-  scenario-analysis.md comparative-international.md devils-advocate.md \
-  intelligence-assessment.md methodology-reflection.md \
-  # Family D — Electoral & Domain Lenses (7)
-  election-2026-analysis.md voter-segmentation.md coalition-mathematics.md \
-  historical-parallels.md media-framing-analysis.md \
-  implementation-feasibility.md forward-indicators.md)
-
-# Tier-C workflows add no new files — all 23 are already mandatory. What Tier-C
-# adds is the cross-type synthesis + period multipliers enforced by
-# ext/tier-c-aggregation.md and the gate in 05-analysis-gate.md.
-
 SKIP_ANALYSIS=false
 ALL_PRESENT=true
-for f in "${REQ[@]}"; do
+EXPECTED=23
+CHECKED=0
+# 23 required artifacts (Families A+B+C+D) — every workflow, every run.
+# We feed them via a here-doc so the loop never builds an inline bash array
+# (the AWF sandbox flags `REQ=(...); for f in "${REQ[@]}"`; see
+# 01-bash-and-shell-safety.md §Banned expansion patterns).
+# The loop short-circuits on the first missing artifact, so $CHECKED is the
+# number examined up to that point — never larger than $EXPECTED. We log
+# both numbers so the difference is unambiguous in the run output.
+while IFS= read -r f; do
+  [ -z "$f" ] && continue
+  CHECKED=$((CHECKED + 1))
   [ -s "$ANALYSIS_DIR/$f" ] || { ALL_PRESENT=false; break; }
-done
+done <<'REQUIRED_ARTIFACTS'
+README.md
+executive-brief.md
+synthesis-summary.md
+significance-scoring.md
+classification-results.md
+swot-analysis.md
+risk-assessment.md
+threat-analysis.md
+stakeholder-perspectives.md
+data-download-manifest.md
+cross-reference-map.md
+scenario-analysis.md
+comparative-international.md
+devils-advocate.md
+intelligence-assessment.md
+methodology-reflection.md
+election-2026-analysis.md
+voter-segmentation.md
+coalition-mathematics.md
+historical-parallels.md
+media-framing-analysis.md
+implementation-feasibility.md
+forward-indicators.md
+REQUIRED_ARTIFACTS
+
+# Tier-C workflows add no new files — all 23 are already mandatory. What
+# Tier-C adds is the cross-type synthesis + period multipliers enforced by
+# ext/tier-c-aggregation.md and the gate in 05-analysis-gate.md.
+
 [ "$ALL_PRESENT" = "true" ] && SKIP_ANALYSIS=true
-echo "SKIP_ANALYSIS=$SKIP_ANALYSIS  (required artifacts present: $ALL_PRESENT, count: ${#REQ[@]})"
+echo "SKIP_ANALYSIS=$SKIP_ANALYSIS  (required artifacts present: $ALL_PRESENT, checked: $CHECKED of $EXPECTED)"
 ```
 
 | `SKIP_ANALYSIS` | Behaviour |
@@ -60,8 +80,7 @@ Populate `analysis/daily/$ARTICLE_DATE/$SUBFOLDER/` with raw Riksdag/Regering da
 | news-weekly-review | `weekly-review` |
 | news-monthly-review | `monthly-review` |
 | news-evening-analysis | `evening-analysis` |
-| news-realtime-monitor | `realtime-$HHMM` |
-| news-realtime-monitor | `realtime-pulse` |
+| news-realtime-monitor | `realtime-$HHMM` (per-event) or `realtime-pulse` (rolling 4-hour pulse) |
 
 If `force_generation=true` is supplied on a day whose base subfolder already contains `synthesis-summary.md` from a prior merged run, auto-suffix the subfolder (`propositions-2`, `propositions-3`, …) so the forced rerun does not overwrite the merged analysis. Under the default `force_generation=false`, the same base subfolder is reused across runs — see §Pre-flight above.
 
@@ -117,5 +136,3 @@ Always produce `analysis/daily/$ARTICLE_DATE/$SUBFOLDER/data-download-manifest.m
 ## Next step
 
 On success, proceed to `04-analysis-pipeline.md`. Never start analysis while `data-download-manifest.md` is missing or empty.
-
-After the manifest is written, run the **phase checkpoint** from `00-base-contract.md` with label `phase-03-download` so the download provenance is persisted to repo memory before any analysis begins.

.github/prompts/04-analysis-pipeline.md

@@ -154,8 +154,6 @@ For each article with charts, produce accompanying JSON under `analysis/daily/$A
 
 Proceed to `05-analysis-gate.md`. Do not start article generation until the gate passes against all 23 artifacts.
 
-After completing Pass 1 (before Pass 2), run the **phase checkpoint** from `00-base-contract.md` with label `phase-04-pass1`. After completing Pass 2 (before the gate), run it again with label `phase-04-pass2`. This guarantees both iterations survive even if the gate, article, or commit phase later fails.
-
 ## External references
 
 - gh-aw runtime (v0.69.3): [abridged](https://github.github.com/gh-aw/llms-small.txt) · [complete](https://github.github.com/gh-aw/llms-full.txt) · [agentic-workflows blog](https://github.github.com/gh-aw/_llms-txt/agentic-workflows.txt) · [source](https://github.com/github/gh-aw)