Skip to content

feat: add scb-fetch.ts, riksbank-fetch.ts CLI wrappers + IMF provider precedence contract tests#2039

Merged
pethers merged 4 commits intomainfrom
copilot/add-polling-vintage-gate
Apr 27, 2026
Merged

feat: add scb-fetch.ts, riksbank-fetch.ts CLI wrappers + IMF provider precedence contract tests#2039
pethers merged 4 commits intomainfrom
copilot/add-polling-vintage-gate

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 27, 2026

Methodology hardening: stops WB/SCB/Riksbank data being mis-attributed as IMF. Adds two new CLI pipeline scripts and a contract test suite that locks in the economicProvenance provider hierarchy from ECONOMIC_DATA_CONTRACT.md v2.1 (imf > scb > riksbank > worldBank).

New scripts

scripts/scb-fetch.ts

Preset-based CLI wrapper around SCBClient for PXWeb MCP queries. Exposes curated KPI/AKU/household-economy/fuel-price presets plus a free-form table command. Every response includes an economicProvenance block — provider is always "scb", never "imf".

tsx scripts/scb-fetch.ts list-presets
tsx scripts/scb-fetch.ts preset --preset cpi [--persist]
tsx scripts/scb-fetch.ts table --table-id TAB5765 --value-codes '{"Tid":"top(12)"}' [--persist]

scripts/riksbank-fetch.ts

Web-fetch CLI wrapper for public Riksbank pages and documents. Fetches the policy-rate path, monetary-policy minutes, and fuel-price context from www.riksbank.se. provider is always "riksbank".

tsx scripts/riksbank-fetch.ts repo-rate-path [--persist]
tsx scripts/riksbank-fetch.ts minutes [--persist]
tsx scripts/riksbank-fetch.ts fuel-price-context [--persist]
tsx scripts/riksbank-fetch.ts fetch --kind repo-rate-path --url https://www.riksbank.se/en-gb/...

Both scripts emit a structured economicProvenance block on every response:

{
  "provider": "scb",
  "economicProvenance": {
    "provider": "scb",          // or "riksbank" — never "imf"
    "dataflow": "SCB PxWeb",    // or "riksbank-web"
    "indicator": "0000003N",
    "retrieved_at": "2026-04-27T00:58:00.000Z"
  }
}

New test: tests/imf-precedence-contract.test.ts (48 tests)

Locks in the provider contract using the current APIs:

  • ImfDataPoint.provider is always "imf" — TypeScript literal type asserted at runtime
  • SCBEconomicProvenance.provider is "scb" — never "imf", "riksbank", or "worldBank"
  • RiksbankEconomicProvenance.provider is "riksbank" — never "imf" or "scb"
  • fetchSCBTablePayload emits provider: "scb" even on network fail-soft (no-data path)
  • fetchRiksbankPayload emits provider: "riksbank" on all response types (HTML, JSON, PDF)
  • analysis/economic-indicators-inventory.json v4.1: macro, fiscal, monetary, externalSector domains resolve to "imf"; swedenPrimary resolves to "scb"
  • Deprecated WB codes (NY.GDP.MKTP.KD.ZG, FP.CPI.TOTL.ZG, SL.UEM.TOTL.ZS) point to imf:WEO:* replacements
  • CLI parsing for both scripts fully exercised including assertRiksbankFetchTarget (HTTPS + allowlist enforcement)

vitest.config.js

No additional coverage exclusions required — both scripts are covered by the dedicated tests/scb-fetch.test.ts and tests/riksbank-fetch.test.ts test files already present in the codebase.

@github-actions github-actions Bot added the size-xs Extra small change (< 10 lines) label Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: size-xs

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…t tests

- scripts/scb-fetch.ts: CLI wrapper around SCBClient with economicProvenance (provider: scb)
- scripts/riksbank-fetch.ts: CLI wrapper for Riksbank SWEA API with economicProvenance (provider: riksbank)
- tests/imf-precedence-contract.test.ts: 49 tests enforcing IMF precedence, SCB/Riksbank never aliased as IMF
- vitest.config.js: exclude new CLI scripts from coverage

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/6f638578-1618-4277-b44e-adab68cda0f6

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions github-actions Bot added javascript JavaScript code changes testing Test coverage refactor Code refactoring size-xl Extra large change (> 1000 lines) labels Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI changed the title [WIP] Add polling vintage gate and multi-source poll aggregator feat: add scb-fetch.ts, riksbank-fetch.ts CLI wrappers + IMF provider precedence contract tests Apr 27, 2026
Copilot AI requested a review from pethers April 27, 2026 01:05
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers marked this pull request as ready for review April 27, 2026 08:04
Copilot AI review requested due to automatic review settings April 27, 2026 08:04
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds new CLI wrappers and a Vitest contract suite to harden the “economicProvenance provider” hierarchy (IMF-first, with SCB and Riksbank as distinct non-IMF providers) and prevent mis-attribution across data sources.

Changes:

  • Added scripts/scb-fetch.ts CLI wrapper emitting economicProvenance.provider = "scb" on all outputs.
  • Added scripts/riksbank-fetch.ts CLI wrapper for SWEA API, emitting economicProvenance.provider = "riksbank" on all outputs.
  • Added tests/imf-precedence-contract.test.ts to lock in provider precedence and inventory expectations; updated Vitest coverage exclusions accordingly.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
vitest.config.js Excludes the two new CLI scripts from coverage calculations (consistent with other CLI-only scripts).
tests/imf-precedence-contract.test.ts Adds contract tests for provider identity/invariants, CLI arg parsing, and inventory/provider-selection expectations.
scripts/scb-fetch.ts Introduces SCB PXWeb CLI wrapper with provenance wrapping + optional persistence.
scripts/riksbank-fetch.ts Introduces Riksbank SWEA CLI wrapper (fetch + timeout) with provenance wrapping + persistence.

Comment thread scripts/scb-fetch.ts Outdated
Comment on lines +225 to +227
if (booleans.has('persist')) {
persistSCBData(`search-${query.replace(/\s+/g, '-')}`, payload, { query, limit });
}
Comment thread scripts/scb-fetch.ts Outdated
process.stdout.write(`${JSON.stringify(payload, null, 2)}\n`);

if (booleans.has('persist')) {
persistSCBData(tableId, payload, { tableId });
Comment thread scripts/riksbank-fetch.ts Outdated
Comment on lines +43 to +46
import { pathToFileURL } from 'node:url';
import path from 'node:path';
import fs from 'node:fs';
import { fileURLToPath } from 'node:url';
Comment thread scripts/riksbank-fetch.ts Outdated
Comment on lines +28 to +30
* `--persist` writes the raw response under `analysis/data/riksbank/` via
* {@link persistRiksbankData}. Outputs JSON (with `economicProvenance`) to
* stdout regardless.
Comment thread scripts/riksbank-fetch.ts Outdated
Comment on lines +344 to +348
function resolveDataRoot(): string {
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const repoRoot = path.resolve(__dirname, '..');
return path.join(repoRoot, 'analysis', 'data');
}
Comment thread scripts/scb-fetch.ts Outdated
Comment on lines +22 to +24
* `--persist` writes the raw response under `analysis/data/scb/` via
* {@link persistSCBData}. Outputs JSON (with `economicProvenance`) to
* stdout regardless.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 27, 2026

@copilot resolve the merge conflicts in this pull request

Resolve add/add conflicts in scb-fetch.ts and riksbank-fetch.ts:
- Take main's preset-based scb-fetch.ts (KPI/AKU/HEK/fuel-price presets)
- Take main's web-fetch riksbank-fetch.ts (repo-rate-path/minutes/fuel-price-context)
- Update imf-precedence-contract.test.ts to use new APIs
- Remove coverage exclusions for scb-fetch/riksbank-fetch (covered by new test files)

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 27, 2026

@copilot resolve the merge conflicts in this pull request

Merge conflicts resolved in commit 59071d5. The branch had add/add conflicts in scripts/scb-fetch.ts and scripts/riksbank-fetch.ts (main had merged a more complete implementation from another PR). Resolution took main's preset-based scb-fetch.ts and web-fetch riksbank-fetch.ts, then updated tests/imf-precedence-contract.test.ts to use the new APIs. All 2306 tests pass.

@github-actions github-actions Bot added size-l Large change (250-1000 lines) labels Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers merged commit c30d511 into main Apr 27, 2026
17 checks passed
@pethers pethers deleted the copilot/add-polling-vintage-gate branch April 27, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

javascript JavaScript code changes refactor Code refactoring size-l Large change (250-1000 lines) size-xl Extra large change (> 1000 lines) size-xs Extra small change (< 10 lines) testing Test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Methodology] Polling-vintage gate + multi-source poll aggregator (Novus / Demoskop / Ipsos / SCB)

3 participants