Skip to content

feat: Statskontoret 30-day cache module, feasibility contract tests, myndigheter skill update#2045

Merged
pethers merged 7 commits intomainfrom
copilot/add-ministerial-responses-coverage
Apr 27, 2026
Merged

feat: Statskontoret 30-day cache module, feasibility contract tests, myndigheter skill update#2045
pethers merged 7 commits intomainfrom
copilot/add-ministerial-responses-coverage

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 27, 2026

  • Audit existing codebase (workflows, scripts, tests, skills)
  • Confirm www.statskontoret.se already in all 11 news-*.md workflows (no change needed)
  • Create scripts/fetch-statskontoret.ts with 30-day TTL cache wrapper over StatskontoretClient
  • Update analysis/statskontoret/indicators-inventory.json to reference new cachedFetch module
  • Update .github/skills/myndigheter-monitoring/SKILL.md with Statskontoret integration section
  • Create tests/statskontoret-feasibility-contract.test.ts — contract tests asserting inventory coverage and module API behaviour
  • Extend tests/statskontoret-inventory.test.ts to verify cachedFetch inventory entry
  • Address code review (round 1): add units comment to CACHE_TTL_MS
  • Address code review (round 2):
    • Stamp provenance after fetch completes (not before)
    • Fix module-doc option name (cacheTtlMs not CacheTtlMs)
    • Remove invalid tsx fetch-statskontoret.ts CLI command from SKILL.md
    • Fix relative-link depth in SKILL.md References (../../../ not ../../)
    • Make per-file coverage test meaningful (record per-file matches; assert ≥1 file mentions an agency)
    • Rename FiU/KU committee test to reflect it's a global structural check

@github-actions github-actions Bot added the size-xs Extra small change (< 10 lines) label Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: size-xs

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…and myndigheter skill update

- scripts/fetch-statskontoret.ts: new 30-day TTL cache wrapper over
  StatskontoretClient for agentic workflows (fetchStatskontoretCached,
  isStatskontoretCacheFresh, statskontoretSourceKeys). Falls back to
  stale cache on network failure for resilience.

- tests/statskontoret-feasibility-contract.test.ts: 14 new Vitest
  contract tests asserting Statskontoret inventory provides myndighets-
  forteckning coverage for agencies named in implementation-feasibility.md
  files, plus full module API and cache behaviour tests.

- tests/statskontoret-inventory.test.ts: extend existing test to verify
  the new cachedFetch entry in indicators-inventory.json.

- analysis/statskontoret/indicators-inventory.json: add clients.cachedFetch
  field pointing to fetch-statskontoret.ts with 30-day TTL annotation.

- .github/skills/myndigheter-monitoring/SKILL.md: add Statskontoret Data
  Integration section with dataset table, CLI/module usage examples,
  data-provenance rule, and reference links.

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/d959ecd8-dfb3-4bf4-beec-a17db50ac538

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions github-actions Bot added documentation Documentation updates testing Test coverage refactor Code refactoring skill Skill configuration size-l Large change (250-1000 lines) labels Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI changed the title [WIP] Add coverage for ministerial responses and SD primary sources feat: Statskontoret 30-day cache module, feasibility contract tests, myndigheter skill update Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI requested a review from pethers April 27, 2026 01:03
@pethers pethers marked this pull request as ready for review April 27, 2026 08:05
Copilot AI review requested due to automatic review settings April 27, 2026 08:05
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Comment on lines +154 to +168
it('every feasibility file that names an agency also has a matching committee ' +
'in at least one Statskontoret dataset', () => {
// Collect all committees covered across all datasets.
const coveredCommittees = new Set<string>();
for (const dataset of Object.values(inv.datasets)) {
for (const committee of dataset.committees ?? []) {
coveredCommittees.add(committee);
}
}

// Files mentioning agencies related to FiU or KU should be coverable.
// This is a structural sanity check, not a per-file enforcement.
expect(coveredCommittees.has('FiU')).toBe(true);
expect(coveredCommittees.has('KU')).toBe(true);
});
Comment on lines +320 to +322
- [Statskontoret Indicators Inventory](../../analysis/statskontoret/indicators-inventory.json)
- [fetch-statskontoret.ts](../../scripts/fetch-statskontoret.ts) — 30-day cache module
- [statskontoret-client.ts](../../scripts/statskontoret-client.ts) — HTTP client library
Comment on lines +259 to +264
# List all available download links for the authority register (30-day cache)
tsx scripts/fetch-statskontoret.ts # (import fetchStatskontoretCached)

# CLI: discover downloadable files for a source
tsx scripts/statskontoret-fetch.ts discover --source myndighetsforteckning

Comment thread scripts/fetch-statskontoret.ts Outdated
*
* ### Cache behaviour
* - Cache root: `analysis/data/statskontoret/<sourceKey>/cache/`
* - TTL: 30 days (configurable via `CacheTtlMs`)
Comment on lines +157 to +186
const source = getStatskontoretSource(sourceKey);
const filePath = cacheFilePath(sourceKey, cacheRoot);
const now = new Date().toISOString();

// --- Cache hit ---
const cached = readCacheEntry(filePath);
if (cached !== undefined && isCacheFresh(cached.fetchedAt, cacheTtlMs)) {
const cacheAgeMs = Date.now() - new Date(cached.fetchedAt).getTime();
return {
sourceKey,
sourceTitle: source.title,
sourceUrl: source.url,
links: cached.links,
cachedAt: cached.fetchedAt,
fetchedAt: cached.fetchedAt,
fromCache: true,
cacheAgeMs,
};
}

// --- Cache miss or stale: fetch from origin ---
const client = new StatskontoretClient(clientConfig);
let links: StatskontoretDownloadLink[];
let fetchedAt: string;

try {
links = await client.discoverDownloads(sourceKey);
fetchedAt = now;
writeCacheEntry(filePath, { fetchedAt, sourceKey, links });
} catch (error) {
Comment on lines +128 to +152
it('for every implementation-feasibility.md that names a Swedish agency, ' +
'the inventory provides myndighetsforteckning coverage', () => {
// Since myndighetsforteckning covers ALL Swedish government bodies by
// definition, one dataset entry suffices for all named agencies.
const missingCoverage: string[] = [];

for (const filePath of feasibilityFiles) {
const content = fs.readFileSync(filePath, 'utf-8');
const mentioned = extractAgencyMentions(content, KNOWN_AGENCIES);

if (mentioned.length === 0) continue; // file mentions no known agencies → skip

// The inventory MUST contain myndighetsforteckning to cover any named agency.
if (!inv.datasets['myndighetsforteckning']) {
missingCoverage.push(
`${path.relative(REPO_ROOT, filePath)} mentions ${mentioned.join(', ')} but inventory has no myndighetsforteckning`,
);
}
}

expect(
missingCoverage,
`Statskontoret inventory is missing myndighetsforteckning coverage for:\n${missingCoverage.join('\n')}`,
).toHaveLength(0);
});
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 27, 2026

@copilot apply changes based on the comments in this thread

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

- scripts/fetch-statskontoret.ts: stamp `fetchedAt` after the network call
  completes (not before) so provenance reflects retrieval time; reuse the
  same value as `cachedAt` in the returned payload. Fix module-doc comment
  that said `CacheTtlMs` (now `cacheTtlMs`).

- .github/skills/myndigheter-monitoring/SKILL.md: remove fake
  `tsx scripts/fetch-statskontoret.ts` invocation (module is library, not
  CLI); replace with `list-sources` command on the actual CLI wrapper.
  Fix relative reference paths from `../../` to `../../../` so they
  resolve from `.github/skills/myndigheter-monitoring/` to the repo root.

- tests/statskontoret-feasibility-contract.test.ts: split the global
  per-file coverage test so it now (a) asserts at least one feasibility
  file references a known agency (no longer vacuous) and (b) records
  per-file coverage and reports any uncovered files. Rename the
  committee-coverage test to reflect that it is a global structural
  check, not a per-file assertion.

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/a2f9326c-b982-4c1d-80ef-66b79b4fe7a9

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers merged commit 851e871 into main Apr 27, 2026
16 checks passed
@pethers pethers deleted the copilot/add-ministerial-responses-coverage branch April 27, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation updates refactor Code refactoring size-l Large change (250-1000 lines) size-xs Extra small change (< 10 lines) skill Skill configuration testing Test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Methodology] Coverage expansion: ministerial responses to interpellations + SD primary-source monitoring

3 participants