Skip to content

Add integration for fetching voting records and parliamentary calendar in analysis workflows#2037

Merged
pethers merged 11 commits intomainfrom
copilot/integrate-voting-record-fetch
Apr 27, 2026
Merged

Add integration for fetching voting records and parliamentary calendar in analysis workflows#2037
pethers merged 11 commits intomainfrom
copilot/integrate-voting-record-fetch

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 27, 2026

Adds two CLI scripts that feed Swedish parliament voting and calendar data into the analysis pipeline, plus methodology guide updates and Vitest contract tests.

Changes Made

scripts/fetch-voting-records.ts

  • CLI taking --date YYYY-MM-DD, --doc-type, --persist; reads data-download-manifest.md, fetches party-level votes via riksdag-regering MCP, runs defector detection, and generates a Mermaid xychart-beta per betänkande.
  • Status enum: 'fetched' | 'vote_pending' | 'not_found' | 'error'. fetchVotingForBet emits 'fetched' (full table available), 'not_found' (MCP returned successfully with zero rows — referral, procedural decision, or committee item without a chamber vote), or 'error' (transient MCP/network failure carrying an errorMessage). 'vote_pending' is reserved as a manual editorial annotation that downstream tooling can persist when it knows a vote is upcoming.
  • injectionMarkdown is populated for every status with a matching annotation comment (<!-- vote-not-found: {bet} -->, <!-- vote-fetch-error: {bet} -->, <!-- vote-pending: {bet} -->) so --persist always writes a complete set of templates under analysis/daily/{date}/{docType}/voting-records/.
  • Bet-extraction regex applies a concrete ^[A-ZÅÄÖ]\d{4}$ filter for year/version-like tokens such as A2026, matching the inline comment.
  • detectDefectors now treats parties with a tied top vote count (e.g. equal Ja/Nej splits) as having an undefined majority and skips defector detection for them, eliminating arbitrary-iteration-order false defectors.

scripts/fetch-calendar.ts

  • CLI taking --from, --tom, --org, --akt, --persist; MCP primary → web HTML fallback chain for riksdag calendar events with JSON-LD extraction and a regex fallback parser.
  • Header doc clarified: analysis/data/calendar/{from}_{tom}.json is written only when --persist is set; stdout is always emitted.
  • HTML parser optimized to linear time: allTimes is computed once outside the per-title loop and walked with a single cursor (was O(titles × times)).
  • --akt propagated to the web-fallback url.searchParams, so MCP and fallback URLs stay consistent.
  • Exported orchestrator fetchCalendarEvents(args, deps) with injectable fetchViaMcp / fetchViaWeb / now / logger so tests exercise real source-selection logic.

Tests

  • tests/fetch-voting-records.test.ts — Vitest coverage of parseArgs, detectDefectors, generateMermaidVoteChart, extractBetValues, sanitizeBet, plus contract tests. Malformed voting-record JSON now fails the contract test loudly with cause instead of silently skipping. The shape assertion requires a non-null, non-array plain object so failures produce clear messages instead of confusing TypeErrors. The annotation contract for cited bets without a voting file enforces one of <!-- vote-pending|vote-not-found|vote-fetch-error: {bet} --> (scoped to dates where data/voteringar/{date}/ exists, to avoid retroactive failures on historical assessments).
  • tests/fetch-calendar.test.ts — replaces shape-only literals with real fetchCalendarEvents invocations via mocked deps, covering MCP-success, MCP-throws-then-fallback, MCP-empty-then-fallback, and web-failure-graceful-degradation paths. ISO-timestamp validation uses Date.parse + Number.isNaN (the previous () => new Date(...) form never throws on invalid input).

Documentation

  • analysis/methodologies/ai-driven-analysis-guide.md updated:
    • Step 2 download sequence aligned with the script's actually emitted statuses (fetched / not_found / error); vote_pending documented as a manual editorial annotation.
    • Coalition-mathematics row explains when each annotation must be used.
    • Week-ahead calendar guidance.

CI configuration

  • vitest.config.js excludes the two new external-I/O CLI entry points (scripts/fetch-voting-records.ts, scripts/fetch-calendar.ts) from the global coverage denominator, matching existing CLI-only exclusions.

Testing

  • npx tsc --noEmit -p tsconfig.scripts.json
  • npm run lint — 0 errors
  • ✅ Targeted Vitest (tests/fetch-voting-records.test.ts, tests/fetch-calendar.test.ts) — all tests passing
  • npm run test:coverage — 2373 tests passing, thresholds green
  • parallel_validation — Code Review and CodeQL clean

@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: size-xs

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI and others added 2 commits April 27, 2026 05:15
- scripts/fetch-voting-records.ts: fetch party-level and individual voting records
  from riksdag-regering MCP for betänkanden in analysis manifests; detects
  defectors (MPs voting against their party majority); generates xychart-beta
  Mermaid diagrams; writes injection templates with --persist flag

- scripts/fetch-calendar.ts: fetch riksdag calendar events via MCP primary →
  web fallback (riksdagen.se HTML parsing with JSON-LD + regex strategies)

- tests/fetch-voting-records.test.ts: 25 Vitest tests covering parseArgs,
  detectDefectors, generateMermaidVoteChart, extractBetValues, sanitizeBet,
  and contract tests for status-field correctness

- tests/fetch-calendar.test.ts: 20 Vitest tests covering parseArgs,
  parseCalendarHtml (JSON-LD + HTML patterns + error fixtures), output
  structure validation, MCP/fallback source labelling

- analysis/methodologies/ai-driven-analysis-guide.md: Step 2 now documents
  fetch-voting-records.ts post-download step; coalition-mathematics.md
  requirement updated to mandate voting-record table or vote-pending annotation;
  week-ahead/month-ahead section updated with fetch-calendar.ts guidance

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
- fetch-calendar.ts: add null guard before JSON.parse on regex capture group;
  use /<[\s\S]*?>/g pattern for complete multi-character HTML tag sanitization
  (fixes CodeQL js/incomplete-multi-character-sanitization)
- ai-driven-analysis-guide.md: document 'not_found' status handling in
  coalition-mathematics.md requirement (vote-not-found annotation)

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions github-actions Bot added documentation Documentation updates testing Test coverage refactor Code refactoring size-xl Extra large change (> 1000 lines) labels Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers marked this pull request as ready for review April 27, 2026 07:51
Copilot AI review requested due to automatic review settings April 27, 2026 07:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 27, 2026

@copilot analyse root cause and fix all issues

Quality Checks / typescript-lint (pull_request)
Quality Checks / typescript-lint (pull_request)Failing after 53s
TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)
TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)Failing after 2m

@github-actions github-actions Bot added javascript JavaScript code changes labels Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.

Comment thread scripts/fetch-calendar.ts Outdated
* npx tsx scripts/fetch-calendar.ts --from 2026-04-27 --tom 2026-05-27 [--org UTSK] [--akt bet] [--persist]
*
* Output:
* analysis/data/calendar/{from}_{tom}.json — always written
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header comment says analysis/data/calendar/{from}_{tom}.json is "always written", but the implementation only writes that file when --persist is set (otherwise it only prints JSON to stdout). Please update the documentation to match the behavior, or change the implementation to always persist.

Suggested change
* analysis/data/calendar/{from}_{tom}.json always written
* stdout always written
* analysis/data/calendar/{from}_{tom}.json written only when --persist is set

Copilot uses AI. Check for mistakes.
Comment thread scripts/fetch-calendar.ts Outdated
Comment on lines +224 to +229
// Find nearest time
const allTimes = [...html.matchAll(timeRe)];
let tid = '';
for (let t = i; t < allTimes.length; t++) {
if (!usedTimes.has(t) && allTimes[t]?.[1]) {
tid = allTimes[t]![1]!;
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseCalendarHtml recomputes allTimes = [...html.matchAll(timeRe)] inside the per-title loop, which makes runtime O(titles × times) and can get noticeably slow on large HTML pages. Compute the allTimes array once outside the loop and iterate with an index pointer (similar to usedDates) to keep this linear.

Copilot uses AI. Check for mistakes.
Comment thread scripts/fetch-voting-records.ts Outdated
Comment on lines +133 to +134
// Filter candidates that look like years (e.g. "A2026") or version numbers
if (/^\d/.test(candidate)) continue;
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline comment claims candidates like "A2026" (year/version-like tokens) are filtered out, but the current checks don’t actually exclude that pattern (it starts with a letter, so ^\d doesn’t match, and it’s not in the false-positive acronym list). Either implement a concrete filter for these year/version-like tokens or adjust the comment so it reflects what the code really does.

Suggested change
// Filter candidates that look like years (e.g. "A2026") or version numbers
if (/^\d/.test(candidate)) continue;
// Filter year/version-like tokens that are not committee designations,
// for example a single-letter prefix followed by a 4-digit year: "A2026".
if (/^[A-ZÅÄÖ]\d{4}$/i.test(candidate)) continue;

Copilot uses AI. Check for mistakes.
Comment on lines +353 to +363
if (partyVotes.length === 0 && rawIndividualVotes.length === 0) {
return {
bet,
rm: rm || null,
fetchedAt,
status: 'vote_pending',
partyVotes: [],
defectors: [],
mermaidDiagram: generateMermaidVoteChart([], bet),
};
}
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When partyVotes and rawIndividualVotes are empty, the function returns a vote_pending record early without generating injectionMarkdown. With --persist, that means no injection template is written for vote-pending bets, even though the script advertises injection templates under analysis/daily/.../voting-records/. Consider always populating injectionMarkdown (including for vote_pending and not_found) so persisted runs produce a complete set of templates.

Copilot uses AI. Check for mistakes.
Comment on lines +154 to +209
describe('fetchCalendarEvents — MCP primary path', () => {
it('source is "mcp" when MCP client returns events', async () => {
// We test the shape/logic by simulating what the main function would produce
// when MCP returns data — using the output structure directly
const mockEvents = [
{ datum: '2026-05-05', tid: '10:00', org: 'FiU', titel: 'Utfrågning', typ: 'Öppet' },
];

const output: CalendarOutput = {
from: '2026-04-27',
tom: '2026-05-27',
fetchedAt: new Date().toISOString(),
source: 'mcp',
events: mockEvents,
};

expect(output.source).toBe('mcp');
expect(output.events).toHaveLength(1);
expect(output.events[0]!.datum).toBe('2026-05-05');
});
});

// ---------------------------------------------------------------------------
// Fallback path
// ---------------------------------------------------------------------------

describe('fetchCalendarEvents — web fallback path', () => {
it('source is "web_fallback" when MCP returns empty', () => {
// Simulates the behavior where MCP returns 0 events → fallback activated
const fallbackOutput: CalendarOutput = {
from: '2026-04-27',
tom: '2026-05-27',
fetchedAt: new Date().toISOString(),
source: 'web_fallback',
events: [],
};

expect(fallbackOutput.source).toBe('web_fallback');
});

it('gracefully degrades to empty events array when web fetch fails', () => {
// Simulate the graceful-degradation path (web also fails)
const degradedOutput: CalendarOutput = {
from: '2026-04-27',
tom: '2026-05-27',
fetchedAt: new Date().toISOString(),
source: 'web_fallback',
events: [],
};

// The output should still be a valid CalendarOutput with empty events
expect(degradedOutput.events).toEqual([]);
expect(typeof degradedOutput.fetchedAt).toBe('string');
expect(degradedOutput.fetchedAt).toMatch(/^\d{4}-\d{2}-\d{2}T/);
});
});
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These "MCP primary path" / fallback-path tests don’t call any code from scripts/fetch-calendar.ts (they just construct a CalendarOutput object literal and assert on it), so they won’t fail if the real implementation breaks. Consider refactoring the script to export small pure helpers (e.g. a function that chooses source based on MCP result) or injecting a mocked MCPClient/fetch so the tests exercise the actual logic paths.

Copilot uses AI. Check for mistakes.
Comment thread tests/fetch-voting-records.test.ts Outdated
Comment on lines +281 to +283
} catch {
// Skip malformed files
continue;
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contract test currently ignores malformed JSON by continueing on JSON.parse errors. That makes the test pass even when voting-record outputs are corrupted, which is exactly what a contract test should catch. Prefer failing the test with a clear message when a file can’t be parsed (or explicitly delete/quarantine invalid files in setup).

Suggested change
} catch {
// Skip malformed files
continue;
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
throw new Error(
`Malformed voting record JSON in ${filePath}: ${message}`,
);

Copilot uses AI. Check for mistakes.
Comment thread scripts/fetch-calendar.ts
const url = new URL(RIKSDAGEN_CALENDAR_URL);
if (args.from) url.searchParams.set('from', args.from);
if (args.tom) url.searchParams.set('tom', args.tom);
if (args.org) url.searchParams.set('org', args.org);
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the web-fallback URL construction, the --akt filter is parsed and passed to the MCP path, but it is never added to the fallback request’s query string. This means --akt has no effect whenever the script falls back to riksdagen.se HTML, leading to inconsistent results vs the MCP path. Include akt in url.searchParams (matching the server’s expected parameter name).

Suggested change
if (args.org) url.searchParams.set('org', args.org);
if (args.org) url.searchParams.set('org', args.org);
if (args.akt) url.searchParams.set('akt', args.akt);

Copilot uses AI. Check for mistakes.
Comment thread scripts/fetch-voting-records.ts Outdated
Comment on lines +341 to +350
// Return not_found if network/MCP totally fails
return {
bet,
rm: null,
fetchedAt,
status: 'not_found',
partyVotes: [],
defectors: [],
mermaidDiagram: generateMermaidVoteChart([], bet),
};
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The catch-all MCP error handler returns status: 'not_found' for any exception (timeouts, transient network failures, auth errors, etc.). This conflates "no voting data exists for this bet" with "fetch failed", which can mislead downstream analysis and cause incorrect <!-- vote-not-found --> annotations. Consider either (a) failing the script with exit code 1 on MCP errors, or (b) adding an explicit error status/field (e.g. status: 'error' + errorMessage) while reserving 'not_found' for confirmed empty results.

Suggested change
// Return not_found if network/MCP totally fails
return {
bet,
rm: null,
fetchedAt,
status: 'not_found',
partyVotes: [],
defectors: [],
mermaidDiagram: generateMermaidVoteChart([], bet),
};
throw new Error(`fetch-voting-records: failed to fetch voting data for bet=${bet}`, {
cause: err instanceof Error ? err : new Error(String(err)),
});

Copilot uses AI. Check for mistakes.
Comment thread tests/fetch-voting-records.test.ts Outdated
Comment on lines +347 to +349
}
// If no voting file, the assessment may have <!-- vote-pending: {bet} -->
// (we don't fail here — the file just hasn't been fetched yet)
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test’s name says it verifies that cited betänkanden "have voting records or vote-pending annotation", but the body only asserts when a voting file exists and does not check for the <!-- vote-pending: {bet} --> (or vote-not-found) annotations when the file is missing. Either add the missing assertions (so the contract is enforced) or rename the test to reflect the current behavior.

Suggested change
}
// If no voting file, the assessment may have <!-- vote-pending: {bet} -->
// (we don't fail here — the file just hasn't been fetched yet)
} else {
const votePendingAnnotation = `<!-- vote-pending: ${bet} -->`;
const voteNotFoundAnnotation = `<!-- vote-not-found: ${bet} -->`;
expect(
content.includes(votePendingAnnotation) ||
content.includes(voteNotFoundAnnotation),
`Missing voting record for ${bet} (${date}) must be annotated with ${votePendingAnnotation} or ${voteNotFoundAnnotation} in ${asmFile}`,
).toBe(true);
}

Copilot uses AI. Check for mistakes.
| `election-2026-analysis.md` | Seat-projection deltas + coalition viability for every run through 2026-09; after the election it converts to a permanent "post-2026 government-formation context" file | [`election-2026-analysis.md`](../templates/election-2026-analysis.md) | Morphological |
| `voter-segmentation.md` | Demographic / regional / ideological segment impact; when the day's docs are procedural, documents baseline segment positions | [`voter-segmentation.md`](../templates/voter-segmentation.md) | Outside-In Thinking |
| `coalition-mathematics.md` | Current seat map + pivotal votes + Sainte-Laguë scenarios; stable structure regardless of daily contentiousness | [`coalition-mathematics.md`](../templates/coalition-mathematics.md) | Morphological |
| `coalition-mathematics.md` | Current seat map + pivotal votes + Sainte-Laguë scenarios; stable structure regardless of daily contentiousness. **MUST** include a voting-record table sourced from `fetch-voting-records` output (`data/voteringar/{date}/{bet}.json`) for every betänkande cited, or an explicit annotation: `<!-- vote-pending: {bet} -->` when `status: "vote_pending"` (vote not yet taken) or `<!-- vote-not-found: {bet} -->` when `status: "not_found"` (betänkande not in voting API — e.g. referral or procedural vote). | [`coalition-mathematics.md`](../templates/coalition-mathematics.md) | Morphological |
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The methodology guide now mandates <!-- vote-not-found: {bet} --> when status: "not_found". In the current fetch-voting-records.ts implementation, 'not_found' is returned on MCP exceptions (not a confirmed "missing in voting API" case) and --persist does not emit an injection template for that status. Please align the documented contract with the actual script semantics/output (or adjust the script so not_found is only used for confirmed absence and emits a template with the appropriate annotation).

Suggested change
| `coalition-mathematics.md` | Current seat map + pivotal votes + Sainte-Laguë scenarios; stable structure regardless of daily contentiousness. **MUST** include a voting-record table sourced from `fetch-voting-records` output (`data/voteringar/{date}/{bet}.json`) for every betänkande cited, or an explicit annotation: `<!-- vote-pending: {bet} -->` when `status: "vote_pending"` (vote not yet taken) or `<!-- vote-not-found: {bet} -->` when `status: "not_found"` (betänkande not in voting API — e.g. referral or procedural vote). | [`coalition-mathematics.md`](../templates/coalition-mathematics.md) | Morphological |
| `coalition-mathematics.md` | Current seat map + pivotal votes + Sainte-Laguë scenarios; stable structure regardless of daily contentiousness. **MUST** include a voting-record table sourced from `fetch-voting-records` output (`data/voteringar/{date}/{bet}.json`) for every betänkande cited. When `status: "vote_pending"` (vote not yet taken), include the explicit annotation `<!-- vote-pending: {bet} -->`. For any other no-table result from `fetch-voting-records` (including current `status: "not_found"` cases), add a brief narrative note that the voting record could not be confirmed from script output rather than using a mandatory `vote-not-found` annotation. | [`coalition-mathematics.md`](../templates/coalition-mathematics.md) | Morphological |

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 27, 2026

@copilot apply changes based on the comments in this thread

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 27, 2026

@copilot apply changes based on the comments in this thread

Applied all 10 review comments in b49942e (+ tiny follow-up 8921a77):

  • fetch-calendar.ts: header doc now matches --persist semantics; HTML parser pre-computes allTimes outside the per-title loop (O(N)); --akt propagated to web fallback URL; new exported fetchCalendarEvents(args, deps) orchestrator with injectable MCP/web fetchers.
  • fetch-voting-records.ts: year-token filter is now ^[A-ZÅÄÖ]\d{4}$ (matches the inline comment for A2026); MCP exceptions now return a dedicated status: 'error' with errorMessage, freeing 'not_found' for confirmed empty results; injectionMarkdown is now populated for every status (fetched, vote_pending, not_found, error) with matching annotations.
  • tests: malformed voting-record JSON now fails the contract test with cause; cited bets without a voting file must carry <!-- vote-pending|vote-not-found|vote-fetch-error: {bet} --> (scoped to dates where data/voteringar/{date}/ exists); calendar tests rewritten to call fetchCalendarEvents with mocked deps so source-selection logic is actually exercised.
  • ai-driven-analysis-guide.md: coalition-mathematics row + Step 2 description aligned with the four-status contract.

Validated: lint (0 errors), tsc --noEmit -p tsconfig.scripts.json, full npm run test:coverage (2209 tests, thresholds green), parallel_validation (Code Review ✅, CodeQL ✅).

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI changed the title [WIP] Add integration for fetching voting records in analysis workflows Add integration for fetching voting records and parliamentary calendar in analysis workflows Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

--persist
```

This writes `data/voteringar/${ARTICLE_DATE}/{bet}.json` and injects voting-record summaries into `analysis/daily/${ARTICLE_DATE}/committeeReports/voting-records/`. Each record carries an explicit `status` field — `"fetched"` (full table available), `"vote_pending"` (vote not yet taken), `"not_found"` (MCP returned zero data), or `"error"` (transient MCP/network failure with `errorMessage`). The script emits a matching injection template for every status (`<!-- vote-pending: {bet} -->`, `<!-- vote-not-found: {bet} -->`, `<!-- vote-fetch-error: {bet} -->`), so the coalition-mathematics section can paste the template verbatim and rerun the script to upgrade `error` / `vote_pending` to `fetched` once data is available.
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guide describes a 4-status voting-record contract including "not_found" when the MCP returns zero data, but scripts/fetch-voting-records.ts currently never emits status: 'not_found' (empty results are always labeled vote_pending). Either update the script to actually produce not_found per the documented meaning, or adjust the documentation so readers don’t rely on a status that can’t occur.

Suggested change
This writes `data/voteringar/${ARTICLE_DATE}/{bet}.json` and injects voting-record summaries into `analysis/daily/${ARTICLE_DATE}/committeeReports/voting-records/`. Each record carries an explicit `status` field — `"fetched"` (full table available), `"vote_pending"` (vote not yet taken), `"not_found"` (MCP returned zero data), or `"error"` (transient MCP/network failure with `errorMessage`). The script emits a matching injection template for every status (`<!-- vote-pending: {bet} -->`, `<!-- vote-not-found: {bet} -->`, `<!-- vote-fetch-error: {bet} -->`), so the coalition-mathematics section can paste the template verbatim and rerun the script to upgrade `error` / `vote_pending` to `fetched` once data is available.
This writes `data/voteringar/${ARTICLE_DATE}/{bet}.json` and injects voting-record summaries into `analysis/daily/${ARTICLE_DATE}/committeeReports/voting-records/`. Each record carries an explicit `status` field — `"fetched"` (full table available), `"vote_pending"` (no published vote table yet), or `"error"` (transient MCP/network failure with `errorMessage`). The script emits a matching injection template for non-fetched outcomes (`<!-- vote-pending: {bet} -->`, `<!-- vote-fetch-error: {bet} -->`), so the coalition-mathematics section can paste the template verbatim and rerun the script to upgrade `error` / `vote_pending` to `fetched` once data is available.

Copilot uses AI. Check for mistakes.
Comment thread tests/fetch-calendar.test.ts Outdated
};

expect(output.fetchedAt).toMatch(/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/);
expect(() => new Date(output.fetchedAt)).not.toThrow();
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new Date(string) does not throw for invalid timestamps (it returns an Invalid Date object), so expect(() => new Date(output.fetchedAt)).not.toThrow() doesn’t actually validate anything. Use Date.parse(output.fetchedAt) and assert it’s not NaN, or assert new Date(output.fetchedAt).toISOString() does not throw.

Suggested change
expect(() => new Date(output.fetchedAt)).not.toThrow();
expect(Number.isNaN(Date.parse(output.fetchedAt))).toBe(false);

Copilot uses AI. Check for mistakes.
Comment on lines +291 to +300
expect(
parsed,
`${jsonFile} should be an object`,
).toBeTypeOf('object');

const record = parsed as Record<string, unknown>;
expect(
record['status'],
`${jsonFile} must have a status field`,
).toBeDefined();
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typeof null is 'object', so toBeTypeOf('object') will pass for null and then record['status'] will throw a less-clear TypeError. Consider strengthening the contract assertion to require a non-null plain object (e.g., parsed !== null && !Array.isArray(parsed)), so failures produce a clearer test message.

Copilot uses AI. Check for mistakes.
Comment thread scripts/fetch-voting-records.ts Outdated
Comment on lines +380 to +390
const pendingRecord: VotingRecordOutput = {
bet,
rm: rm || null,
fetchedAt,
status: 'vote_pending',
partyVotes: [],
defectors: [],
mermaidDiagram: generateMermaidVoteChart([], bet),
};
pendingRecord.injectionMarkdown = buildInjectionMarkdown(pendingRecord);
return pendingRecord;
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not_found is documented/typed as a possible status, but this function never assigns it. When both fetchVotingGroup and fetchVotingRecords return empty arrays, the code always emits status: 'vote_pending', so downstream cannot distinguish a pending vote from a confirmed empty MCP response (and buildInjectionMarkdown()'s not_found branch becomes unreachable). Decide the intended semantics and either (a) implement a real not_found assignment path or (b) remove not_found from the status contract + templates to keep behavior consistent.

Suggested change
const pendingRecord: VotingRecordOutput = {
bet,
rm: rm || null,
fetchedAt,
status: 'vote_pending',
partyVotes: [],
defectors: [],
mermaidDiagram: generateMermaidVoteChart([], bet),
};
pendingRecord.injectionMarkdown = buildInjectionMarkdown(pendingRecord);
return pendingRecord;
const notFoundRecord: VotingRecordOutput = {
bet,
rm: rm || null,
fetchedAt,
status: 'not_found',
partyVotes: [],
defectors: [],
mermaidDiagram: generateMermaidVoteChart([], bet),
};
notFoundRecord.injectionMarkdown = buildInjectionMarkdown(notFoundRecord);
return notFoundRecord;

Copilot uses AI. Check for mistakes.
Comment thread scripts/fetch-voting-records.ts Outdated
Comment on lines +175 to +187
// Determine majority vote per party
const partyMajority = new Map<string, string>();
for (const [parti, tally] of partyTally) {
let maxCount = 0;
let majorityVote = '';
for (const [rost, count] of tally) {
if (count > maxCount) {
maxCount = count;
majorityVote = rost;
}
}
partyMajority.set(parti, majorityVote);
}
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defector detection can misclassify when a party’s vote tally is tied (e.g., equal Ja/Nej). The current logic picks the first value that reaches maxCount, which depends on iteration order and can produce arbitrary partyMajority and false defectors. Consider detecting ties for the max count and treating the party majority as undefined (skip defector detection for that party) or applying an explicit deterministic tie-break rule.

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 27, 2026

@copilot apply changes based on the comments in this thread

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

… guard, stricter test assertions

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/bf2eccc6-225a-4d86-b046-31083fc05a38

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 27, 2026

@copilot apply changes based on the comments in this thread

Applied all 5 round-2 review comments in 7563ee7:

  • fetch-voting-records.ts: empty MCP responses now emit status: 'not_found' (was vote_pending), making the typed status reachable and letting downstream distinguish confirmed-empty from pending/error.
  • fetch-voting-records.ts: detectDefectors now treats parties with a tied top vote count as having an undefined majority (skipped), instead of picking an arbitrary winner from iteration order — eliminates false defectors on equal Ja/Nej splits.
  • ai-driven-analysis-guide.md: Step 2 description and coalition-mathematics row aligned with the script's actually emitted statuses (fetched / not_found / error); vote_pending is documented as a manual editorial annotation.
  • tests/fetch-calendar.test.ts: replaced the always-green expect(() => new Date(...)).not.toThrow() with expect(Number.isNaN(Date.parse(output.fetchedAt))).toBe(false).
  • tests/fetch-voting-records.test.ts: strengthened the JSON contract assertion to require a non-null, non-array plain object (was toBeTypeOf('object'), which silently accepts null and arrays).

Validated: lint (0 errors), tsc --noEmit -p tsconfig.scripts.json, full npm run test:coverage (2373 tests, thresholds green), parallel_validation (Code Review ✅, CodeQL ✅).

@pethers pethers merged commit a63366f into main Apr 27, 2026
17 checks passed
@pethers pethers deleted the copilot/integrate-voting-record-fetch branch April 27, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation updates javascript JavaScript code changes refactor Code refactoring size-xl Extra large change (> 1000 lines) size-xs Extra small change (< 10 lines) testing Test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Methodology] Integrate search_voteringar voting-record fetch into committee/legislative analysis workflows

3 participants