You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to closed issue #461 / PR #474. The OSINT QA contract suite landed (tests/integration/osint/contract.test.ts β envelope, no-silent-zero, determinism), INTEGRATION_TESTING.md and CONTRIBUTING.md were updated, and ARCHITECTURE.md now links the suite. However, three deliverables from the original #461 acceptance criteria did not ship:
Mutation testing baseline (Stryker, β₯70 % mutation score on the 15 OSINT tool files) β package.json has no test:mutation script; no stryker.config.json exists. Code search confirms Stryker is only mentioned aspirationally in FUTURE_WORKFLOWS.md.
Per-file coverage thresholds (β₯90 % for OSINT tools, β₯95 % for DOCEO-touching files) β vitest.config.ts only enforces the global lines: 80; there is no coverage.thresholds.perFile entry.
Dedicated CI job for the OSINT QA harness β neither osint-qa.yml nor an extension of test-and-report.yml runs the contract suite as a discrete job; it executes inside the general integration test pass without isolated reporting.
The contract test suite alone catches structural violations but cannot detect logic regressions in scoring/anomaly methodology β exactly the kind of regressions the seven companion correctness PRs (#467, #469, #472, #473, #476, #477, #479) introduced surface area for. Mutation testing + tight per-file thresholds close that gap.
π― Objectives
Add Stryker (@stryker-mutator/core, @stryker-mutator/vitest-runner) as a dev dependency.
stryker.config.json scoped to the 15 OSINT tool files + their shared utilities (src/utils/votingBaseline.ts, src/utils/graphAlgorithms.ts, src/utils/networkVotingSimilarity.ts, src/utils/effectivenessAggregator.ts, src/utils/lifecycleStatistics.ts, src/utils/doceoMepAggregator.ts, src/utils/politicalGroupNormalization.ts).
Initial mutation-score threshold informational (thresholds: { high: 80, low: 60, break: null }); promote break: 70 once one green run is observed on main.
npm run test:mutation script and matching osint-mutation job in CI.
Extend vitest.config.ts with coverage.thresholds.perFile entries:
Extract the contract suite into its own CI job (osint-qa) so failures are visibly attributed and golden-snapshot drift surfaces in PR checks.
Make the mutation job non-blocking initially (status check required: false), then promote in a follow-up PR after baseline stability is observed.
π¦ Deliverables
package.json β new devDependencies, test:mutation script, version bump in release-notes not required (CI-only change).
stryker.config.json at repo root scoped per above.
vitest.config.ts extended with per-file thresholds and an opt-in --changed-files mode for fast PR runs.
.github/workflows/osint-qa.yml (or extension of test-and-report.yml) with three sequential jobs: osint-contract, osint-coverage, osint-mutation (the last marked continue-on-error: true for the first 30 days).
CONTRIBUTING.md β section explaining how to run the mutation suite locally and what surviving-mutant categories are acceptable (e.g. log-string mutants).
INTEGRATION_TESTING.md β appendix on mutation-testing scope and exclusions.
π Security & Compliance
Supply Chain: Stryker packages must pass the existing npm run test:licenses allow-list (MIT/Apache-2.0/BSD/ISC) β verify before adding.
CI Resource Bound: scope must keep total mutation runtime under 15 minutes; use --concurrency 4 and exclude *.test.ts.
ISMS Reference: A.8.29 (Security testing in development & acceptance), A.8.34 (Protection during audit testing).
Compliance: ISO 27001, SLSA Level 3 (provenance covers test artefacts).
ποΈ Technical Approach
Run gh-advisory-database against the two Stryker packages before adding to lockfile.
Scope Stryker mutate glob to the 15 OSINT tool files and 7 shared utilities; ignorePatterns for tests and generated.
vitest-runner keeps the existing test infrastructure with no per-test fork overhead.
Per-file coverage uses Vitest 3's thresholds.perFile map syntax; CI prints offenders in a digestible table.
The new workflow uses actions/setup-node with the pinned SHA pattern from test-and-report.yml.
π§ͺ Testing Requirements
Mutation run on main completes with score β₯60 % (baseline) before the threshold is bumped.
Per-file coverage gate fails CI when any OSINT tool drops below its assigned threshold.
osint-qa workflow visible as a discrete check on every PR touching src/tools/ or src/utils/votingBaseline.ts/graphAlgorithms.ts/etc.
Test runtime budget: contract β€2 min, coverage β€3 min, mutation β€15 min.
ARCHITECTURE.md β reference to mutation testing as the test-quality enforcement point.
π€ Recommended Agent
Agent: hack23-test-specialist (with hack23-devops-engineer for the CI workflow wiring) Rationale: This is a pure QA-infrastructure task β mutation testing scope, Vitest threshold configuration, and CI job extraction map directly onto the test-specialist's remit; DevOps owns the workflow file.
π Task Overview
Follow-up to closed issue #461 / PR #474. The OSINT QA contract suite landed (
tests/integration/osint/contract.test.tsβ envelope, no-silent-zero, determinism),INTEGRATION_TESTING.mdandCONTRIBUTING.mdwere updated, andARCHITECTURE.mdnow links the suite. However, three deliverables from the original #461 acceptance criteria did not ship:package.jsonhas notest:mutationscript; nostryker.config.jsonexists. Code search confirms Stryker is only mentioned aspirationally inFUTURE_WORKFLOWS.md.vitest.config.tsonly enforces the globallines: 80; there is nocoverage.thresholds.perFileentry.osint-qa.ymlnor an extension oftest-and-report.ymlruns the contract suite as a discrete job; it executes inside the general integration test pass without isolated reporting.The contract test suite alone catches structural violations but cannot detect logic regressions in scoring/anomaly methodology β exactly the kind of regressions the seven companion correctness PRs (#467, #469, #472, #473, #476, #477, #479) introduced surface area for. Mutation testing + tight per-file thresholds close that gap.
π― Objectives
@stryker-mutator/core,@stryker-mutator/vitest-runner) as a dev dependency.stryker.config.jsonscoped to the 15 OSINT tool files + their shared utilities (src/utils/votingBaseline.ts,src/utils/graphAlgorithms.ts,src/utils/networkVotingSimilarity.ts,src/utils/effectivenessAggregator.ts,src/utils/lifecycleStatistics.ts,src/utils/doceoMepAggregator.ts,src/utils/politicalGroupNormalization.ts).thresholds: { high: 80, low: 60, break: null }); promotebreak: 70once one green run is observed onmain.npm run test:mutationscript and matchingosint-mutationjob in CI.vitest.config.tswithcoverage.thresholds.perFileentries:src/tools/{the 14 non-DOCEO OSINT tools}.tsβ{ lines: 90, branches: 85, functions: 90, statements: 90 }.src/tools/assessMepInfluence.ts,src/tools/detectVotingAnomalies.ts,src/tools/sentimentTracker.ts,src/tools/networkAnalysis.ts,src/tools/analyzeCoalitionDynamics.ts(DOCEO-touching) β{ lines: 95, branches: 90, functions: 95, statements: 95 }.osint-qa) so failures are visibly attributed and golden-snapshot drift surfaces in PR checks.required: false), then promote in a follow-up PR after baseline stability is observed.π¦ Deliverables
package.jsonβ new devDependencies,test:mutationscript, version bump inrelease-notesnot required (CI-only change).stryker.config.jsonat repo root scoped per above.vitest.config.tsextended with per-file thresholds and an opt-in--changed-filesmode for fast PR runs..github/workflows/osint-qa.yml(or extension oftest-and-report.yml) with three sequential jobs:osint-contract,osint-coverage,osint-mutation(the last markedcontinue-on-error: truefor the first 30 days).CONTRIBUTING.mdβ section explaining how to run the mutation suite locally and what surviving-mutant categories are acceptable (e.g. log-string mutants).INTEGRATION_TESTING.mdβ appendix on mutation-testing scope and exclusions.π Security & Compliance
npm run test:licensesallow-list (MIT/Apache-2.0/BSD/ISC) β verify before adding.--concurrency 4and exclude*.test.ts.ποΈ Technical Approach
gh-advisory-databaseagainst the two Stryker packages before adding to lockfile.mutateglob to the 15 OSINT tool files and 7 shared utilities;ignorePatternsfor tests and generated.vitest-runnerkeeps the existing test infrastructure with no per-test fork overhead.thresholds.perFilemap syntax; CI prints offenders in a digestible table.actions/setup-nodewith the pinned SHA pattern fromtest-and-report.yml.π§ͺ Testing Requirements
maincompletes with score β₯60 % (baseline) before the threshold is bumped.osint-qaworkflow visible as a discrete check on every PR touchingsrc/tools/orsrc/utils/votingBaseline.ts/graphAlgorithms.ts/etc.π Documentation Updates
CONTRIBUTING.mdβ mutation testing instructions.INTEGRATION_TESTING.mdβ OSINT QA appendix.WORKFLOWS.mdβ newosint-qaworkflow entry.ARCHITECTURE.mdβ reference to mutation testing as the test-quality enforcement point.π€ Recommended Agent
Agent:
hack23-test-specialist(withhack23-devops-engineerfor the CI workflow wiring)Rationale: This is a pure QA-infrastructure task β mutation testing scope, Vitest threshold configuration, and CI job extraction map directly onto the test-specialist's remit; DevOps owns the workflow file.
π Related Issues
monitor_legislative_pipelinecold-cache flakeΒ #473, OSINT: DOCEO-backed voting anomaly detection with z-score baselinesΒ #476, OSINT: voting-similarity edges, depth-bounded BFS, and weighted centrality innetwork_analysisΒ #477, Make sentiment_tracker timeframe functional with DOCEO cohesion-driven scoringΒ #479 (seven OSINT correctness/completeness PRs landed without mutation-test protection).vitest.config.ts:thresholds,tests/integration/osint/contract.test.ts.π ISMS References