test: CI coverage gate, harden command assertions, injectable health timeout

- Enforce coverage thresholds in CI (npm run test:coverage)
- Replace 12 bare .toHaveBeenCalled() in commands.test.ts with specific checks
- Add healthCheckTimeout to CustomConfig (default 5s, injectable for tests)
- Custom backend test suite drops from ~5.2s to ~0.4s

Author: AdeAnima

.github/workflows/ci.yml

@@ -24,8 +24,8 @@ jobs:
       - name: Install dependencies
         run: npm ci
 
-      - name: Run tests
-        run: npm test
+      - name: Run tests with coverage
+        run: npm run test:coverage
 
       - name: Build
         run: npm run build

src/backends/custom.ts

@@ -23,7 +23,7 @@ export class CustomBackend implements TtsBackend {
     await new Promise<void>((resolve, reject) => {
       const req = client.get(
         `${url.origin}/health`,
-        { timeout: 5000 },
+        { timeout: this.config.healthCheckTimeout ?? 5000 },
         (res) => {
           res.resume();
           if (res.statusCode === 200) resolve();

src/types.ts

@@ -57,6 +57,8 @@ export interface F5Config {
 /** Custom HTTP endpoint backend configuration. */
 export interface CustomConfig {
   endpoint: string;
+  /** Health-check timeout in ms. Defaults to 5000 (5 s). */
+  healthCheckTimeout?: number;
   /** Synthesis HTTP request timeout in ms. Defaults to 120000 (2 min). */
   synthesisTimeout?: number;
 }

test-suite-review-v2-2026-02-20-k9m4t1.md

@@ -0,0 +1,128 @@
+# Test Suite Review V2 — Combined Findings (2026-02-20)
+
+## Documents Merged
+
+- `a1b2c3-test-suite-review.md`
+- `test-suite-review-2026-02-20-a7k3p9.md`
+
+## Executive Summary
+
+The test suite is already strong and well-structured, with high coverage and stable green runs. The highest-value improvements are not broad rewrites, but targeted hardening in four areas: CI quality gates, assertion precision, slow timeout-path tests, and selected uncovered orchestration/platform branches.
+
+## Current State Snapshot
+
+- Test files: 16
+- Tests: 245
+- Status: 245/245 passing
+- Runtime: ~5.5s
+- Coverage:
+  - Statements: 95.64%
+  - Branches: 87.12%
+  - Functions: 93.79%
+  - Lines: 97.05%
+
+## Consolidated Strengths
+
+- Clear module-based test organization (commands, setup, provider, chunker, backends).
+- Good mix of unit and integration-style tests.
+- Existing coverage of cancellation/error behavior in critical paths.
+- Extension workflow behaviors are exercised via VS Code mocks.
+
+## Consolidated Gaps
+
+### 1) CI Gate Incomplete (Process Risk)
+
+- CI currently runs tests/build/typecheck but does not enforce coverage thresholds via coverage run.
+- Impact: coverage regressions can merge undetected.
+
+### 2) Assertion Strictness (Quality Risk)
+
+- Some tests rely on permissive assertions (for example: broad “called” checks, `>= 1` chunk expectations).
+- Impact: behavior regressions may pass despite changing semantics.
+
+### 3) Timeout Test Cost (Speed/Flake Risk)
+
+- `customBackend` timeout-path tests consume several seconds due to real timeout waits.
+- Impact: slower feedback loops and greater CI variability.
+
+### 4) Targeted Branch Gaps (Coverage Quality Risk)
+
+- Remaining uncovered lines are concentrated in orchestration/platform and edge handling branches.
+- High-value targets:
+  - `src/commands.ts` wrapper/error branches
+  - `src/setup.ts` URL validation / custom endpoint path
+  - `src/extension.ts` runtime reconfiguration branches
+  - `src/player.ts` and `src/installer.ts` platform-specific branches
+  - `src/chunker.ts` buffer-full/backpressure wait branch
+  - `src/backends/kokoro.ts` abort-after-generation branch
+
+### 5) Mock Fidelity and Real Host Validation (Confidence Risk)
+
+- Mock-driven integration tests are good but cannot fully replicate Extension Host behavior.
+- Child process and filesystem behavior can be tested with higher-fidelity simulation.
+
+## Final Prioritized Plan
+
+## P0 — Immediate
+
+1. Enforce coverage in CI
+- Add `npm run test:coverage` to CI workflow for PR gating.
+
+2. Harden permissive assertions in high-risk tests
+- Focus first on:
+  - `test/chunker.test.ts`
+  - `test/extensionIntegration.test.ts`
+  - `test/commands.test.ts`
+
+## P1 — Next
+
+3. Optimize timeout-path tests
+- Prefer injectable short timeout values in tests for determinism.
+- Optional: fake timers only where request/response behavior remains deterministic.
+- Target: no single timeout-path unit test > ~250ms.
+
+4. Close highest-value branch gaps
+- Add focused tests for specific uncovered orchestration/platform branches listed above.
+
+5. Improve mock fidelity where behavior matters
+- Expand process lifecycle simulation (`SIGSTOP`/`SIGCONT`) and filesystem interaction checks in backend flow tests.
+
+## P2 — Maturity
+
+6. Add true VS Code E2E smoke tests
+- Introduce `@vscode/test-electron` (Insiders-targeted) for minimal smoke coverage:
+  - activation
+  - command registration
+  - status bar visibility
+  - walkthrough open behavior
+
+7. Pilot mutation testing on critical modules
+- Scope first pass to small set (`chunker`, `commands`, `speechProvider`) and track mutation score trend.
+
+## Recommended Implementation Sequence (Low-Risk, High-ROI)
+
+1. CI coverage gate
+2. Assertion hardening pass
+3. Timeout-path speedup
+4. Branch-gap closure pass
+5. E2E smoke introduction
+6. Mutation pilot
+
+## Success Criteria
+
+- Coverage thresholds enforced in CI for pull requests.
+- Reduced suite runtime and no multi-second single-test bottlenecks in routine local runs.
+- Measurable branch coverage increase in orchestration/platform modules.
+- Fewer permissive assertions in critical-path tests.
+- Basic real Extension Host E2E smoke tests passing.
+
+## Sources
+
+1. Vitest Coverage Guide (official): https://vitest.dev/guide/coverage.html  
+   Credibility: 10/10 (official, current, directly applicable)
+
+2. VS Code Extension Testing Guide (official): https://code.visualstudio.com/api/working-with-extensions/testing-extension  
+   Credibility: 10/10 (official Microsoft guidance, current)
+
+3. Stryker Mutation Testing Docs (official): https://stryker-mutator.io/docs/  
+   Credibility: 8/10 (official tool docs, valuable for mutation strategy)

test/commands.test.ts

@@ -136,7 +136,7 @@ describe("commands", () => {
       disableTts(services);
 
       expect(services.speechRegistration).toBeUndefined();
-      expect(backend.dispose).toHaveBeenCalled();
+      expect(backend.dispose).toHaveBeenCalledTimes(1);
     });
 
     it("is safe when no backend or registration", () => {
@@ -163,7 +163,7 @@ describe("commands", () => {
 
       togglePause(services);
 
-      expect(spy).toHaveBeenCalled();
+      expect(spy).toHaveBeenCalledTimes(1);
     });
   });
 
@@ -216,7 +216,9 @@ describe("commands", () => {
 
       await readSelectionAloud(services);
 
-      expect(mockPlayerPlay).toHaveBeenCalled();
+      expect(mockPlayerPlay).toHaveBeenCalledWith(
+        expect.objectContaining({ sampleRate: 24000 })
+      );
     });
   });
 
@@ -230,7 +232,7 @@ describe("commands", () => {
 
       await initializeAndRegister(context, services, backend);
 
-      expect(backend.initialize).toHaveBeenCalled();
+      expect(backend.initialize).toHaveBeenCalledTimes(1);
       expect(vscode.speech.registerSpeechProvider).toHaveBeenCalledWith(
         "eloquent",
         services.provider
@@ -261,7 +263,7 @@ describe("commands", () => {
 
       await initializeAndRegister(context, services, backend);
 
-      expect(oldDispose).toHaveBeenCalled();
+      expect(oldDispose).toHaveBeenCalledTimes(1);
     });
 
     it("sets status bar to loading then active", async () => {
@@ -285,7 +287,9 @@ describe("commands", () => {
       // testVoice plays audio
       await initializeAndRegister(context, services, fakeBackend());
 
-      expect(mockPlayerPlay).toHaveBeenCalled();
+      expect(mockPlayerPlay).toHaveBeenCalledWith(
+        expect.objectContaining({ sampleRate: 24000 })
+      );
     });
   });
 
@@ -298,7 +302,9 @@ describe("commands", () => {
 
       await testVoice(services, fakeBackend());
 
-      expect(mockPlayerPlay).toHaveBeenCalled();
+      expect(mockPlayerPlay).toHaveBeenCalledWith(
+        expect.objectContaining({ sampleRate: 24000 })
+      );
       expect(spy).toHaveBeenCalledWith(true); // finally block
     });
 
@@ -340,7 +346,7 @@ describe("commands", () => {
       const services = makeServices();
       await setupBackend(makeContext(), services);
 
-      expect(backend.initialize).toHaveBeenCalled();
+      expect(backend.initialize).toHaveBeenCalledTimes(1);
       expect(services.speechRegistration).toBeDefined();
     });
   });
@@ -353,7 +359,7 @@ describe("commands", () => {
 
       await enableTts(makeContext(), makeServices());
 
-      expect(mockRunSetupWizard).toHaveBeenCalled();
+      expect(mockRunSetupWizard).toHaveBeenCalledTimes(1);
     });
 
     it("creates and initializes configured backend", async () => {
@@ -365,7 +371,7 @@ describe("commands", () => {
       await enableTts(makeContext(), services);
 
       expect(mockCreateBackend).toHaveBeenCalledWith("kokoro", expect.anything());
-      expect(backend.initialize).toHaveBeenCalled();
+      expect(backend.initialize).toHaveBeenCalledTimes(1);
     });
   });
 
@@ -391,7 +397,7 @@ describe("commands", () => {
       await toggleTts(makeContext(), makeServices());
 
       // Falls through to enableTts → setupBackend because no backend configured
-      expect(mockRunSetupWizard).toHaveBeenCalled();
+      expect(mockRunSetupWizard).toHaveBeenCalledTimes(1);
     });
   });
 
@@ -428,7 +434,7 @@ describe("commands", () => {
       await changeVoice(makeContext(), services);
 
       expect(mockCreateBackend).toHaveBeenCalledWith("kokoro", expect.anything());
-      expect(backend.initialize).toHaveBeenCalled();
+      expect(backend.initialize).toHaveBeenCalledTimes(1);
     });
   });
 });

test/customBackend.test.ts

@@ -169,16 +169,17 @@ describe("CustomBackend", () => {
     });
 
     try {
-      const backend = new CustomBackend({ endpoint: hangUrl });
-      // The health check has a 5s timeout — but we can't wait that long.
-      // Verify the timeout path by checking the error message.
+      const backend = new CustomBackend({
+        endpoint: hangUrl,
+        healthCheckTimeout: 200,
+      });
       await expect(backend.initialize()).rejects.toThrow(
         "Custom TTS endpoint timed out"
       );
     } finally {
       hangServer.close();
     }
-  }, 10_000);
+  });
 
   it("selects https client for https:// endpoints", async () => {
     // Use an HTTPS URL pointing at a port that will refuse the connection.