0.5.37 - 🧪 Lock in #120 fix (HALF_OPEN regression) + score split

Bundle the v0.5.36 router priority fix with two regression tests and a
scoreCandidates refactor that prevents #120 from silently regressing and
removes the priority-mixed-into-score footgun for future code.

- scoreCandidates: score is now pure latency+uptime (0.5/0.5 weights,
  normalized [0,1]); priorityBonus kept as a separate field for back-
  compat dashboards. No more priorityWeight confusing the tiebreaker.
- getRoutingCandidates: unchanged (already priority-first from 0.5.36).
- DEFAULT_ROUTER_SETTINGS.scoring.priorityWeight preserved for back-
  compat but ignored at runtime; will be removed in a future major.
- Tests: +2 regression tests for issue #120 (HALF_OPEN scenario from the
  user screenshot + score tiebreaker determinism). 540 → 542 passing.

Author: vava-nessa

changelog/v0.5.37.md

@@ -0,0 +1,14 @@
+# Changelog v0.5.37 - 2026-06-20
+
+### Changed
+- **Router `score` is now a pure latency+uptime composite (issue #120 hardening).** Previously the routing score was `0.4*latency + 0.4*uptime + 0.2*priorityBonus` — mixing explicit priority into the score could mislead tiebreakers and dashboards, because the routing comparator in `getRoutingCandidates` already enforces priority authoritatively. `scoreCandidates()` now exposes `score = 0.5*latency + 0.5*uptime` (pure model quality, normalized to `[0, 1]`), and `priorityBonus` is kept as a separate field for back-compat dashboards and legacy UIs. The remaining `latencyWeight` / `uptimeWeight` were rebalanced to `0.5 / 0.5` so the score stays in `[0, 1]` after priority is removed. Applies uniformly to CLI, Web Dashboard, and Desktop surfaces because the change lives in shared core (`src/core/router-daemon.js` + `src/core/config.js`).
+
+### Fixed
+- **HALF_OPEN recovery is now respected by the priority-first comparator (issue #120 regression test).** The v0.5.36 fix changed `getRoutingCandidates` to sort by explicit priority before circuit state, but had no test for the exact scenario from the issue screenshot: a high-priority model sitting in `HALF_OPEN` recovery vs. a lower-priority `CLOSED` model. Added a regression test that pins `runtime.circuit.get(key).state = 'HALF_OPEN'` for priority `#1` while priority `#5` stays `CLOSED`, then verifies both `/stats.routingOrder[0]` and the actual chat-completions response target the HALF_OPEN priority-#1 model — locking in the fix so it can't silently regress.
+- **Score tiebreaker is deterministic when two models share the same priority (issue #120 regression test).** Added a second regression test that puts two models at the same explicit priority (rare but reachable via direct API or auto-heal) with deliberately asymmetric probe data — fast groq (80 ms) vs. slow nvidia (2000 ms) — and verifies that the higher-score model wins `routingOrder[0]`. This locks in the new pure-latency+uptime score as the deterministic tiebreaker for same-priority same-state candidates, preventing future regressions where Map iteration order or stale priority data could leak back into routing.
+
+### Docs
+- **`DEFAULT_ROUTER_SETTINGS.scoring.priorityWeight` marked as preserved-for-back-compat.** The field still round-trips through `normalizeRouterScoring()` so user configs that customize it are not silently dropped on next save, but it is now ignored by the runtime `scoreCandidates()`. Will be removed in a future major bump. Comment block in `src/core/config.js` documents the rationale.
+
+### Tests
+- **+2 new tests for issue #120** (`test/test.js`, `router daemon integration hardening` suite): `keeps a higher-priority HALF_OPEN model above a lower-priority CLOSED one` and `breaks score ties deterministically by latency/uptime, not priority`. Test count moves from 540 → 542, all passing.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "free-coding-models",
-  "version": "0.5.36",
+  "version": "0.5.37",
   "description": "Find the fastest coding LLM models in seconds — ping free models from multiple providers, pick the best one for OpenCode, Cursor, or any AI coding assistant.",
   "keywords": [
     "nvidia",

src/core/config.js

@@ -167,8 +167,15 @@ export const DEFAULT_ROUTER_SETTINGS = Object.freeze({
     requestTimeoutMs: 15000,
   }),
   scoring: Object.freeze({
-    latencyWeight: 0.4,
-    uptimeWeight: 0.4,
+    latencyWeight: 0.5,
+    uptimeWeight: 0.5,
+    // 📖 priorityWeight is preserved for back-compat with user configs that
+    // 📖 set a custom value, but is no longer mixed into the routing score
+    // 📖 (issue #120 fix, v0.5.37). Priority is now enforced authoritatively
+    // 📖 by the comparator in getRoutingCandidates; folding it into score
+    // 📖 could only mislead tiebreakers and dashboards. The remaining
+    // 📖 latency+uptime weights are rebalanced to 0.5/0.5 so the score stays
+    // 📖 in [0, 1] after priority is removed (previously 0.4/0.4/0.2).
     priorityWeight: 0.2,
   }),
   logLevel: 'info',
@@ -377,6 +384,10 @@ function normalizeRouterScoring(scoring) {
     const numeric = Number(value)
     return Number.isFinite(numeric) && numeric >= 0 ? numeric : fallback
   }
+  // 📖 priorityWeight is normalized but currently IGNORED by the runtime
+  // 📖 scoreCandidates() (issue #120, v0.5.37). Kept here so existing user
+  // 📖 configs that customize this field round-trip cleanly and don't lose
+  // 📖 their setting on next save. Will be removed in a future major bump.
   return {
     latencyWeight: numberOrDefault(safeScoring.latencyWeight, DEFAULT_ROUTER_SETTINGS.scoring.latencyWeight),
     uptimeWeight: numberOrDefault(safeScoring.uptimeWeight, DEFAULT_ROUTER_SETTINGS.scoring.uptimeWeight),

src/core/router-daemon.js

@@ -1149,15 +1149,27 @@ class RouterRuntime {
       const hasData = stats.total > 0
       const latencyScore = stats.p95 === null ? 0.5 : Math.max(0, 1 - (stats.p95 / maxP95))
       const uptimeScore = stats.uptime === null ? 0.5 : stats.uptime
+      // 📖 priorityBonus - kept as a separate field for dashboards/legacy UIs
+      // 📖 that previously rendered a single composite "score". Priority is
+      // 📖 NOT folded into `score` anymore (issue #120): the routing comparator
+      // 📖 in getRoutingCandidates sorts by explicit priority authoritatively,
+      // 📖 so mixing priority into the score only confused tiebreakers.
       const priorityBonus = 1 - ((entry.priority - 1) / setSize)
+      // 📖 score - pure latency+uptime composite. Used only as the FINAL
+      // 📖 tiebreaker between candidates that share the same priority AND
+      // 📖 same circuit state (see getRoutingCandidates). A model with no
+      // 📖 probe data yet scores neutral (0.5) - we deliberately do NOT use
+      // 📖 priorityBonus as a cold-start fallback, because that would re-
+      // 📖 introduce the priority-in-score confusion this refactor removes.
       const score = hasData
-        ? (weights.latencyWeight * latencyScore) + (weights.uptimeWeight * uptimeScore) + (weights.priorityWeight * priorityBonus)
-        : priorityBonus
+        ? (weights.latencyWeight * latencyScore) + (weights.uptimeWeight * uptimeScore)
+        : 0.5
       const state = this.updateCircuitForCooldown(key) || {}
       return {
         ...entry,
         key,
         score,
+        priorityBonus,
         stats,
         circuit: state,
         catalog: this.modelCatalog.get(key) || null,

test/test.js

@@ -3119,6 +3119,108 @@ describe('router daemon integration hardening', () => {
     })
   })
 
+  // 📖 Regression for issue #120 — HALF_OPEN recovery must NOT be skipped.
+  // 📖 The screenshot from the user showed a HALF_OPEN priority-#1 model
+  // 📖 being skipped for a CLOSED priority-#20 model. The priority-first
+  // 📖 comparator in getRoutingCandidates now guarantees explicit priority
+  // 📖 wins over circuit state, so a high-priority model mid-recovery is
+  // 📖 still tried before any lower-priority CLOSED fallback.
+  it('keeps a higher-priority HALF_OPEN model above a lower-priority CLOSED one (issue #120)', async () => {
+    await withMockProvider(() => ({
+      body: { id: 'chatcmpl-halfopen', choices: [{ message: { role: 'assistant', content: 'from-halfopen' } }] },
+    }), async (groqProvider) => {
+      await withMockProvider(() => ({
+        body: { id: 'chatcmpl-closed', choices: [{ message: { role: 'assistant', content: 'from-closed' } }] },
+      }), async (nvidiaProvider) => {
+        await withSourceUrls({ groq: groqProvider.url, nvidia: nvidiaProvider.url }, async () => {
+          const config = buildRouterTestConfig([
+            { provider: 'groq', model: ROUTER_TEST_MODELS.groqFast, priority: 1 },
+            { provider: 'nvidia', model: ROUTER_TEST_MODELS.nvidiaFast, priority: 5 },
+          ])
+          await withRouterTestServer(config, async ({ baseUrl, runtime }) => {
+            // 📖 Force groq (priority #1) into HALF_OPEN recovery. nvidia
+            // 📖 (priority #5) stays CLOSED. The OLD code would have sorted
+            // 📖 HALF_OPEN AFTER CLOSED regardless of priority; the NEW
+            // 📖 comparator must keep groq at the top.
+            const groqKey = `groq/${ROUTER_TEST_MODELS.groqFast}`
+            const groqCircuit = runtime.circuit.get(groqKey)
+            assert.ok(groqCircuit, 'groq circuit entry must exist')
+            groqCircuit.state = 'HALF_OPEN'
+            groqCircuit.openedAt = Date.now() - 60_000
+
+            // 📖 routingOrder surface: groq (HALF_OPEN, priority 1) MUST be
+            // 📖 before nvidia (CLOSED, priority 5) — issue #120's exact case.
+            const stats = await (await fetch(`${baseUrl}/stats`)).json()
+            assert.equal(stats.routingOrder[0].key, groqKey,
+              'priority-#1 HALF_OPEN must come before priority-#5 CLOSED')
+            assert.equal(stats.routingOrder[0].state, 'HALF_OPEN')
+            assert.equal(stats.routingOrder[1].key, `nvidia/${ROUTER_TEST_MODELS.nvidiaFast}`)
+            assert.equal(stats.routingOrder[1].state, 'CLOSED')
+
+            // 📖 End-to-end check: a real chat-completions request hits groq
+            // 📖 (the HALF_OPEN priority-#1), NOT nvidia (the CLOSED priority-#5).
+            const response = await postRouterChat(baseUrl)
+            assert.equal(response.status, 200)
+            assert.equal(response.headers.get('x-fcm-router-model'), groqKey)
+            assert.equal(groqProvider.requests.length, 1)
+            assert.equal(nvidiaProvider.requests.length, 0)
+          })
+        })
+      })
+    })
+  })
+
+  // 📖 Regression for issue #120 — score tiebreaker determinism.
+  // 📖 When two candidates share the same explicit priority AND the same
+  // 📖 circuit state, the comparator falls back to score. With the v0.5.37
+  // 📖 refactor, score is pure latency+uptime (no priorityWeight) so the
+  // 📖 tiebreaker is deterministic and reflects actual model quality.
+  it('breaks score ties deterministically by latency/uptime, not priority (issue #120)', async () => {
+    await withMockProvider(() => ({
+      body: { id: 'chatcmpl-winner', choices: [{ message: { role: 'assistant', content: 'high-score' } }] },
+    }), async (groqProvider) => {
+      await withMockProvider(() => ({
+        body: { id: 'chatcmpl-loser', choices: [{ message: { role: 'assistant', content: 'low-score' } }] },
+      }), async (nvidiaProvider) => {
+        await withSourceUrls({ groq: groqProvider.url, nvidia: nvidiaProvider.url }, async () => {
+          // 📖 Two models deliberately at the SAME priority (rare but
+          // 📖 possible via direct API or auto-heal). Tiebreaker must fall
+          // 📖 back to score (latency+uptime), not priority (they're equal),
+          // 📖 not random Map iteration order.
+          const config = buildRouterTestConfig([
+            { provider: 'groq', model: ROUTER_TEST_MODELS.groqFast, priority: 1 },
+            { provider: 'nvidia', model: ROUTER_TEST_MODELS.nvidiaFast, priority: 1 },
+          ])
+          await withRouterTestServer(config, async ({ baseUrl, runtime }) => {
+            // 📖 Build a clear asymmetry: groq has fast probes (high score),
+            // 📖 nvidia has slow probes (low score). Same priority, same state.
+            const groqKey = `groq/${ROUTER_TEST_MODELS.groqFast}`
+            const nvidiaKey = `nvidia/${ROUTER_TEST_MODELS.nvidiaFast}`
+            for (let i = 0; i < 6; i++) runtime.recordProbeResult(groqKey, { ok: true, latencyMs: 80, code: 200 })
+            for (let i = 0; i < 6; i++) runtime.recordProbeResult(nvidiaKey, { ok: true, latencyMs: 2000, code: 200 })
+
+            const stats = await (await fetch(`${baseUrl}/stats`)).json()
+            assert.equal(stats.routingOrder[0].key, groqKey,
+              'higher-score model must win same-priority same-state tiebreaker')
+            assert.equal(stats.routingOrder[1].key, nvidiaKey)
+
+            // 📖 priorityBonus is still exposed separately for back-compat
+            // 📖 dashboards, but is NOT mixed into the routing score.
+            const groqHealth = stats.models.find((m) => m.key === groqKey)
+            const nvidiaHealth = stats.models.find((m) => m.key === nvidiaKey)
+            assert.ok(groqHealth.score > nvidiaHealth.score,
+              `groq score (${groqHealth.score}) must exceed nvidia (${nvidiaHealth.score})`)
+
+            // 📖 End-to-end: the chat-completions request hits the high-
+            // 📖 score groq model.
+            const response = await postRouterChat(baseUrl)
+            assert.equal(response.headers.get('x-fcm-router-model'), groqKey)
+          })
+        })
+      })
+    })
+  })
+
   it('returns precise quota metadata when every routed model is exhausted', async () => {
     await withMockProvider(() => ({
       status: 429,