Skip to content

Commit 48aeea8

Browse files
committed
test(jelly-micro): validate RECALL_FLOORS keys against discovered fixtures
Add a startup check after discoverTests() that throws if any key in RECALL_FLOORS does not match an actual fixture directory. A renamed or mistyped fixture key would otherwise silently lower the recall floor to 0 with no warning, defeating the regression gate. The check is gated on tests.length > 0 so it does not fire in CI environments where the jelly-micro directory is gitignored.
1 parent b7553ca commit 48aeea8

1 file changed

Lines changed: 19 additions & 2 deletions

File tree

tests/benchmarks/resolution/jelly-micro.test.ts

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,8 +54,6 @@ function discoverTests(): string[] {
5454
.sort();
5555
}
5656

57-
const tests = discoverTests();
58-
5957
/**
6058
* Per-fixture minimum recall floors based on the baseline measured on origin/main
6159
* (commit 784951d, June 2026). Fixtures not listed here default to 0 — they
@@ -84,6 +82,25 @@ const RECALL_FLOORS: Record<string, number> = {
8482
this: 1.0, // 1/1
8583
};
8684

85+
const tests = discoverTests();
86+
87+
// Sanity-check: every RECALL_FLOORS key must match a discovered fixture name.
88+
// A mismatch means either a fixture was renamed or the key was mistyped, and
89+
// the regression floor would silently degrade to 0 without this guard.
90+
// Only enforce when the fixture directory is present (CI skips the whole suite
91+
// when fixtures are absent).
92+
if (tests.length > 0) {
93+
const testSet = new Set(tests);
94+
for (const key of Object.keys(RECALL_FLOORS)) {
95+
if (!testSet.has(key)) {
96+
throw new Error(
97+
`RECALL_FLOORS key "${key}" does not match any discovered fixture in ${FIXTURES_DIR}. ` +
98+
'Update the key or remove the stale entry.',
99+
);
100+
}
101+
}
102+
}
103+
87104
// Per-test results collected for summary
88105
const allResults: Record<
89106
string,

0 commit comments

Comments
 (0)