Skip to content

Commit dd9d7d8

Browse files
test: fixture-count floor assertion in conformance runner
Guard against the fail-open case where the runner discovers no (or too few) shared fixtures and passes green having verified almost nothing. If the sibling gcf fixture set is present but yields fewer than 150 files (mispathed/partial/ empty checkout), fail loudly. A wholly-absent checkout keeps its existing skip (local ergonomics); in CI the separate gcf checkout step fails loudly if it cannot clone. Current discovered count is 198.
1 parent f6777ee commit dd9d7d8

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

tests/test_conformance_v2.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,22 @@ def _load_fixtures():
2222

2323
FIXTURES = _load_fixtures()
2424

25+
MIN_FIXTURES = 150
26+
27+
28+
def test_minimum_fixtures_discovered():
29+
"""Floor assertion: a green conformance run MUST have exercised the full shared
30+
suite. The parametrized test below skips when FIXTURES is empty, so a mispathed or
31+
partial checkout would otherwise pass having verified almost nothing. A wholly-absent
32+
directory skips (local ergonomics); in CI the separate gcf checkout step fails loudly
33+
if the repo cannot be cloned."""
34+
if not FIXTURE_DIR.exists():
35+
pytest.skip("conformance fixtures not found")
36+
assert len(FIXTURES) >= MIN_FIXTURES, (
37+
f"discovered only {len(FIXTURES)} conformance fixtures, expected at least "
38+
f"{MIN_FIXTURES}; the shared gcf fixture set is incomplete or mispathed"
39+
)
40+
2541

2642
def _json_norm(v):
2743
return json.loads(json.dumps(v))

0 commit comments

Comments
 (0)