Skip to content

Commit 523cd3d

Browse files
committed
chore(governance): sync holdout tracking metrics in QUALITY_GATES
Update Holdout Tracking paragraph to 101/14/35 fixture minimums and add anti-drift test so stale 100-phrase wording cannot regress.
1 parent 2b9ad3a commit 523cd3d

2 files changed

Lines changed: 16 additions & 1 deletion

File tree

reflective-prompt-library/plans/QUALITY_GATES_SUMMARY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,7 @@ The latest router improvement uses concept-level boundary rules for these cases
304304

305305
### Holdout Tracking
306306

307-
ROUTE-002 measures unseen phrasing separately from ROUTE-001. Round 7 (2026-06-25) added Traditional Chinese holdout groups with matching router intent keywords — fairness test without full `SKILL.md` translation. Round 65 expanded to 36 groups / 100 phrases; post-panel maintenance added ROUTING_CONTRACT **R11** (approved-spec delivery) with ROUTE-003 `implement_not_plan_trap` at 100%. Treat this as a seeded holdout, not proof of broad semantic routing; add fresh cases before further router tuning.
307+
ROUTE-002 measures unseen phrasing separately from ROUTE-001. Round 7 (2026-06-25) added Traditional Chinese holdout groups with matching router intent keywords — fairness test without full `SKILL.md` translation. Round 65 expanded to 36 holdout groups; post-Round 68 maintenance added ROUTING_CONTRACT **R11** boundaries (101 ROUTE-002 phrases, 14 ROUTE-003 adversarial groups / 35 phrases including `implement_not_plan_trap`, `approved_spec_plan_not_implement_trap`, and `dispatch_meta_skill_trap`) at 100%. Treat this as a seeded holdout, not proof of broad semantic routing; add fresh cases before further router tuning.
308308

309309
## Phase 2 Status (post-Round 68 maintenance)
310310

reflective-prompt-library/plans/tests/test_quality_gates_summary.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,21 @@ def test_ci_paraphrase_totals_match_route_minimums(summary_text: str):
8686
)
8787
assert expected_total in summary_text
8888

89+
90+
def test_holdout_tracking_paragraph_matches_fixture_minimums(summary_text: str):
91+
section = _section_between(
92+
summary_text, "### Holdout Tracking", "## Phase 2 Status (post-Round 68 maintenance)"
93+
)
94+
assert f"{ROUTE_002_MIN_PHRASES} ROUTE-002 phrases" in section
95+
assert (
96+
f"{ROUTE_003_MIN_ADVERSARIAL_GROUPS} ROUTE-003 adversarial groups / "
97+
f"{ROUTE_003_MIN_PHRASES} phrases"
98+
) in section
99+
assert "approved_spec_plan_not_implement_trap" in section
100+
assert "dispatch_meta_skill_trap" in section
101+
assert "100 phrases" not in section
102+
103+
89104
def test_route_fixture_gate_mentions_cheatsheet_parity_tests(summary_text: str):
90105
section = _section_between(
91106
summary_text,

0 commit comments

Comments
 (0)