Skip to content

Commit 16787b2

Browse files
committed
chore(governance): sync cheatsheet parity for post-holdout probes
Add EN/zh-TW cheatsheet cues for ROUTE-002 design-comparison holdout and ROUTE-003 dispatch-meta trap; export probe tuples with anti-drift tests. Refresh QUALITY_GATES pytest floor (180+) with collection guard.
1 parent 523cd3d commit 16787b2

7 files changed

Lines changed: 96 additions & 2 deletions

reflective-prompt-library/PROJECT_KNOWLEDGE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ deferred promotions are recurrence-gated — see [panel backlog](plans/multi-age
8181
- 2026-06-18 Adopt STORM's source-grounded perspective discovery as an optional `reflective-research` method, not a new skill → [record](plans/storm-perspective-discovery-reflection-2026-06-18.md)
8282
- 2026-06-20 Synthesize external-adoption case studies, then correct the evidence and promotion-gate scope after the Test Plan routing counterexample → [record](plans/external-adoption-case-studies-2026-06-20.md)
8383
- 2026-06-21 Evaluate Hyperplan / multi-agent adversarial planning — no change (runtime = non-goal; methodology overlaps; possible gaps not promoted) → [record](plans/external-adoption-case-studies-2026-06-20.md)
84+
- 2026-06-25 Post-Round 68 cheatsheet parity — ROUTE-002 design-comparison holdout phrase + ROUTE-003 dispatch-meta probe in EN/zh-TW cheatsheets → [QUALITY_GATES_SUMMARY.md](plans/QUALITY_GATES_SUMMARY.md)
8485
- 2026-06-25 Post-Round 68 holdout expansion — ROUTE-002 design comparison phrase; ROUTE-003 approved-spec plan-only trap + dispatch routing probe → [ROUTING_CONTRACT.md](plans/ROUTING_CONTRACT.md#r11-approved-spec-delivery)
8586
- 2026-06-25 Round 68 panel — full-doc drift sync, README/methodology-map anti-drift, reject holdout expansion at 100% → [record](plans/multi-agent-panel-consensus-2026-06-25.md)
8687
- 2026-06-25 Post-Round 68 maintenance — nine-skill wording parity (METHODOLOGY_MAP, skill-map, QUALITY_GATES), benchmark B024 dispatch (24 tasks, 9/9 workflows) → [QUALITY_GATES_SUMMARY.md](plans/QUALITY_GATES_SUMMARY.md)

reflective-prompt-library/plans/QUALITY_GATES_SUMMARY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -314,7 +314,7 @@ ROUTE-002 measures unseen phrasing separately from ROUTE-001. Round 7 (2026-06-2
314314
2. **ROUTE-001/002/003 in CI** — 128 + 101 + 35 paraphrases at 100% consistency (seeded fixtures); `validate_route_fixture.py` gates minimum coverage
315315
3. **Governance validators** — links, lint, governance metadata, PROJECT_KNOWLEDGE, benchmark fixture, skill examples
316316
4. **Harness policy docs** — CONTRIBUTING, AGENTS, SKILL_INSTALLATION, maintenance playbook
317-
5. **Doc anti-drift**`test_routing_contract.py`, cheatsheet parity tests, `test_readme_governance.py` (170+ pytest in CI)
317+
5. **Doc anti-drift**`test_routing_contract.py`, cheatsheet parity tests, `test_readme_governance.py` (180+ pytest anti-drift suite in CI)
318318

319319
### Ongoing maintenance (not blockers)
320320

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
"""Anti-drift checks for dispatch-meta ROUTE-003 cues in EN/zh-TW cheatsheets."""
2+
3+
import sys
4+
from pathlib import Path
5+
6+
import pytest
7+
8+
sys.path.insert(0, str(Path(__file__).parent))
9+
10+
from test_validate_route_fixture import DISPATCH_META_SKILL_TRAP_PROBES # noqa: E402
11+
12+
SKILLS = Path(__file__).parent.parent.parent / "skills"
13+
EN_CHEATSHEET = SKILLS / "SKILL_TRIGGER_CHEATSHEET.md"
14+
ZH_CHEATSHEET = SKILLS / "SKILL_TRIGGER_CHEATSHEET.zh-TW.md"
15+
16+
17+
@pytest.fixture(scope="module")
18+
def en_cheatsheet_text() -> str:
19+
assert EN_CHEATSHEET.is_file(), f"missing {EN_CHEATSHEET}"
20+
return EN_CHEATSHEET.read_text(encoding="utf-8")
21+
22+
23+
@pytest.fixture(scope="module")
24+
def zh_cheatsheet_text() -> str:
25+
assert ZH_CHEATSHEET.is_file(), f"missing {ZH_CHEATSHEET}"
26+
return ZH_CHEATSHEET.read_text(encoding="utf-8")
27+
28+
29+
@pytest.mark.parametrize("cue", DISPATCH_META_SKILL_TRAP_PROBES)
30+
def test_dispatch_meta_cues_present_in_english_cheatsheet(en_cheatsheet_text: str, cue: str):
31+
haystack = en_cheatsheet_text.lower()
32+
assert cue.lower() in haystack, f"English cheatsheet missing dispatch-meta cue: {cue!r}"
33+
34+
35+
@pytest.mark.parametrize("cue", DISPATCH_META_SKILL_TRAP_PROBES)
36+
def test_dispatch_meta_cues_present_in_zh_tw_cheatsheet(zh_cheatsheet_text: str, cue: str):
37+
haystack = zh_cheatsheet_text.lower()
38+
assert cue.lower() in haystack, f"zh-TW cheatsheet missing dispatch-meta cue: {cue!r}"

reflective-prompt-library/plans/tests/test_quality_gates_summary.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,36 @@ def test_benchmark_section_covers_nine_workflows(summary_text: str):
126126
assert "8 different skills" not in section
127127
assert "24 golden tasks" in section or "24 benchmark tasks" in section
128128

129+
130+
131+
def _pytest_item_count() -> int:
132+
import subprocess
133+
134+
result = subprocess.run(
135+
["python3", "-m", "pytest", str(Path(__file__).parent), "--collect-only", "-q"],
136+
capture_output=True,
137+
text=True,
138+
check=True,
139+
)
140+
for line in reversed(result.stdout.splitlines()):
141+
if " tests collected" in line:
142+
return int(line.split()[0])
143+
raise AssertionError("could not determine pytest collection count")
144+
145+
129146
def test_key_metrics_benchmark_task_count_matches_fixture(summary_text: str):
130147
metrics = _section_between(summary_text, "## Key Metrics", "### Routing Consistency Tracking")
131148
assert f"| Benchmark tasks | {MIN_TASK_COUNT} |" in metrics
132149

150+
def test_phase2_pytest_floor_matches_collection(summary_text: str):
151+
"""Anti-drift: Phase 2 Done bullet must not under-report collected pytest count."""
152+
collected = _pytest_item_count()
153+
match = re.search(r"(\d+)\+ pytest anti-drift suite in CI", summary_text)
154+
assert match, "missing pytest floor in Phase 2 Done bullet"
155+
documented = int(match.group(1))
156+
assert documented <= collected, (
157+
f"QUALITY_GATES documents {documented}+ pytest but {collected} collected"
158+
)
159+
assert documented >= collected - 20, (
160+
f"QUALITY_GATES pytest floor {documented} is stale vs {collected} collected"
161+
)

reflective-prompt-library/plans/tests/test_validate_route_fixture.py

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,12 +66,16 @@ def test_route_fixture_minimums_match_validator_constants():
6666
)
6767

6868
# ROUTE-002 holdout phrases that should appear in EN/zh-TW cheatsheets (anti-drift).
69+
DESIGN_COMPARISON_HOLDOUT_PROBE = (
70+
"compare API designs on paper without touching the repository",
71+
)
72+
6973
BOUNDARY_CHEATSHEET_CUES = tuple(text for text, _ in ROUND_51_BOUNDARY_PROBES) + (
7074
"write tickets and acceptance criteria without touching the repo",
7175
"check the diff for readability not production deploy",
7276
"審查 README 清晰度不是安全風險",
7377
"把規格寫出來但不要改程式",
74-
)
78+
) + DESIGN_COMPARISON_HOLDOUT_PROBE
7579

7680

7781
def test_round_51_boundary_probes():
@@ -97,6 +101,10 @@ def test_round_51_boundary_probes():
97101
"plan 已核准 spec without repo changes",
98102
)
99103

104+
DISPATCH_META_SKILL_TRAP_PROBES = (
105+
"which reflective workflow skill fits a routing-only mixed intent",
106+
)
107+
100108
ROUTE_003_ADVERSARIAL_BOUNDARY_PROBES = (
101109
("design a handoff workflow specification without runtime code", "reflective-spec-plan"),
102110
("patch the trivial null check in code", "reflective-implement"),
@@ -222,3 +230,17 @@ def test_approved_spec_plan_not_implement_trap_covers_probes():
222230
missing = [p for p in IMPLEMENT_NOT_PLAN_SPEC_PLAN_PROBES if p not in fixture_phrases]
223231
assert not missing, f"approved_spec_plan_not_implement_trap missing probes: {missing}"
224232

233+
def _dispatch_meta_skill_trap_phrases() -> set[str]:
234+
config = load_route_eval_config(PLANS / "route-003-adversarial-eval.yaml")
235+
for group in config["adversarial_sets"]:
236+
if group["name"] == "dispatch_meta_skill_trap":
237+
return set(group.get("phrases", []))
238+
raise AssertionError("missing ROUTE-003 dispatch_meta_skill_trap group")
239+
240+
241+
def test_dispatch_meta_skill_trap_covers_probes():
242+
"""Anti-drift: ROUTE-003 dispatch-meta probes live in dedicated trap group."""
243+
fixture_phrases = _dispatch_meta_skill_trap_phrases()
244+
missing = [p for p in DISPATCH_META_SKILL_TRAP_PROBES if p not in fixture_phrases]
245+
assert not missing, f"dispatch_meta_skill_trap missing probes: {missing}"
246+

reflective-prompt-library/skills/SKILL_TRIGGER_CHEATSHEET.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ Trigger cues:
4242
- "Which skill handles session handoff in this library?"
4343
- "Which workflow skill should run for this library task?"
4444
- "Which reflective workflow skill covers handoff retro?"
45+
- "Which reflective workflow skill fits a routing-only mixed intent?"
4546

4647
Do not use when:
4748

@@ -86,6 +87,7 @@ Trigger cues:
8687

8788
- "Write tickets and acceptance criteria without touching the repo."
8889
- "Compare two API designs on paper only no implementation."
90+
- "Compare API designs on paper without touching the repository."
8991
- "Design comparison on paper without repo changes."
9092
- "比較兩個 API 設計方案但不要寫 code."
9193
- "把規格寫出來但不要改程式."

reflective-prompt-library/skills/SKILL_TRIGGER_CHEATSHEET.zh-TW.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ Skill 契約語言:
4242
- 「Which skill handles session handoff in this library?」
4343
- 「Which workflow skill should run for this library task?」
4444
- 「Which reflective workflow skill covers handoff retro?」
45+
- 「Which reflective workflow skill fits a routing-only mixed intent?」
4546

4647
不要用在:
4748

@@ -82,6 +83,7 @@ L1 快速路徑:
8283

8384
- 「Write tickets and acceptance criteria without touching the repo。」
8485
- 「Compare two API designs on paper only no implementation。」
86+
- 「Compare API designs on paper without touching the repository。」
8587
- 「比較兩個 API 設計方案但不要寫 code。」
8688
- 「把規格寫出來但不要改程式。」
8789
- 「Design a handoff workflow specification without runtime code。」

0 commit comments

Comments
 (0)