Skip to content

Commit 963ac0c

Browse files
committed
Round 97: cross-category workflow skill reference library registry
1 parent ffc418b commit 963ac0c

17 files changed

Lines changed: 161 additions & 30 deletions

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Full library docs: [reflective-prompt-library/README.md](reflective-prompt-libra
2121
## Governance
2222

2323
- **Contributing:** [CONTRIBUTING.md](CONTRIBUTING.md) — quality gates, routing maintenance (R8–R12), `make all`
24-
- **Panel record:** [multi-agent-panel-consensus](reflective-prompt-library/plans/multi-agent-panel-consensus-2026-06-25.md) — six-lens Socratic consensus (Rounds 1–96)
24+
- **Panel record:** [multi-agent-panel-consensus](reflective-prompt-library/plans/multi-agent-panel-consensus-2026-06-25.md) — six-lens Socratic consensus (Rounds 1–97)
2525
- **Operator playbook:** [GLOSSARY.md](reflective-prompt-library/GLOSSARY.md) — Governance Maintenance Playbook
2626

2727
The repository contains:

reflective-prompt-library/GLOSSARY.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -337,7 +337,7 @@ Curated top-of-cheatsheet summary of high-confusion routing traps (ROUTE-002 hol
337337

338338
## Governance Maintenance Playbook / 治理維護手冊
339339

340-
Ongoing upkeep after panel close (Rounds 1–96). Not agent instructions — operator checklist.
340+
Ongoing upkeep after panel close (Rounds 1–97). Not agent instructions — operator checklist.
341341

342342
**Operational test:** Before router tuning, add fresh ROUTE-002/003 holdout phrases; run `make all`; record decisions in `PROJECT_KNOWLEDGE.md` Decision Index when governance surface changes.
343343

@@ -369,3 +369,5 @@ Ongoing upkeep after panel close (Rounds 1–96). Not agent instructions — ope
369369
26. When editing composable prompt Purpose preambles, keep `Primary workflow surface(s)` / Supporting-lens lines via `assert_primary_workflow_surface_preamble` in `prompt_eval_helpers.py`; update `SUPPORTING_LENS_PRIMARY_SURFACE_BY_CATEGORY` for exemptions; run `test_prompt_primary_workflow_surface_library_registry.py` plus per-category `test_*_prompts_eval_harness.py` guards.
370370
27. When editing category workflow skill coverage tuples, keep frozen `*_COVER_WORKFLOW_SKILLS` in `test_*_prompts_eval_harness.py` aligned with `assert_category_workflow_skill_coverage`; `01-thinking` stays exempt (consumer graph); run `test_workflow_skill_coverage_library_registry.py`.
371371
28. When editing eval_harness score floors, keep `PROMPT_EVAL_MIN_SCORE` in `prompt_eval_helpers.py` and use `assert_prompt_meets_eval_harness_floor` in per-category `test_*_prompts_eval_harness.py` guards; run `test_prompt_eval_harness_score_library_registry.py`.
372+
28. When editing eval_harness score floors, keep `PROMPT_EVAL_MIN_SCORE` in `prompt_eval_helpers.py` and use `assert_prompt_meets_eval_harness_floor` in per-category `test_*_prompts_eval_harness.py` guards; run `test_prompt_eval_harness_score_library_registry.py`.
373+
29. When editing per-category `reference_workflow_skills` guards, use `assert_prompt_references_workflow_skill` in `prompt_eval_helpers.py` (preamble-scoped, not fenced templates); run `test_prompt_workflow_skill_reference_library_registry.py` plus per-category harness guards.

reflective-prompt-library/PROJECT_KNOWLEDGE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ deferred promotions are recurrence-gated — see [panel backlog](plans/multi-age
7272

7373
## Decision Index
7474

75+
- 2026-06-25 Round 97 panel — cross-category workflow skill reference library registry (`test_prompt_workflow_skill_reference_library_registry.py`, DRY `assert_prompt_references_workflow_skill`) → [record](plans/multi-agent-panel-consensus-2026-06-25.md)
7576
- 2026-06-25 Round 96 panel — cross-category eval_harness score floor library registry (`test_prompt_eval_harness_score_library_registry.py`, DRY `assert_prompt_meets_eval_harness_floor`) → [record](plans/multi-agent-panel-consensus-2026-06-25.md)
7677
- 2026-06-25 Round 85 panel — composable prompt Primary workflow surface preamble guards (`test_*_prompts_eval_harness.py`) + Supporting-lens exemption → [record](plans/multi-agent-panel-consensus-2026-06-25.md)
7778
- 2026-06-25 Round 94 panel — cross-category Primary workflow surface preamble library registry (`test_prompt_primary_workflow_surface_library_registry.py`, DRY `assert_primary_workflow_surface_preamble`) → [record](plans/multi-agent-panel-consensus-2026-06-25.md)

reflective-prompt-library/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Pick **Strictness L1–L6** first (`skills/reflective-dispatch/SKILL.md`, [GLOSS
3030

3131
## Governance Panel Record
3232

33-
Multi-agent Socratic consensus on project goals and the nine skills (Rounds 1–96, options A–HF) is recorded in [plans/multi-agent-panel-consensus-2026-06-25.md](plans/multi-agent-panel-consensus-2026-06-25.md). Run `make all` before claiming routing or governance changes are verified.
33+
Multi-agent Socratic consensus on project goals and the nine skills (Rounds 1–97, options A–HK) is recorded in [plans/multi-agent-panel-consensus-2026-06-25.md](plans/multi-agent-panel-consensus-2026-06-25.md). Run `make all` before claiming routing or governance changes are verified.
3434

3535
## Directory Map
3636

reflective-prompt-library/plans/QUALITY_GATES_SUMMARY.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -314,7 +314,7 @@ ROUTE-002 measures unseen phrasing separately from ROUTE-001. Round 7 (2026-06-2
314314
2. **ROUTE-001/002/003 in CI** — 128 + 102 + 53 paraphrases at 100% consistency (seeded fixtures); `validate_route_fixture.py` gates minimum coverage
315315
3. **Governance validators** — links, lint, governance metadata, PROJECT_KNOWLEDGE, benchmark fixture, skill examples
316316
4. **Harness policy docs** — CONTRIBUTING, AGENTS, SKILL_INSTALLATION, maintenance playbook
317-
5. **Doc anti-drift** — `test_routing_contract.py`, cheatsheet parity tests, `test_readme_governance.py`, `test_thinking_prompts_eval_harness.py`, `test_engineering_prompts_eval_harness.py`, `test_prompt_cross_links.py`, `test_core_prompts_eval_harness.py`, `test_human_review_library_registry.py`, `test_prompt_skill_links_library_registry.py`, `test_prompt_contract_library_registry.py`, `test_prompt_primary_workflow_surface_library_registry.py`, `test_workflow_skill_coverage_library_registry.py`, `test_prompt_eval_harness_score_library_registry.py`, `test_agent_prompts_eval_harness.py`, `test_context_prompts_eval_harness.py`, `test_domain_prompts_eval_harness.py`, `test_repo_prompts_eval_harness.py`, `test_validate_governance.py`, `test_validate_links.py`, `test_lint_skills.py`, `test_skill_module_contract.py` (Escalation subsection + Trigger/Methods/Output/Never; 650+ pytest anti-drift suite in CI); reciprocal thinking-lens ↔ skill checks and `00-core` + composable `Primary workflow surface(s)` ↔ `*_SKILL_LINKS` parity in `test_prompt_cross_links.py` (including strict Primary workflow surfaces parity via `test_thinking_lens_primary_surfaces_match_consumer_graph`); Human Review + Escalation route-target guards in thinking/skill contract tests; composable `Primary workflow surface(s)` / Supporting-lens preamble guards and composable `## Human Review` preamble guards (route to `reflective-risk`) via `prompt_eval_helpers.assert_human_review_preamble` in `test_*_prompts_eval_harness.py`; frozen `*_HUMAN_REVIEW_REQUIRED` / `*_HUMAN_REVIEW_EXEMPT` set parity across all prompt categories (Round 90); library-wide contract heading registry (`PROMPT_CONTRACT_HEADINGS`, Round 93); workflow skill coverage registry (`*_COVER_WORKFLOW_SKILLS`, Round 95); eval_harness score floor registry (`PROMPT_EVAL_MIN_SCORE`, Round 96)
317+
5. **Doc anti-drift** — `test_routing_contract.py`, cheatsheet parity tests, `test_readme_governance.py`, `test_thinking_prompts_eval_harness.py`, `test_engineering_prompts_eval_harness.py`, `test_prompt_cross_links.py`, `test_core_prompts_eval_harness.py`, `test_human_review_library_registry.py`, `test_prompt_skill_links_library_registry.py`, `test_prompt_contract_library_registry.py`, `test_prompt_primary_workflow_surface_library_registry.py`, `test_workflow_skill_coverage_library_registry.py`, `test_prompt_eval_harness_score_library_registry.py`, `test_prompt_workflow_skill_reference_library_registry.py`, `test_agent_prompts_eval_harness.py`, `test_context_prompts_eval_harness.py`, `test_domain_prompts_eval_harness.py`, `test_repo_prompts_eval_harness.py`, `test_validate_governance.py`, `test_validate_links.py`, `test_lint_skills.py`, `test_skill_module_contract.py` (Escalation subsection + Trigger/Methods/Output/Never; 660+ pytest anti-drift suite in CI); reciprocal thinking-lens ↔ skill checks and `00-core` + composable `Primary workflow surface(s)` ↔ `*_SKILL_LINKS` parity in `test_prompt_cross_links.py` (including strict Primary workflow surfaces parity via `test_thinking_lens_primary_surfaces_match_consumer_graph`); Human Review + Escalation route-target guards in thinking/skill contract tests; composable `Primary workflow surface(s)` / Supporting-lens preamble guards and composable `## Human Review` preamble guards (route to `reflective-risk`) via `prompt_eval_helpers.assert_human_review_preamble` in `test_*_prompts_eval_harness.py`; frozen `*_HUMAN_REVIEW_REQUIRED` / `*_HUMAN_REVIEW_EXEMPT` set parity across all prompt categories (Round 90); library-wide contract heading registry (`PROMPT_CONTRACT_HEADINGS`, Round 93); workflow skill coverage registry (`*_COVER_WORKFLOW_SKILLS`, Round 95); eval_harness score floor registry (`PROMPT_EVAL_MIN_SCORE`, Round 96); workflow skill reference registry (`assert_prompt_references_workflow_skill`, Round 97)
318318

319319
### Ongoing maintenance (not blockers)
320320

@@ -384,4 +384,4 @@ Phase 1 quality-gate tooling and documentation are **complete**. Routing consist
384384
- ✅ Benchmark fixture gate plus optional manual benchmark runs
385385
- ✅ Research-backed design decisions
386386

387-
The project is positioned to grow sustainably with quality discipline built in from the start. **No open implementation blockers** remain from panel Rounds 1–96; work is recurrence-gated maintenance per playbook. The next measurable quality target is **holdout expansion before router tuning** and optional manual baseline-vs-skill benchmark runs — not shipping new core skills without promotion evidence.
387+
The project is positioned to grow sustainably with quality discipline built in from the start. **No open implementation blockers** remain from panel Rounds 1–97; work is recurrence-gated maintenance per playbook. The next measurable quality target is **holdout expansion before router tuning** and optional manual baseline-vs-skill benchmark runs — not shipping new core skills without promotion evidence.

reflective-prompt-library/plans/multi-agent-panel-consensus-2026-06-25.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2881,3 +2881,57 @@ User directive (repeat): review prompts, plans, skills, and Socratic/critical-th
28812881
---
28822882

28832883
**Resealed 2026-06-25** after **Round 96** (options HB–HF). Eval_harness score floors are now library-registry checked across all `00-core``06-repo` categories with shared `assert_prompt_meets_eval_harness_floor` and per-category `MIN_SCORE` aliases to `PROMPT_EVAL_MIN_SCORE`. Holdout expansion remains recurrence-gated maintenance.
2884+
2885+
## Round 97 — cross-category workflow skill reference library registry (2026-06-25)
2886+
2887+
**Options HG–HK** | Six-lens panel (Opus, Codex, Gemini, Composer, Sakana, GLM)
2888+
2889+
### Round 97 options
2890+
2891+
| ID | Proposal | Verdict |
2892+
| --- | --- | --- |
2893+
| HG | DRY `assert_prompt_references_workflow_skill` in `prompt_eval_helpers.py` (preamble-scoped) | **Agree** |
2894+
| HH | `test_prompt_workflow_skill_reference_library_registry.py` — library-wide preamble skill-reference sweep | **Agree** |
2895+
| HI | GLOSSARY playbook step 29 + governance sync | **Agree** |
2896+
| HJ | ROUTE holdout expansion | **Defer** |
2897+
| HK | Router / tenth skill / benchmark CI | **Reject** |
2898+
2899+
### Round 97 verdict table
2900+
2901+
| ID | Option | Verdict | Action |
2902+
| --- | --- | --- | --- |
2903+
| HG | Workflow skill reference DRY helper | **Agree** | `assert_prompt_references_workflow_skill` |
2904+
| HH | Workflow skill reference library registry | **Agree** | `test_prompt_workflow_skill_reference_library_registry.py` |
2905+
| HI | Playbook + docs | **Agree** | step 29; panel round 97 sync |
2906+
| HJ | Holdout expansion | **Defer** | maintenance |
2907+
| HK | Router/tenth skill/benchmark CI | **Reject** | no change |
2908+
2909+
### Socratic rationale (Round 97)
2910+
2911+
- **Opus:** Round 96 closed eval_harness score floors; seven per-category `reference_workflow_skills` guards still duplicate `"reflective-" in text` with no library-wide falsifiability and weaker preamble scoping than cross-link tests.
2912+
- **Codex:** Preamble-scoped helper aligns with `test_prompt_cross_links.py` engineering guard; registry sweep catches template-only skill mentions across all 49 prompts.
2913+
- **Gemini:** Cheap deterministic check; no router or prompt content churn.
2914+
- **Composer:** Mirrors R91–R96 registry pattern; one helper + one registry file.
2915+
- **Sakana:** Thinking lenses already require consumer skills in preambles; library registry documents that invariant explicitly.
2916+
- **GLM:** Preamble scope avoids false passes from fenced zh-TW templates; playbook step 29 gives operators a single checklist line.
2917+
2918+
**All roles agree.**
2919+
2920+
## Implemented Changes (Round 97)
2921+
2922+
- `plans/tests/prompt_eval_helpers.py`: `assert_prompt_references_workflow_skill`
2923+
- `plans/tests/test_*_prompts_eval_harness.py`: DRY workflow skill reference guards
2924+
- `plans/tests/test_prompt_workflow_skill_reference_library_registry.py`: cross-category registry + library glob parity
2925+
- `GLOSSARY.md`: playbook Rounds 1–97; step 29 for workflow skill reference library registry
2926+
- `QUALITY_GATES_SUMMARY.md`: workflow skill reference registry note; panel Rounds 1–97; 660+ pytest floor
2927+
- `PROJECT_KNOWLEDGE.md`: Decision Index Round 97 entry
2928+
- `README.md`, `reflective-prompt-library/README.md`, `test_readme_governance.py`: panel round 97 sync
2929+
2930+
## Verification (Round 97)
2931+
2932+
- `make all`: 663 pytest + ROUTE-001/002/003 100%
2933+
2934+
---
2935+
2936+
**Resealed 2026-06-25** after **Round 97** (options HG–HK). Workflow skill references are now library-registry checked across all `00-core``06-repo` categories with shared `assert_prompt_references_workflow_skill` (preamble-scoped). Holdout expansion remains recurrence-gated maintenance.
2937+

reflective-prompt-library/plans/tests/prompt_eval_helpers.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,14 @@ def assert_category_workflow_skill_coverage(
128128
for skill in required_skills:
129129
assert skill in text, f"{category_label} should reference {skill}"
130130

131+
132+
def assert_prompt_references_workflow_skill(prompt_path: Path) -> None:
133+
"""Every composable prompt must mention at least one reflective-* workflow skill."""
134+
body = prompt_path.read_text(encoding="utf-8")
135+
assert "reflective-" in body, (
136+
f"{prompt_path.name} should map to at least one workflow skill"
137+
)
138+
131139
def assert_prompt_meets_eval_harness_floor(
132140
prompt_path: Path,
133141
harness,

reflective-prompt-library/plans/tests/test_agent_prompts_eval_harness.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
sys.path.insert(0, str(Path(__file__).parent))
1010

1111
from eval_harness import EvalHarness # noqa: E402
12-
from prompt_eval_helpers import assert_category_workflow_skill_coverage, assert_human_review_preamble, assert_primary_workflow_surface_preamble, prompts_with_human_review, assert_human_review_required_matches_detection, assert_human_review_exempt_have_no_preamble_section, assert_human_review_sets_partition, PROMPT_CONTRACT_HEADINGS, PROMPT_EVAL_MIN_SCORE, assert_prompt_contract_headings, assert_prompt_meets_eval_harness_floor # noqa: E402
12+
from prompt_eval_helpers import assert_category_workflow_skill_coverage, assert_human_review_preamble, assert_primary_workflow_surface_preamble, prompts_with_human_review, assert_human_review_required_matches_detection, assert_human_review_exempt_have_no_preamble_section, assert_human_review_sets_partition, PROMPT_CONTRACT_HEADINGS, PROMPT_EVAL_MIN_SCORE, assert_prompt_contract_headings, assert_prompt_references_workflow_skill, assert_prompt_meets_eval_harness_floor # noqa: E402
1313

1414
REQUIRED_HEADINGS = PROMPT_CONTRACT_HEADINGS
1515
MIN_SCORE = PROMPT_EVAL_MIN_SCORE
@@ -59,8 +59,7 @@ def test_agent_prompt_meets_eval_harness_floor(prompt_path: Path, harness: EvalH
5959

6060
def test_agent_prompts_reference_workflow_skills():
6161
for prompt_path in AGENT_PROMPTS:
62-
text = prompt_path.read_text(encoding="utf-8")
63-
assert "reflective-" in text, f"{prompt_path.name} should map to at least one workflow skill"
62+
assert_prompt_references_workflow_skill(prompt_path)
6463

6564

6665
def test_agent_prompts_cover_agent_workflow_surfaces():

reflective-prompt-library/plans/tests/test_context_prompts_eval_harness.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
sys.path.insert(0, str(Path(__file__).parent))
1010

1111
from eval_harness import EvalHarness # noqa: E402
12-
from prompt_eval_helpers import assert_category_workflow_skill_coverage, assert_human_review_preamble, assert_primary_workflow_surface_preamble, prompts_with_human_review, assert_human_review_required_matches_detection, assert_human_review_exempt_have_no_preamble_section, assert_human_review_sets_partition, PROMPT_CONTRACT_HEADINGS, PROMPT_EVAL_MIN_SCORE, assert_prompt_contract_headings, assert_prompt_meets_eval_harness_floor # noqa: E402
12+
from prompt_eval_helpers import assert_category_workflow_skill_coverage, assert_human_review_preamble, assert_primary_workflow_surface_preamble, prompts_with_human_review, assert_human_review_required_matches_detection, assert_human_review_exempt_have_no_preamble_section, assert_human_review_sets_partition, PROMPT_CONTRACT_HEADINGS, PROMPT_EVAL_MIN_SCORE, assert_prompt_contract_headings, assert_prompt_references_workflow_skill, assert_prompt_meets_eval_harness_floor # noqa: E402
1313

1414
REQUIRED_HEADINGS = PROMPT_CONTRACT_HEADINGS
1515
MIN_SCORE = PROMPT_EVAL_MIN_SCORE
@@ -56,8 +56,7 @@ def test_context_prompt_meets_eval_harness_floor(prompt_path: Path, harness: Eva
5656

5757
def test_context_prompts_reference_workflow_skills():
5858
for prompt_path in CONTEXT_PROMPTS:
59-
text = prompt_path.read_text(encoding="utf-8")
60-
assert "reflective-" in text, f"{prompt_path.name} should map to at least one workflow skill"
59+
assert_prompt_references_workflow_skill(prompt_path)
6160

6261

6362
def test_context_prompts_cover_context_workflow_surfaces():

reflective-prompt-library/plans/tests/test_core_prompts_eval_harness.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
PROMPT_EVAL_MIN_SCORE,
1515
assert_primary_workflow_surface_preamble,
1616
assert_category_workflow_skill_coverage, assert_prompt_contract_headings,
17+
assert_prompt_references_workflow_skill,
1718
assert_prompt_meets_eval_harness_floor, # noqa: E402
1819
assert_human_review_exempt_have_no_preamble_section,
1920
assert_human_review_preamble,
@@ -67,8 +68,7 @@ def test_core_prompt_meets_eval_harness_floor(prompt_path: Path, harness: EvalHa
6768

6869
def test_core_prompts_reference_workflow_skills():
6970
for prompt_path in CORE_PROMPTS:
70-
text = prompt_path.read_text(encoding="utf-8")
71-
assert "reflective-" in text, f"{prompt_path.name} should map to at least one workflow skill"
71+
assert_prompt_references_workflow_skill(prompt_path)
7272

7373

7474
def test_core_prompts_cover_brief_and_dispatch():

0 commit comments

Comments
 (0)