Add validation evals and tighten protocol checks

Author: snapsynapse

.github/workflows/validate.yml

@@ -0,0 +1,21 @@
+name: Validate
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  validate:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+      - run: npm ci
+      - run: npm run validate

.gitignore

@@ -3,6 +3,14 @@
 .AppleDouble
 .LSOverride
 ._*
+.Spotlight-V100
+.Trashes
+Icon?
+
+# Windows local metadata
+Thumbs.db
+ehthumbs.db
+Desktop.ini
 
 # Editor / IDE
 *.swp

README.md

@@ -109,7 +109,7 @@ Each agent maintains a self-contained skill file encoding the full protocol work
 |------|-------------|
 | [skills/claude-opus_4.6/SKILL.md](skills/claude-opus_4.6/SKILL.md) | Claude protocol execution guide (v0.2.0) |
 | [skills/codex_5.3/SKILL.md](skills/codex_5.3/SKILL.md) | Codex protocol execution guide |
-| [skills/skill-versioning/SKILL.md](skills/skill-versioning/SKILL.md) | Shared metaskill for versioning skill bundles across Codex + Claude |
+| [skills/skill-versioning/SKILL.md](skills/skill-versioning/SKILL.md) | Shared metaskill for versioning skill bundles across Codex + Claude. Current installs may expose the same bundle as `skill-provenance`. |
 | [templates/SKILL.md](templates/SKILL.md) | Skill template for onboarding new agents |
 
 ### Coordination artifacts
@@ -119,6 +119,7 @@ Each agent maintains a self-contained skill file encoding the full protocol work
 | [VISION.md](VISION.md) | Maintainer intent and alignment reference |
 | [schemas/turnfile/](schemas/turnfile/) | JSON Schema for TURNFILE.yaml validation |
 | [tools/](tools/) | Validators and helpers (turnfile-lint, mailbox invariants, PRD promotion, payload envelopes) |
+| [Validation](docs/VALIDATION.md) | Readiness gate, eval suite, and CI validation commands |
 
 ### Templates (copy and fill)
 
@@ -163,6 +164,7 @@ node tools/validate-skills-preflight.mjs        # verify skill install/parity/ve
 Package scripts:
 
 ```bash
+npm run validate             # run the full repo readiness suite
 npm run validate:skills      # strict preflight (requires global skill install)
 npm run validate:skills:ci   # repo-only checks (CI-safe)
 ```

docs/NOTIFICATION_PROTOCOL.md

@@ -92,6 +92,14 @@ Field requirements:
 3. Long-form content is moved/preserved in `MAILBOX_ARCHIVE.md`.
 4. Archive references must retain message IDs so context is recoverable.
 
+Closed Summary rows use this compact schema:
+
+```text
+| ID | Date | From -> To | Final status | Outcome |
+|----|------|------------|--------------|---------|
+| MSG-YYYYMMDD-### | YYYY-MM-DD | <sender> -> <receiver> | <closed|actioned|superseded|withdrawn|abandoned> | <short outcome summary> |
+```
+
 ## Delivery semantics
 
 1. Sender appends new message in newest-first position.

docs/VALIDATION.md

@@ -0,0 +1,40 @@
+# Validation
+
+Use `npm run validate` as the repo readiness gate before starting or resuming a Turnfile experiment.
+
+## Commands
+
+```bash
+npm run validate
+```
+
+Runs the full local gate:
+1. `npm run lint:turnfile`
+2. `npm run lint:mailbox`
+3. `npm run lint:prds`
+4. `npm run validate:skills:ci`
+5. `npm run evals`
+
+```bash
+npm run evals
+```
+
+Runs fixture-based regression evals for the validators and exporters. The evals create temporary files and assert CLI exit codes plus stable output snippets.
+
+```bash
+npm run validate:skills
+```
+
+Runs strict skills preflight, including global Codex skill install checks. Use this on a configured local workstation. CI uses `validate:skills:ci` because global installs are environment-specific.
+
+## Before an Experiment
+
+1. Run `npm ci` after dependency changes.
+2. Run `npm run validate`.
+3. Confirm `working-session/MAILBOX.md` has expected active messages and Open Queue state.
+4. Regenerate `working-session/MAILBOX.json` after mailbox edits with `npm run export:mailbox`.
+5. Decide whether `working-session/` is the active experiment workspace or whether the previous run should be archived under `examples/`.
+
+## CI
+
+GitHub Actions runs `npm ci` and `npm run validate` on pushes to `main` and on pull requests.

package.json

@@ -9,12 +9,14 @@
     "url": "https://github.com/snapsynapse/turnfile.git"
   },
   "scripts": {
+    "validate": "npm run lint:turnfile && npm run lint:mailbox && npm run lint:prds && npm run validate:skills:ci && npm run evals",
     "lint:turnfile": "node tools/turnfile-lint.mjs",
     "lint:mailbox": "node tools/validate-mailbox-invariants.mjs",
     "lint:prds": "node tools/validate-prd-promotion.mjs",
     "export:mailbox": "node tools/export-mailbox-json.mjs",
     "validate:skills": "node tools/validate-skills-preflight.mjs --strict-global",
-    "validate:skills:ci": "node tools/validate-skills-preflight.mjs"
+    "validate:skills:ci": "node tools/validate-skills-preflight.mjs",
+    "evals": "node tools/run-evals.mjs"
   },
   "devDependencies": {
     "ajv": "^8.17.1",

skills/STRUCTURE.md

@@ -12,9 +12,10 @@ All files must encode the same protocol outcomes while using agent-specific tool
 
 ## Shared Metaskills
 
-1. `skills/skill-versioning/` — shared cross-agent metaskill for skill bundle version tracking (`SKILL.md`, `MANIFEST.yaml`, `CHANGELOG.md`, `README.md`, `evals.json`).
+1. `skills/skill-versioning/` — repo-local compatibility path for the shared cross-agent metaskill for skill bundle version tracking (`SKILL.md`, `MANIFEST.yaml`, `CHANGELOG.md`, `README.md`, `evals.json`).
 
 This metaskill is jointly consumed by all agents (Codex, Claude, Gemini) and versioned as part of the Turnfile repository.
+Current installs may expose the same bundle identity as `skill-provenance`; tools should accept both `skill-versioning` and `skill-provenance` while preserving `skills/skill-versioning/` as the stable repository path.
 
 ## Shared Policy Assertions
 

templates/working-session/MAILBOX.md

@@ -54,10 +54,10 @@ per PRD-008 payload-first rule.
 
 ## Closed Summary
 
-| ID | From -> To | Subject | Final status | Closed by |
-|----|------------|---------|--------------|-----------|
+| ID | Date | From -> To | Final status | Outcome |
+|----|------|------------|--------------|---------|
 
 <!--
 Example row:
-| MSG-YYYYMMDD-001 | Claude -> Gemini | PRD review findings | actioned | Claude |
+| MSG-YYYYMMDD-001 | YYYY-MM-DD | Claude -> Gemini | closed | PRD review findings resolved. |
 -->

tools/export-mailbox-json.mjs

@@ -81,7 +81,8 @@ function parseTableAt(lines, tableHeaderIndex) {
 }
 
 function parseSectionTable(lines, headingText) {
-  const headingIndex = lines.findIndex((line) => line.trim() === headingText);
+  const headingTexts = Array.isArray(headingText) ? headingText : [headingText];
+  const headingIndex = lines.findIndex((line) => headingTexts.includes(line.trim()));
   if (headingIndex < 0) {
     return [];
   }
@@ -140,7 +141,9 @@ function parseActiveMessages(lines) {
   }
 
   const end = lines.findIndex(
-    (line, idx) => idx > start && line.trim() === "## Closed Summary (Newest First)",
+    (line, idx) =>
+      idx > start &&
+      ["## Closed Summary (Newest First)", "## Closed Summary"].includes(line.trim()),
   );
   const sliceEnd = end >= 0 ? end : lines.length;
   const section = lines.slice(start + 1, sliceEnd);
@@ -290,7 +293,7 @@ function main() {
     ),
     active_messages: parseActiveMessages(lines),
     closed_summary: addStandardKeys(
-      parseSectionTable(lines, "## Closed Summary (Newest First)"),
+      parseSectionTable(lines, ["## Closed Summary (Newest First)", "## Closed Summary"]),
     ),
   };