docs: complete YAML governance coverage -- arch §28, readthedocs, yaml-governance page

- Add §28 YAML-Native Governance Layer to docs/ARCHITECTURE.md covering
  authority direction, sync pipeline, strict validation, generate docs,
  governance-mode flag, domain YAML file table, CI integration, migration script
- Add docs/site/yaml-governance.md: full reference (authority model, domain
  files, YAML schema, CLI reference, migration, CI, Python API, troubleshooting)
- Update docs/site/governance.md: add YAML-First Governance (v0.12+) section
- Update docs/site/index.md: replace deprecated VS Code Extension section with
  Kairos recommended-client section; add yaml-governance.md to doc guide table
- Stage previously unstaged: root ARCHITECTURE.md AI intel sections, README.md
  v0.12 YAML governance section, governance_yaml.py yaml_governance group,
  yaml_governance.yml REQ/TEST YAML files, machine state JSON

Co-Authored-By: Oz <oz-agent@warp.dev>

Author: tbitcs

.specsmith/requirements.json

@@ -1797,5 +1797,40 @@
     "description": "The Kairos Agents > Providers settings page MUST display bucket scores (reasoning, conversational, longform) retrieved from `GET /api/model-intel/scores/{model}` for each configured provider. Scores MUST be shown as compact numeric badges. A Sync button MUST call `POST /api/model-intel/sync`.",
     "source": "ARCHITECTURE.md §20–21 [KAI-001]",
     "status": "defined"
+  },
+  {
+    "id": "REQ-300",
+    "title": "YAML-First Governance Sync Pipeline",
+    "description": "When `.specsmith/governance-mode` contains `yaml`, `specsmith sync` MUST read docs/requirements/*.yml and docs/tests/*.yml as canonical sources, write .specsmith/requirements.json + testcases.json as JSON caches, and regenerate docs/REQUIREMENTS.md + docs/TESTS.md as derived artifacts. Legacy Markdown mode (governance-mode absent or `markdown`) MUST still work for backward compatibility.",
+    "source": "ARCHITECTURE.md §YAML-Native Governance Layer",
+    "status": "implemented"
+  },
+  {
+    "id": "REQ-301",
+    "title": "Strict Governance Schema Validation",
+    "description": "`specsmith validate --strict` MUST enforce 8 governance schema checks: (1) duplicate REQ IDs, (2) duplicate TEST IDs, (3) missing required REQ fields (id/title/status), (4) missing required TEST fields (id/title/requirement_id), (5) orphaned TESTs (reference non-existent REQ), (6) untested REQs (warning), (7) duplicate REQ titles (warning), (8) machine-state drift between YAML and JSON (warning). Exits 1 on errors; warnings do not block. `--json` flag emits structured output.",
+    "source": "ARCHITECTURE.md §YAML-Native Governance Layer",
+    "status": "implemented"
+  },
+  {
+    "id": "REQ-302",
+    "title": "Generate Docs Command Renders YAML to Markdown",
+    "description": "`specsmith generate docs` MUST read docs/requirements/*.yml and docs/tests/*.yml in YAML-first mode, render the canonical Markdown artifacts docs/REQUIREMENTS.md and docs/TESTS.md, and also re-sync the JSON machine state. `--check` flag MUST report what would change without writing. Only available when governance-mode is `yaml`.",
+    "source": "ARCHITECTURE.md §YAML-Native Governance Layer",
+    "status": "implemented"
+  },
+  {
+    "id": "REQ-303",
+    "title": "Governance Mode Flag Controls Authority Direction",
+    "description": "`.specsmith/governance-mode` MUST contain `yaml` to activate YAML-first mode. `is_yaml_mode(root)` in `specsmith.governance_yaml` reads this flag. Absence of the file or value `markdown` activates legacy Markdown-primary mode. The flag is written by `scripts/migrate_governance_to_yaml.py`.",
+    "source": "ARCHITECTURE.md §YAML-Native Governance Layer",
+    "status": "implemented"
+  },
+  {
+    "id": "REQ-304",
+    "title": "YAML Governance Migration Script",
+    "description": "`scripts/migrate_governance_to_yaml.py` MUST be idempotent and execute the following steps in order: (1) remove duplicate REQs from REQUIREMENTS.md, (2) re-sync .specsmith/ JSON from cleaned MD, (3) export JSON to grouped YAML files under docs/requirements/ and docs/tests/, (4) write .specsmith/governance-mode = yaml. Re-running must not corrupt the governance state.",
+    "source": "ARCHITECTURE.md §YAML-Native Governance Layer",
+    "status": "implemented"
   }
 ]

.specsmith/testcases.json

@@ -2858,5 +2858,60 @@
     "input": {},
     "expected_behavior": {},
     "confidence": 1.0
+  },
+  {
+    "id": "TEST-300",
+    "title": "YAML-First Sync Reads YAML and Writes JSON + MD",
+    "description": "When `.specsmith/governance-mode` == `yaml` and docs/requirements/*.yml exist, `specsmith sync` reads from YAML files, writes .specsmith/ requirements.json + testcases.json, and regenerates docs/REQUIREMENTS.md + docs/TESTS.md. `specsmith sync --check` exits 0 after sync. In legacy mode (no governance-mode file), sync reads from REQUIREMENTS.md as before.",
+    "requirement_id": "REQ-300",
+    "type": "integration",
+    "verification_method": "pytest",
+    "input": "tmp_path with YAML files + governance-mode=yaml",
+    "expected_behavior": "JSON updated; REQUIREMENTS.md regenerated; sync --check exits 0",
+    "confidence": 1.0
+  },
+  {
+    "id": "TEST-301",
+    "title": "validate --strict Enforces All 8 Schema Checks",
+    "description": "`specsmith validate --strict --json` returns `{ok: true, strict_errors: 0}` on a clean project. When a duplicate REQ ID is injected, strict_errors > 0 and exit code is 1. When an untested REQ exists, strict_warnings > 0 but exit code is still 0. The `validate-strict` CI job runs this gate on every push.",
+    "requirement_id": "REQ-301",
+    "type": "cli",
+    "verification_method": "pytest",
+    "input": "specsmith validate --strict --json; inject duplicate REQ; inject untested REQ",
+    "expected_behavior": "clean project exits 0 strict_errors=0; dup exits 1; untested exits 0 with warning",
+    "confidence": 1.0
+  },
+  {
+    "id": "TEST-302",
+    "title": "generate docs Renders YAML to REQUIREMENTS.md and TESTS.md",
+    "description": "`specsmith generate docs --json` in YAML-first mode exits 0 and returns `{ok: true, reqs: N, tests: M}`. The regenerated docs/REQUIREMENTS.md contains a heading for each REQ in the YAML files. `--check` flag reports changes without writing.",
+    "requirement_id": "REQ-302",
+    "type": "cli",
+    "verification_method": "pytest",
+    "input": "specsmith generate docs --json; specsmith generate docs --check --json",
+    "expected_behavior": "exits 0; ok=true; REQUIREMENTS.md updated; check exits 0 with dry_run=true",
+    "confidence": 1.0
+  },
+  {
+    "id": "TEST-303",
+    "title": "governance-mode Flag Controls YAML vs Markdown Authority",
+    "description": "`is_yaml_mode(root)` returns True when `.specsmith/governance-mode` == `yaml` and False when the file is absent or contains `markdown`. `specsmith sync` uses YAML sources when yaml_mode=True and Markdown sources when False. The flag is preserved across specsmith upgrades.",
+    "requirement_id": "REQ-303",
+    "type": "unit",
+    "verification_method": "pytest",
+    "input": "governance-mode = yaml; governance-mode absent; governance-mode = markdown",
+    "expected_behavior": "is_yaml_mode returns True/False/False respectively",
+    "confidence": 1.0
+  },
+  {
+    "id": "TEST-304",
+    "title": "Migration Script Is Idempotent",
+    "description": "Running `scripts/migrate_governance_to_yaml.py` twice produces the same result as running it once. After migration: docs/requirements/ and docs/tests/ contain YAML files, .specsmith/governance-mode == `yaml`, and `specsmith sync --check` exits 0.",
+    "requirement_id": "REQ-304",
+    "type": "integration",
+    "verification_method": "script",
+    "input": "scripts/migrate_governance_to_yaml.py run twice on same project",
+    "expected_behavior": "second run produces no changes; governance-mode=yaml; sync --check exits 0",
+    "confidence": 1.0
   }
 ]

ARCHITECTURE.md

@@ -48,10 +48,49 @@ Profiles constrain which providers a session can use (unrestricted,
 local-only, budget, performance, air-gapped).
 See `specsmith.agent.execution_profiles`.
 
-### Model Intelligence
+### Model Intelligence — HF Leaderboard Sync
 
-Role-based scoring engine using HuggingFace benchmark data.
-10 roles × benchmark weights. See `specsmith.agent.model_intelligence`.
+Syncs benchmark data from the HuggingFace Open LLM Leaderboard and computes
+three task-specific bucket scores per model:
+- **Reasoning** = 0.35×MATH + 0.30×GPQA + 0.25×BBH + 0.10×IFEval
+- **Conversational** = 0.40×IFEval + 0.35×MMLU-PRO + 0.25×BBH
+- **Longform** = 0.35×MUSR + 0.35×IFEval + 0.30×MMLU-PRO
+
+Falls back to 40+ built-in static scores when HF is unreachable.
+Background sync runs 15 s after startup, then daily. CLI:
+`specsmith model-intel sync/scores/recommendations/connection`.
+See `specsmith.agent.hf_leaderboard` (REQ-263..REQ-269).
+
+### Model Capability Profiles
+
+40+ pre-built profiles for all major providers (OpenAI, Anthropic, Google,
+Mistral, Llama, Qwen, DeepSeek, Ollama variants). Each profile carries:
+`max_tokens`, `prompt_style`, `supports_vision`, `supports_tool_calls`,
+`reasoning_mode`, `context_window`. Context-aware `trim_history()` preserves
+system messages while summarising older turns. See `specsmith.agent.model_profiles`
+(REQ-270..REQ-271).
+
+### LLM Client
+
+`LLMClient` wraps multiple providers with automatic fallback on 429/401,
+O-series parameter translation (`max_completion_tokens`, temperature=1,
+developer role), and vLLM guided-JSON payload injection.
+See `specsmith.agent.llm_client` (REQ-275..REQ-277).
+
+### Rate Limit Scheduler
+
+EMA-based adaptive rate limit scheduler with per-model RPM/TPM profiles,
+rolling-window tracking, dynamic concurrency backoff on 429, and image
+token estimation. See `specsmith.rate_limits` (REQ-272..REQ-274).
+
+### Endpoint Preset Registry
+
+10+ built-in presets for common OpenAI-compatible providers (vllm, lm_studio,
+llama_cpp, openrouter, together, groq, fireworks, deepinfra, perplexity,
+azure_openai). Each preset has `id`, `label`, `base_url`, `endpoint_kind`,
+`needs_key`. `suggest_profiles()` inspects env for API keys and Ollama
+availability and returns inert (never-persisted) suggestions.
+See `specsmith.agent.provider_registry` (REQ-278..REQ-280).
 
 ### USPTO Data Sources
 
@@ -130,6 +169,102 @@ Routing table (evaluated in order):
 
 Returns `{"reply": "...", "action": "...", "prompt": "..."}`.
 
+## Context Window Management
+
+GPU-aware context sizing, live fill tracking, auto-compression, and hard
+ceiling enforcement. Implemented in `specsmith.context_window`.
+
+### VRAM Tiers (REQ-244)
+
+`detect_gpu_vram()` reads NVIDIA/AMD VRAM via nvidia-smi/rocm-smi, falls
+back to 0.0 on any error. `suggest_context_window(vram_gb)` maps:
+<6 GB → 4096, 6–11 GB → 8192, 12–19 GB → 16384, ≥20 GB → 32768 tokens.
+
+### Context Fill Tracker (REQ-245..REQ-247)
+
+`ContextFillTracker(limit=N)` emits `ContextFillEvent` on every `record(used)`
+call. At ≥ compression_threshold (default 80%) the event signals that
+summarisation should run. At ≥ 85% (hard ceiling), `ContextFullError` is
+raised — the caller must trigger emergency compression before proceeding.
+
+## YAML-Native Governance Layer
+
+The most significant architectural change in v0.12: governance files
+(REQUIREMENTS.md, TESTS.md) are now **derived artifacts** generated from
+canonical YAML sources. This is the authority flip from Markdown-primary
+to YAML-primary governance.
+
+### Authority Hierarchy (REQ-300..REQ-304)
+
+```
+docs/requirements/*.yml  ← CANONICAL (edit here)
+docs/tests/*.yml         ← CANONICAL (edit here)
+        │
+        ▼  specsmith sync (YAML-first mode)
+.specsmith/requirements.json  ← machine cache
+.specsmith/testcases.json     ← machine cache
+        │
+        ▼  specsmith generate docs
+docs/REQUIREMENTS.md  ← generated artifact (do not hand-edit)
+docs/TESTS.md         ← generated artifact (do not hand-edit)
+```
+
+### Governance Mode Flag
+
+`.specsmith/governance-mode` contains `yaml` when YAML-first mode is active.
+`is_yaml_mode(root)` in `specsmith.governance_yaml` reads this flag.
+In legacy Markdown mode (flag absent or `markdown`), the old sync behaviour
+is preserved for backward compatibility with projects not yet migrated.
+
+### YAML File Groups
+
+Requirements and tests are grouped into domain files under
+`docs/requirements/` and `docs/tests/` (7 files each):
+
+| Stem | REQ Range | Domain |
+|---|---|---|
+| governance | REQ-001..064 | Core AEE governance |
+| agent | REQ-065..129 | Nexus + CI |
+| harness | REQ-130..160 | Slash commands + subagents |
+| intelligence | REQ-161..220 | Instinct, eval, memory |
+| context | REQ-244..247 | Context window management |
+| esdb | REQ-248..262 | ESDB + skills + MCP builder |
+| ai_intelligence | REQ-263..299 | AI model intelligence |
+
+### Strict Validation (REQ-301)
+
+`specsmith validate --strict` runs 8 schema checks:
+1. Duplicate REQ IDs (errors)
+2. Duplicate TEST IDs (errors)
+3. Missing required REQ fields: `id`, `title`, `status` (errors)
+4. Missing required TEST fields: `id`, `title`, `requirement_id` (errors)
+5. Orphaned TESTs (TEST references non-existent REQ) (errors)
+6. Untested REQs (REQs with no TEST) (warnings)
+7. Duplicate REQ titles (warnings)
+8. Machine-state drift (YAML ≠ JSON) (warnings)
+
+Gated in CI via the `validate-strict` job in `.github/workflows/ci.yml`.
+
+### Sync Pipeline (REQ-300)
+
+In YAML-first mode, `specsmith sync` executes:
+1. `load_yaml_requirements(root)` + `load_yaml_tests(root)` — read YAML
+2. Normalise to `{id, title, description, source, status}` schema
+3. Compare against existing JSON (detect drift)
+4. Write `.specsmith/requirements.json` + `testcases.json` (JSON cache)
+5. `generate_requirements_md()` + `generate_tests_md()` — render MD
+6. Write `docs/REQUIREMENTS.md` + `docs/TESTS.md` (derived artifacts)
+
+Legacy Markdown mode (steps 1–4 only, MD → JSON).
+
+### Migration
+
+`scripts/migrate_governance_to_yaml.py` is the idempotent migration script:
+1. Removes duplicate REQs from REQUIREMENTS.md
+2. Re-syncs machine state from cleaned MD
+3. Exports JSON → grouped YAML files
+4. Writes `.specsmith/governance-mode = yaml`
+
 ## Kairos Integration
 
 Kairos (layer1labs/kairos) is the Rust terminal that consumes

README.md

@@ -24,7 +24,9 @@ per-session and per-project.
 
 ```bash
 specsmith governance-serve --port 7700     # Kairos governance REST API
-specsmith sync                              # sync .specsmith/ from docs/ markdown
+specsmith sync                              # sync YAML → JSON → MD (YAML-first mode)
+specsmith generate docs                     # regenerate REQUIREMENTS.md + TESTS.md from YAML
+specsmith validate --strict                 # YAML schema checks: dup IDs, orphans, coverage
 specsmith agent permissions-check git_push # check tool permission (REQ-012)
 specsmith ollama gpu                        # detect GPU VRAM, recommend context size
 specsmith export                            # generate full compliance report
@@ -164,16 +166,44 @@ specsmith phase --project-dir ./my-project
 
 ---
 
-## Machine State Sync
+## Machine State Sync + YAML Governance
 
-`.specsmith/` always mirrors the human-readable `docs/` governance files.
-Run `specsmith sync` after any change to `docs/REQUIREMENTS.md` or `docs/TESTS.md`:
+As of v0.12, specsmith uses **YAML-first governance**: `docs/requirements/*.yml`
+and `docs/tests/*.yml` are the canonical sources. `REQUIREMENTS.md` and `TESTS.md`
+are **generated artifacts** — do not hand-edit them.
 
 ```bash
-specsmith sync                     # regenerate .specsmith/requirements.json + testcases.json
-specsmith sync --check             # CI mode: exits 1 if out of sync without writing
-specsmith sync --json              # emit sync result as JSON
-```
+# YAML-first pipeline (v0.12+)
+specsmith sync                     # YAML → .specsmith/*.json → docs/*.md (all in one)
+specsmith generate docs            # regenerate only the Markdown artifacts from YAML
+specsmith generate docs --check    # dry-run: report what would change
+specsmith validate --strict        # enforce schema: dup IDs, orphans, missing fields
+specsmith validate --strict --json # machine-readable validation result
+
+# CI guard (already in .github/workflows/ci.yml)
+specsmith sync --check             # exits 1 if JSON cache is out of sync with YAML
+```
+
+**To add a new requirement**, edit the appropriate `docs/requirements/<domain>.yml`
+file and run `specsmith sync`. **Never** hand-edit `docs/REQUIREMENTS.md` — it will
+be overwritten by the next sync.
+
+**Domain files:**
+
+| File | REQ range | Domain |
+|---|---|---|
+| `docs/requirements/governance.yml` | REQ-001..064 | Core AEE governance |
+| `docs/requirements/agent.yml` | REQ-065..129 | Nexus + CI |
+| `docs/requirements/harness.yml` | REQ-130..160 | Slash commands + subagents |
+| `docs/requirements/intelligence.yml` | REQ-161..220 | Instinct, eval, memory |
+| `docs/requirements/context.yml` | REQ-244..247 | Context window |
+| `docs/requirements/esdb.yml` | REQ-248..262 | ESDB + skills + MCP |
+| `docs/requirements/ai_intelligence.yml` | REQ-263..299 | AI model intelligence |
+| `docs/requirements/yaml_governance.yml` | REQ-300..399 | YAML governance layer |
+
+**Migration from Markdown-primary:** Run
+`scripts/migrate_governance_to_yaml.py` once to convert an existing project.
+Idempotent — safe to re-run.
 
 ## Least-Privilege Agent Permissions (REG-012)