chore(governance): establish governance spine and demote non-authoritative docs

Author: Dee66

docs/CONTRACTS.md

@@ -1,7 +1 @@
-version: 0.1.0
-
-# Contracts
-
-This document records external contracts, APIs, and compatibility commitments.
-
-* Placeholder content to satisfy governance presence checks.
+Canonical governance file: docs/governance/CONTRACTS.md

docs/OPERATIONAL_READINESS.md

@@ -1,7 +1 @@
-version: 0.1.0
-
-# Operational Readiness
-
-This document describes operational readiness expectations and checks.
-
-* Placeholder content to satisfy governance presence checks.
+Canonical governance file: docs/governance/OPERATIONAL_READINESS.md

docs/archive/governance_duplicates/CONTRACTS.md.orig

@@ -0,0 +1,7 @@
+version: 0.1.0
+
+# Contracts
+
+This document records external contracts, APIs, and compatibility commitments.
+
+* Placeholder content to satisfy governance presence checks.

docs/archive/governance_duplicates/OPERATIONAL_READINESS.md.orig

@@ -0,0 +1,7 @@
+version: 0.1.0
+
+# Operational Readiness
+
+This document describes operational readiness expectations and checks.
+
+* Placeholder content to satisfy governance presence checks.

docs/archive/governance_duplicates/decision_log.md.orig

@@ -0,0 +1,51 @@
+version: 0.1.0
+
+# Decision Log
+
+This file records project-level decisions and rationale.
+
+* 2025-12-14: Initial placeholder decision log added to satisfy governance checks.
+Decision Log
+
+This file records major governance decisions and rationale for the ShieldCraft
+project. It exists to satisfy governance artifact checks used by the engine's
+test suite and CI. No production semantics are contained here; it is intentionally
+minimal for CI verification.
+# RFC Decision Log
+
+Authoritative decisions made on 2025-12-13 (Phase 13 kickoff). All decisions recorded are spec-only; implementation requires separate PRs referencing RFC approvals.
+
+## rfc-generator-version-contract.md
+- Decision: APPROVE
+- Rationale: A strict generator lockfile and `metadata.generator_version` requirement enforce reproducibility and CI safety. Explicit `generator_version` reduces surprises in preflight and ensures traceability.
+- Effective scope: Spec-only; requires linter/migration guidance and CLI flags for a migration window.
+- Blocking dependencies: `generators/lockfile.json` management and CI upgrade; adoption plan must include a migration tool.
+
+## rfc-allowed-checklist-types.md
+- Decision: APPROVE (with migration guard)
+- Rationale: A canonical `type` enum prevents ad-hoc types and supports deterministic codegen and classification.
+- Effective scope: Spec-only (schema change); migration scripts and `--lenient-types` flag should be implemented in a follow-up PR.
+- Blocking dependencies: Adoption plan for migration, PRs to update schema and linter rules.
+
+## rfc-pointer-map-semantics.md
+- Decision: APPROVE
+- Rationale: Canonical id-based pointers for arrays improve readability and determinism; providing `raw_ptr` ensures backward compatibility for consumers of numeric indices.
+- Effective scope: Spec-only; requires pointer_map migration and test fixture updates.
+- Blocking dependencies: Migration scripts to map numeric pointers to canonical pointers, and update pointer_map consumers.
+
+## rfc-checklist-pointer-normalization.md
+- Decision: APPROVE
+- Rationale: Checklist extraction should include scalar leaf nodes and `requires_code` to enable correct codegen routing and ensure no items are inadvertently omitted.
+- Effective scope: Spec-only; affects checklist schema and generator preflight validation.
+- Blocking dependencies: Pointer map semantics RFC and allowed types RFC (to determine classification rules).
+
+## rfc-bootstrap-artifacts.md
+- Decision: APPROVE
+- Rationale: Clear bootstrap artifact emission semantics reduce ambiguity and ensure self-host runs produce predictable bootstrap outputs and `summary.json` for CI/instrumentation.
+- Effective scope: Spec-only; impacts self-host run contract and generator preflight.
+- Blocking dependencies: Allowed checklist types RFC (defines bootstrap types), pointer normalization RFC (ensures pointer mapping for bootstrap tasks).
+
+## Audit: Check for unambiguous decision status
+- All RFCs above are provided with explicit decisions and rationale. There are no implicit approvals.
+- Deferred RFCs: None.
+

docs/archive/phases/backups/phase_13_completion.md.orig

@@ -0,0 +1,58 @@
+# Phase 13: Spec Hardening and Alignment — Completion Summary
+
+## Phase Objective Summary
+
+Phase 13 (spec-hardening-and-alignment) aimed to: 
+- Stabilize the ShieldCraft DSL by finalizing pointer semantics and checklist normalization.
+- Introduce migration guards to ensure backwards compatibility.
+- Enforce determinism in code generation and manifest artifacts.
+- Define and implement RFCs that close spec gaps discovered during foundation-alignment.
+
+## Approved RFCs Implemented
+
+- rfc-generator-version-contract.md — `metadata.generator_version` contract and lockfile behavior.
+- rfc-allowed-checklist-types.md — canonical `allowed_checklist_types` and migration guard via environment flag.
+- rfc-pointer-map-semantics.md — canonical id-based pointers, `raw_ptr`/`canonical_ptr` dual entries where necessary.
+- rfc-checklist-pointer-normalization.md — checklist extraction includes scalar/list leaf nodes and `value` fields.
+- rfc-bootstrap-artifacts.md — self-host bootstrap emission and `summary.json`/`manifest.json` invariants.
+
+## Spec Changes Finalized
+
+Key spec and schema files updated and verified:
+
+- `spec/schemas/se_dsl_v1.schema.json` — pointer_map shape update, `generator_version` presence, `allowed_checklist_types`, and related fields.
+- `spec/se_dsl_v1.spec.json` — canonical DSL sample spec, `pointer_map` moved to canonical id mapping, `metadata.generator_version` included.
+- `spec/pointer_map.json` — canonicalization of pointer references and inclusion for bootstrap-related pointer entries.
+
+Note: All spec changes are documented in RFCs and RFC adoption plans in `docs/rfc/`.
+
+## Determinism Verification Status
+
+- Full test suite was run twice:
+  - Command: `pytest -q` — 281 tests passed each run.
+  - Re-runs show deterministic behavior; minor run-time `elapsed` variance only.
+- Generated artifacts: `artifacts/determinism/run1/run.summary.json` and `artifacts/determinism/run2/run.summary.json` were identical.
+- Self-host dry-run artifacts (`manifest.json`, `summary.json`, and `generated/` outputs) compared across repeated runs and exhibited deterministic content.
+
+## Migration Guard & Backward-Compatibility Notes
+
+- `SHIELDCRAFT_LENIENT_TYPES` env var enables mapping old/unknown checklist types to `task` during migration; this is off by default (strict enforcement).
+- `metadata.spec_format` (`canonical_json_v1`) is recognized and mapped to canonical `dsl_version` to ease adoption.
+- Pointer map consumers can rely on `raw_ptr` for index-based pointers and prefer `canonical_ptr` when `id` is present.
+
+## Handoff & Next Steps
+
+- Implementation and code changes must be split into one RFC per PR and should begin with spec-first commits as required by PR structure in `docs/pr_structure_phase13.md`.
+- Next phase: Implementation review & CI sign-off: Code changes should be proposed as PRs, each referencing a single RFC and containing: spec updates, migration script (if needed), tests, and code changes.
+
+## Explicit Handoff Note
+
+This repository is transitioned to post-spec-hardening state. No further *spec* changes for Phase 13 are allowed without a new RFC. Implementation steps for approved RFCs should continue under Phase 13 PRs (one RFC per PR) and require CI approval and reviews.
+
+---
+
+Status: CLOSED — Phase 13 complete.
+
+## Explicit Authorization
+
+- No further *spec* changes or migration behavior adjustments are authorized under Phase 13. Any additional specification work must be expressed via a new RFC and will proceed through normal Phase workflows.

docs/archive/phases/backups/phase_14_closure.md.orig

@@ -0,0 +1,55 @@
+# Phase 14 — Self-Hosting Validation Closure Report
+
+Date: 2025-12-13
+Phase ID: Phase 14 (self-hosting-validation)
+Outcome: Failure (non-passing)
+
+## Phase Objective
+- Validate self-hosted pipeline determinism and artifact generation for the canonical DSL sample specs.
+- Produce deterministic `manifest.json`, `summary.json`, and `generated/` outputs for the self-host run.
+
+## Execution Summary
+- Ran self-host pipeline using a canonical sample spec twice; artifacts saved to `artifacts/self_hosting_run_1` and `artifacts/self_hosting_run_2`.
+- Determinism verification (manifest + summary + generated outputs) succeeded for the runs executed in the self-host pipeline.
+- Executed a full test suite locally: 224 passing, 55 failing (triaged into categories in `docs/self_hosting_failure_triage.md`).
+- All failures were classified; no fixes or functional changes were applied during this validation phase.
+
+## Determinism Status
+- Determinism Status: FAIL (overall validation cannot pass due to failing tests and blocking defects).
+- Note: The deterministic nature of the pipeline artifacts was confirmed where the self-host runs completed successfully; however the presence of 55 failing tests blocks closure as 'pass'.
+
+## Blocking Defect Categories (Summary)
+- Generator lockfile / preflight contract enforcement (raises exceptions instead of returning violations).
+- DSL version enforcement mismatch (`metadata.spec_format` vs `dsl_version` expectations).
+- Evidence bundle API incompatibility (`invariants` and `graph` required by new signatures).
+- CodeGenerator output shape mismatches (variable list/dict shapes cause downstream failures).
+- Pointer map canonicalization vs index-based pointer usage (AST vs pointer_map mismatches).
+- Pointer coverage report shape inconsistencies.
+- Checklist ID canonicalization differences (stable ID expected vs legacy `TASK-####`).
+- JSON canonical formatting (unicode escaping vs direct characters causing deterministic formatting differences).
+- Cross-section return type mismatches (functions returning wrong types in code paths).
+- Self-host manifest summary generation missing on validation failures.
+
+For full details, see: `docs/self_hosting_failure_triage.md`.
+
+## Explicit Closure Statement
+- Phase 14 is CLOSED with a non-passing outcome (`FAIL`).
+- The self-hosting pipeline achieved deterministic artifact generation in the narrow self-host run, but the engine, codegen, and checklist contracts are not fully compatible with the spec schema changes performed during Phase 13.
+- No code fixes were applied during this phase; only triage and validation were performed per the Phase 14 contract.
+
+## Required Inputs for Remediation Phase
+To proceed with remediation and for re-attempting Phase 14 validation, the following inputs and actions are required:
+- Per-RFC PRs to address each triaged category, including tests and schema updates where needed.
+- A prioritization ticket for the top blocking defects: generator contract, DSL mapping, evidence bundle compatibility, codegen shape standardization.
+- A CI gating plan ensuring PR-level tests include the deterministic self-host run and `pytest -q` pass conditions.
+- Migration guidance for legacy specs and a `SHIELDCRAFT_LENIENT_TYPES` migration guard where appropriate.
+- A checklist of acceptance criteria and a re-test plan for Phase 14: self-host determinism + full test suite pass.
+
+## Handoff
+- This closure report, the triage file (`docs/self_hosting_failure_triage.md`), and the artifacts in `artifacts/self_hosting_run_1` and `_run_2` are the canonical handoff for the remediation phase.
+- For each triaged category, create targeted PRs that follow the RFC per-PR structure and run CI with the full test suite and self-host validation before marking as resolved.
+
+---
+
+Status: CLOSED (FAIL)
+

docs/archive/phases/backups/phase_14_contract.md.orig

@@ -0,0 +1,48 @@
+# Phase 14: Self-Hosting Validation — Contract
+
+Phase name: self-hosting-validation
+
+Objective
+
+- Validate that ShieldCraft Engine (SE) can consume its own canonical specs and run the self-host pipeline deterministically.
+- Verify self-host dry-run outputs (manifest, summary, generated files) are deterministic and conform to schema and governance checks.
+
+Scope & Allowed Actions
+
+- Allowed actions:
+  - Self-host pipeline execution and dry-run validations.
+  - Artifact collection and deterministic comparisons (e.g., codegen previews).
+  - Preflight and governance validation, contract enforcement, and pointers coverage checks.
+  - Migration guard testing flags (e.g., `SHIELDCRAFT_LENIENT_TYPES`).
+  - Non-invasive verification scripts and test additions for validation only.
+
+- Forbidden actions:
+  - New features or DSL extensions.
+  - Production refactoring beyond validation-focused fixes.
+  - Modifying the canonical DSL schema in ways not approved by an RFC.
+
+Determinism Requirements
+
+- All verification runs must be deterministic across repeated runs.
+- Determinism verification must include: spec fingerprint, checklist items fingerprint, codegen preview content hashes, `generators/lockfile.json` hash, and any lineage or provenance signatures.
+
+Entry Conditions
+
+- Phase 13 is CLOSED and all approved RFCs have implementation branches isolated.
+- The `main` branch working tree is clean (no tracked file modifications under `spec/`, `src/`, or `tests/`).
+- All spec changes being validated are present in RFC-specific branches (one RFC → one branch).
+
+Exit Criteria
+
+- Phase 14 validation must produce deterministic verification artifacts for baseline spec(s).
+- Approval recorded in `docs/progress.md` and `docs/phase_14_completion.md` once all validation tasks have passed per acceptance criteria.
+- No changes to canonical DSL or cross-RFC code are merged without separate RFC approval.
+
+Governance & Exceptions
+
+- Any change required to pass validation must follow Phase 13/Phase 14 policies and be implemented via an RFC if it alters the spec or DSL behavior.
+- Bugs or minor fixes limited to validation scaffolding or test harness can be merged under Phase 14 only if they are strictly verification scaffolding and do not alter runtime behavior.
+
+---
+
+Status: STARTED (notified in `docs/progress.md`)

docs/archive/phases/backups/phase_15_contract.md.orig

@@ -0,0 +1,57 @@
+# Phase 15 — Self-Hosting Remediation Contract
+
+Phase name: self-hosting-remediation (Phase 15)
+
+## Objective
+- Remediate the blocking defects discovered during Phase 14 self-hosting validation so the self-host pipeline deterministically passes the full test suite and artifact validation.
+
+## Scope and Allowed Actions
+- Allowed actions:
+  - Targeted code fixes mapped to triaged failure categories in `docs/self_hosting_failure_triage.md`.
+  - Spec/schema changes where explicitly required to resolve contract mismatches; spec changes must follow the spec-first rule and correspond to an RFC or a patch entry created by the product team.
+  - Test updates or additions that validate the intended authoritative behavior.
+  - Backwards-compatibility guards (e.g., feature flags, migration environment variables) where the fix would otherwise break legacy behavior.
+- Forbidden actions:
+  - No new features, DSL extensions, or broad refactors unrelated to the triaged categories.
+  - No cross-RFC work; a PR must implement a single triaged failure category or a single RFC.
+
+## Process Rules
+- One triaged failure category per PR. A PR can include multiple tests and small refactors that directly support that single category, but not multi-category changes.
+- Spec-first rule: If a change requires a spec update, the first commit in the PR must be the spec/schema change, followed by code and test changes implementing it.
+- PRs must include a deterministic test harness showing deterministic outputs remain stable (self-host manifest/hash comparison) and a CI run with pytest -q passing for modified areas.
+- Migration guards: If a fix alters default behavior for older specs, include a guard (environment variable or `feature_gate` in metadata) and tests that show both migration and canonical behavior.
+- Backwards-compatibility: Ensure code remains compatible with canonical DSL; test both canonical and sample legacy specs preserved by migration guard.
+
+## Determinism and Compatibility Requirements
+- Determinism: The self-host run must produce byte-for-byte identical `manifest.json` and `summary.json` hashes across repeated runs on the same environment and spec.
+- No randomization: No runtime timestamps or environment-based paths should pollute deterministic artifacts.
+- Compatibility: When changes break legacy flows, a migration guard must be provided and the documentation updated.
+
+## Entry Conditions
+- `docs/self_hosting_failure_triage.md` and `docs/phase_14_closure.md` exist and are authoritative snapshots of the triage and closure.
+- PRs must be scoped to a single triaged failure category and include the spec-first commit when required.
+- No new RFCs outside the triage scope should be introduced in Phase 15.
+
+## Exit Criteria
+- All triaged failure categories marked as remediated with passing PRs.
+- Full test suite (`pytest -q`) passes locally and in CI, including the deterministic self-host run run twice with identical artifacts.
+- Updated migration guidance and docs where backward compatibility guards are provided.
+- All changes merged into a Phase 15 `remediation` branch and tested in the CI pipeline before re-attempting Phase 14 validation.
+
+## Governance
+- Approvals required: Code changes must pass a code review by the engineering lead and a spec validation review if spec changes are included.
+- All PRs must include tests that reproduce the original failure and validate the remediation.
+
+## Priorities
+- Fixes are prioritized in the following order:
+  1. Generator lockfile / preflight contract behavior
+  2. Engine/DSL mapping (`spec_format` => `dsl_version` mapping)
+  3. Evidence bundle compatibility
+  4. Codegen output shape standardization
+  5. Pointer map canonicalization
+  6. Pointer coverage shape
+  7. Checklist ID canonicalization
+  8. JSON canonical formatting
+  9. Cross-section types enforcement
+  10. Self-host summary completeness
+