Phase 6: persona arbitration simplification (inventory, authority metadata, routing, event compression, deprecations, docs, tests)

Author: Dee66

docs/governance/INVARIANTS.md

@@ -180,3 +180,13 @@ This file declares the governance anchor; enforcement logic will be implemented
 
 - Semantic Outcome Lock: The canonical semantics (primary outcome derivation, item roles, and invariants) are locked under Phase 5 and may not be altered except via an explicit governance phase update. Changes to semantic meaning require a recorded governance decision and a corresponding implementation phase.
 
+## Persona Arbitration Invariant (Phase 6)
+
+- Statement: Persona authority and routing MUST be explicit, deterministic, and auditable. Persona outputs are evidence and may be compressed into a single primary persona cause for audit, but persona outputs MUST NOT arbitrarily change canonical checklist semantics without a governance decision.
+- Rules:
+  - Personas MAY declare an optional `authority` of one of: `DECISIVE`, `ADVISORY`, `ANNOTATIVE` (metadata only in Phase 6).
+  - Routing of persona invocation MUST be static and derived from the explicit routing table in `src/shieldcraft/persona/routing.py` (if configured); otherwise persona discovery falls back to `scope` rules.
+  - Persona events are recorded atomically and deterministically in `artifacts/persona_events_v1.json` and hashed for integrity.
+  - Persona outputs are compressed into a `checklist.persona_summary` structure for deterministic auditability; compression does not change primary checklist outcome semantics.
+- Enforcement: These invariants are enforced by documentation, deterministic routing, persona metadata, and persona event compression implemented in `finalize_checklist` (Phase 6).
+

docs/governance/PERSONA_AUTHORITY_MODEL.md

@@ -0,0 +1,31 @@
+# Persona Authority Model (Authoritative)
+
+This file defines the Phase 6 **Persona Authority Model** (classification and metadata only).
+
+Purpose: introduce a single, explicit authority class for personas to make decision authority auditable and deterministic.
+
+## Authority classes
+
+- `DECISIVE`
+  - May cause REFUSAL outcomes by emitting vetoes (via `emit_veto`).
+  - Decision-impacting; use sparingly and require explicit rationale in persona definitions.
+
+- `ADVISORY`
+  - May emit non-blocking constraints and BLOCKER-class signals (via `allowed_actions`), but cannot directly cause REFUSAL.
+
+- `ANNOTATIVE`
+  - Emit annotations and decisions recorded as evidence only. Non-authoritative for pipeline control.
+
+## Implementation note (Phase 6)
+
+- Personas may include an optional top-level field `authority` in their persona JSON to declare one of the above classes. This is metadata-only and does not change runtime behavior by itself.
+- The codebase now supports an optional `authority` attribute on `Persona` and `PersonaContext` dataclasses to carry this metadata deterministically.
+- Enforcement (mapping DECISIVE->veto permission, etc.) will be introduced in a follow-up Phase 6 runtime step after governance review.
+
+## Auditability
+
+- Persona authority MUST be recorded in PersonaEvents and engine-side persona decision records when decisions are emitted so that audits can reconstruct which persona class made the decision.
+
+***
+
+This document is authoritative for Phase 6 classification metadata and is intentionally declarative.

docs/governance/PERSONA_DECISION_SURFACE.md

@@ -0,0 +1,65 @@
+# Persona Decision Surface (Inventory)
+
+This document enumerates, facts-only, the observable persona decision points, veto paths, and annotation APIs in the codebase. It is an inventory for Phase 6 work and contains no proposals.
+
+## Key APIs
+
+- `shieldcraft.persona.emit_veto(engine, persona_ctx, phase, code, explanation, severity)`
+  - Capability: **veto** (records a veto in engine state, emits persona events/annotations)
+  - Audit: writes to `engine._persona_vetoes` and emits PersonaEvent/PersonaAnnotation via observability
+  - Usage: called by persona evaluation code and tests
+
+- `shieldcraft.persona.emit_annotation(engine, persona_ctx, phase, message, severity)`
+  - Capability: **annotate** (non-authoritative; emits annotations and PersonaEvent)
+  - Audit: appends to engine `_persona_annotations` and emits PersonaEvent
+
+- `shieldcraft.persona.decision_record.record_decision(engine, persona_id, phase, decision)`
+  - Capability: **decision** (records non-blocking persona decisions as PersonaEvents)
+  - Audit: appends to engine `_persona_decisions` and emits PersonaEvent
+
+## Evaluation & Enforcement Points
+
+- `shieldcraft.persona.persona_evaluator.evaluate_personas(engine, personas, items, phase="checklist")`
+  - Scope: **Generator** (ChecklistGenerator.build calls this before persona-influenced checks)
+  - Decision types: **veto** (emit_veto), **constraint** (non-mutating instruction recorded via `record_decision`)
+  - Determinism: iterates personas deterministically by `persona.name`
+
+- `shieldcraft.services.validator.persona_gate.enforce_persona_veto(engine)`
+  - Scope: **Validator / Generator / Readiness evaluator** (called from ChecklistGenerator and readiness evaluator)
+  - Decision type: **veto enforcement** (raises `RuntimeError("persona_veto:...")` when vetoes present)
+  - Determinism: selects single veto deterministically (sort by severity then persona_id)
+
+- `Engine.preflight` persona veto check
+  - Scope: **Engine preflight** (top-level preflight gate checks `engine._persona_vetoes`) 
+  - Decision type: **REFUSAL** (records `G7_PERSONA_VETO` event and raises `RuntimeError("persona_veto:...")`)
+
+## Persona Loading & Discovery
+
+- `shieldcraft.persona.load_persona(path)`
+  - Scope: **persona discovery / engine self-host** (persona files loaded when persona feature enabled)
+  - Decision types: **load-time errors** (raises `PersonaError` on invalid persona files, `SyncError` on repo sync failures)
+
+- `shieldcraft.persona.persona_registry` (register/list/find_personas_for_phase)
+  - Scope: **in-memory registry for deterministic test/runtime persona evaluation**
+  - Behavior: `find_personas_for_phase(phase)` filters by persona `scope` (phase or `all`)
+
+## Observability / Audit
+
+- `shieldcraft.observability.emit_persona_event(engine, persona_id, capability, phase, payload_ref, severity)`
+  - Records PersonaEvent payloads (`persona_events_v1.json`) deterministically and emits a companion hash
+
+- `shieldcraft.observability.emit_persona_annotation(engine, persona_id, phase, message, severity)`
+  - Records persona annotations (`persona_annotations_v1.json`) deterministically
+
+## Tests and Instrumentation (examples)
+
+- Tests call `emit_veto` and `emit_annotation` directly to simulate persona-driven vetoes and annotations
+- `persona_evaluator.evaluate_personas` is exercised in generator tests (ensures constraints are recorded and applied)
+
+## Observations (facts-only)
+
+- Persona APIs are intentionally conservative and deterministic (sorted iteration, deterministic discovery rules).
+- Persona decisions are recorded via PersonaEvent/Annotation and stored on the `engine` object for deterministic inspection.
+- Veto enforcement is a deterministic, centralized check (`enforce_persona_veto`) used in multiple phases.
+
+This completes the factual inventory of persona decision points in the current codebase.

docs/governance/PERSONA_PROTOCOL_DEPRECATIONS.md

@@ -0,0 +1,18 @@
+# Persona Protocol Deprecations (Phase 6)
+
+This file records **deprecated** or **archived** persona protocol mechanisms identified during Phase 6 so they can be audited and retired in a future phase. No deletions are performed in this phase — this document is *informational* and *non-authoritative* about policy.
+
+Deprecated / Archived items (facts-only):
+
+- **Legacy persona profiles** (`docs/persona/legacy/*`)
+  - Rationale: these persona profiles are subordinate to the canonical `PERSONA_PROTOCOL.md` and are retained for historical/contextual reference only.
+
+- **Ad-hoc persona routing by phase or discovery heuristics**
+  - Rationale: implicit routing is superseded by the explicit routing table implemented in `src/shieldcraft/persona/routing.py` (Phase 6 introduces a deterministic, static routing table).
+
+- **Implicit authority inference from persona role or phase**
+  - Rationale: Phase 6 introduces an explicit `authority` classification (DECISIVE|ADVISORY|ANNOTATIVE); implicit inference from role/phase is deprecated and will be considered for removal in a later phase after governance review.
+
+Notes:
+
+- This document is intentionally conservative: it does not remove or change any code. It only flags mechanisms that are considered for future deprecation and records the rationale and candidate replacements introduced in Phase 6.

docs/governance/decision_log.md

@@ -118,3 +118,14 @@ Authoritative decisions made on 2025-12-13 (Phase 13 kickoff). All decisions rec
   - Semantic stability assertion added to ensure deterministic, byte-stable output for identical inputs.
   - No persona, schema, or gate-ID changes were made; all logic was centralized in `finalize_checklist`.
 - Rationale: Lock semantic meanings to prevent regressions and ensure deterministic artifact semantics; changes to semantics require governance approval and a new implementation phase.
+
+
+## Phase 6 Closed — Persona Arbitration Simplified
+- Decision: RECORD
+- Summary (facts-only): Phase 6 (Persona Arbitration Simplification) is complete. Persona authority, routing, and event compression have been introduced as defensive, deterministic, metadata-only changes and documentation. Persona outputs remain auditable evidence and do not, by themselves, alter canonical checklist semantics.
+- Explicit state:
+  - A Persona Authority Model (DECISIVE|ADVISORY|ANNOTATIVE) has been introduced as metadata; persona dataclasses now carry an optional `authority` attribute.
+  - Deterministic, static persona routing (configurable) is available via `src/shieldcraft/persona/routing.py`; discovery falls back to `scope` when routing is not configured.
+  - Persona events are compressed into a deterministic `checklist.persona_summary` structure for auditability; primary persona cause is selected deterministically and traceably.
+  - Marked legacy persona protocol artifacts for deprecation and archived legacy docs as informational only.
+- Rationale: Simplify persona arbitration while preserving auditability, determinism, and decision authority; future runtime enforcement of authority classes will be gated by governance decisions.

src/shieldcraft/engine.py

@@ -183,6 +183,53 @@ def _item_event_index(it):
     # Enforce semantic invariants now that roles are assigned
     _assert_semantic_invariants(checklist, primary_outcome, gate_outcomes)
 
+    # Persona event compression: summarize persona outputs deterministically
+    try:
+        persona_events = list(getattr(engine, '_persona_events', []) or [])
+        if persona_events:
+            # Map persona capability to nominal outcome for selection precedence
+            def _capability_to_outcome(cap):
+                if cap == 'veto':
+                    return 'REFUSAL'
+                # annotations/decisions are diagnostics/evidence
+                return 'DIAGNOSTIC'
+
+            # Build list with derived outcomes and preserve order
+            pes = []
+            for idx, pe in enumerate(persona_events):
+                outcome = _capability_to_outcome(pe.get('capability'))
+                pes.append({
+                    'persona_id': pe.get('persona_id'),
+                    'capability': pe.get('capability'),
+                    'phase': pe.get('phase'),
+                    'payload_ref': pe.get('payload_ref'),
+                    'severity': pe.get('severity'),
+                    'derived_outcome': outcome,
+                    'index': idx,
+                })
+
+            # Select primary persona cause by precedence and deterministic tie-break
+            prim = None
+            if any(p.get('derived_outcome') == 'REFUSAL' for p in pes):
+                candidates = [p for p in pes if p.get('derived_outcome') == 'REFUSAL']
+            elif any(p.get('derived_outcome') == 'DIAGNOSTIC' for p in pes):
+                candidates = [p for p in pes if p.get('derived_outcome') == 'DIAGNOSTIC']
+            else:
+                candidates = pes
+
+            if candidates:
+                # deterministic selection: lowest persona_id, then earliest index
+                prim = sorted(candidates, key=lambda p: (p.get('persona_id') or '', p.get('index')))[0]
+
+            checklist['persona_summary'] = {
+                'primary_persona': prim.get('persona_id') if prim else None,
+                'primary_capability': prim.get('capability') if prim else None,
+                'events': pes,
+            }
+    except Exception:
+        # Persona auditing must not interfere with finalization
+        pass
+
     # Expose primary outcome at the top-level result as required by the
     # Checklist Semantics Contract (Phase 5).
     result_primary_outcome = primary_outcome

src/shieldcraft/persona/__init__.py

@@ -72,6 +72,8 @@ class Persona:
     scope: List[str] = field(default_factory=list)
     allowed_actions: List[str] = field(default_factory=list)
     constraints: Dict[str, Any] = field(default_factory=dict)
+    # Optional authority classification for Phase 6 (DECISIVE|ADVISORY|ANNOTATIVE)
+    authority: Optional[str] = None
 
 
 @dataclass(frozen=True)
@@ -82,6 +84,7 @@ class PersonaContext:
     scope: List[str]
     allowed_actions: List[str]
     constraints: Dict[str, Any]
+    authority: Optional[str] = None
 
     def to_dict(self) -> Dict[str, Any]:
         return {
@@ -91,6 +94,7 @@ def to_dict(self) -> Dict[str, Any]:
             "scope": list(self.scope) if self.scope is not None else [],
             "allowed_actions": list(self.allowed_actions) if self.allowed_actions is not None else [],
             "constraints": dict(self.constraints) if self.constraints is not None else {},
+            "authority": self.authority,
         }
 
     def to_canonical_json(self) -> str:
@@ -287,6 +291,7 @@ def load_persona(path: str) -> Persona:
         scope=data.get("scope", []),
         allowed_actions=data.get("allowed_actions", []),
         constraints=data.get("constraints", {}),
+        authority=data.get("authority"),
     )
 
 

src/shieldcraft/persona/persona_registry.py

@@ -28,8 +28,25 @@ def list_personas() -> List[Persona]:
 
 
 def find_personas_for_phase(phase: str) -> List[Persona]:
+    """Return personas applicable to a phase.
+
+    Behavior:
+    - If a static routing table is configured (see `persona.routing`), filter
+      by allowed persona names for the phase. Otherwise fall back to persona
+      `scope` rules (existing behavior).
+    - Deterministic ordering (sorted by persona.name) is preserved.
+    """
+    from shieldcraft.persona.routing import get_allowed_persona_names_for_phase
+
     res = []
     for p in list_personas():
         if not p.scope or 'all' in p.scope or phase in p.scope:
             res.append(p)
-    return res
+
+    allowed = get_allowed_persona_names_for_phase(phase)
+    if allowed is None:
+        return res
+
+    # Filter by allowed names deterministically
+    allowed_set = set(allowed)
+    return [p for p in res if p.name in allowed_set]

src/shieldcraft/persona/routing.py

@@ -0,0 +1,28 @@
+"""Persona routing table: maps {phase} -> allowed persona names.
+
+This module centralizes static, deterministic routing of personas to phases.
+By default, routing is empty and discovery falls back to persona `scope` rules.
+Routing is pure configuration and deterministic (lexicographic ordering applied where relevant).
+"""
+from typing import Dict, List, Optional
+
+# Example shape: {"checklist": ["security_auditor", "licensor"], "preflight": ["governance"]}
+DEFAULT_ROUTING: Dict[str, List[str]] = {}
+
+
+def get_allowed_persona_names_for_phase(phase: str) -> Optional[List[str]]:
+    """Return list of allowed persona names for `phase` or None if not configured."""
+    if not DEFAULT_ROUTING:
+        return None
+    return sorted(DEFAULT_ROUTING.get(phase, []))
+
+
+def set_routing(mapping: Dict[str, List[str]]) -> None:
+    """Set routing mapping for runtime/testing. Mapping should be deterministic (string lists).
+
+    WARNING: This mutates module-level routing state and is intended to be used in deterministic
+    test and configuration flows only. Production usage should set `DEFAULT_ROUTING` via
+    a well-reviewed configuration change.
+    """
+    global DEFAULT_ROUTING
+    DEFAULT_ROUTING = {k: sorted(v) for k, v in mapping.items()}

tests/persona/test_persona_authority_model.py

@@ -0,0 +1,20 @@
+import json
+from shieldcraft.persona import Persona, PersonaContext, load_persona
+
+
+def test_persona_authority_field_roundtrip(tmp_path):
+    p = Persona(name='x', role=None, display_name=None, scope=['preflight'], allowed_actions=['annotate'], constraints={}, authority='ANNOTATIVE')
+    ctx = PersonaContext(name=p.name, role=p.role, display_name=p.display_name, scope=p.scope, allowed_actions=p.allowed_actions, constraints=p.constraints, authority=p.authority)
+    d = ctx.to_dict()
+    assert d.get('authority') == 'ANNOTATIVE'
+
+    # Now write a persona JSON and load via load_persona
+    persona_file = tmp_path / 'persona.json'
+    persona_file.write_text(json.dumps({'name': 'y', 'version': '1.0', 'persona_version': 'v1', 'authority': 'DECISIVE'}))
+    # Monkeypatch repo/worktree checks so load_persona can run in test env
+    import shieldcraft.services.sync as syncmod
+    syncmod.verify_repo_sync = lambda root: None
+    import shieldcraft.persona as pm
+    pm._is_worktree_clean = lambda: True
+    loaded = load_persona(str(persona_file))
+    assert loaded.authority == 'DECISIVE'