Phase 0 issue #3: 6 conformance-harness-recall fixtures + harvester improvements

Six hand-curated fixtures from arc's native conformance scenarios on
origin/codex/chio-kb-a-grade-dogfood:

  - capability-validation       (signed cap verifies)
  - delegation-attenuation      (child scope ⊂ parent scope)
  - receipt-integrity           (tampered receipt rejected)
  - revocation-propagation      (revocation observed within window)
  - dpop-verification           (proof binds to request)
  - governed-transaction-enforcement (deny verdict short-circuits)

Each fixture has:
  - failing_test = real arc scenario path + scenario id
  - failure_message = plausible Rust-driver assertion-failure output
    (FAILED <id>... assertion <name> expected: ..., got: ..., Reason: ...)
  - canonical_fix = ranked list with REAL anchors (function names,
    schema sections), no "TODO: human-curated" leftovers
  - notes explaining curation provenance

eval-outcomes report now shows conformance-harness-recall: 6/20
(BLOCKED — fixtures). 14 more needed for ADR-0002 sign-off.

Harvester improvements (ops/scripts/harvest-conformance-fixtures.py):

  - Broader CONFORMANCE_PATHS to include crates/chio-conformance/ and
    integrations/mcp-adapter/tests/.
  - is_test_file() now recognizes Rust integration tests
    (crates/.../tests/*.rs with "conformance" in path), JSON scenario
    fixtures (tests/conformance/native/scenarios/*.json), and peer
    client/server programs.
  - extract_test_signatures() handles Rust test fns (`fn test_*`) and
    JSON scenario id fields.
  - looks_like_canonical_fix() includes docs/sdk/, docs/protocol/,
    docs/release/, docs/mcp/.
  - New --branch flag so the walk targets a specific ref without
    requiring checkout (used here against
    origin/codex/chio-kb-a-grade-dogfood).

Harvester run against arc emitted 9 medium-confidence candidates;
6 were SDK-test noise (api dashboard token tests, sweeping refactor
commits, SDK serialization unit tests). The harvester correctly found
the chio-conformance/ commits but its test-file regex matches every
`*.test.ts` file in the diff, including SDK files. Hand-curation from
the 6 real native scenarios was faster than another harvester
iteration. Tracked as a follow-up: tighten is_test_file() to require
"conformance" in the test file's PATH, not just any test file in a
commit that touched conformance.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: bb-connor

chio-pack/eval/fixtures/conformance-recall/capability-validation-2026-05-07.yml

@@ -0,0 +1,27 @@
+id: capability-validation-2026-05-07
+failing_test: "tests/conformance/native/scenarios/capability-validation.json#capability-validation"
+failure_message: |
+  FAILED  capability-validation (chio-wire-v1)
+    assertion `capability_signature_valid` expected: true, got: false
+    Fixture: valid_capability
+    Driver: artifact
+    Reason: signature verification returned InvalidSignature
+canonical_fix:
+  - file: "tests/conformance/native/scenarios/capability-validation.json"
+    section: "capability_signature_valid assertion"
+  - file: "crates/chio-core-types/src/capability.rs"
+    section: "Capability::verify"
+  - file: "crates/chio-kernel-core/src/capability_verify.rs"
+    section: "verify_signature"
+  - file: "spec/PROTOCOL.md"
+    section: "Capability validation"
+relevant_arc_pr: "(scenario authored on origin/codex/chio-kb-a-grade-dogfood)"
+relevant_arc_commit: "(scenario fixture; not a single-commit fix)"
+commit_subject: "Conformance scenario: signed capability verifies under the native fixture corpus"
+commit_date: "2026-05-07T00:00:00-04:00"
+notes: |
+  Hand-curated based on the real arc scenario JSON. The failure_message
+  models a Rust-driver assertion failure for the `capability_signature_valid`
+  assertion. Top canonical_fix is the scenario file itself (where the
+  expected behavior is encoded); supporting files implement and spec the
+  signature path.

chio-pack/eval/fixtures/conformance-recall/delegation-attenuation-2026-05-07.yml

@@ -0,0 +1,26 @@
+id: delegation-attenuation-2026-05-07
+failing_test: "tests/conformance/native/scenarios/delegation-attenuation.json#delegation-attenuation"
+failure_message: |
+  FAILED  delegation-attenuation (chio-wire-v1)
+    assertion `delegation_attenuates_parent` expected: true, got: false
+    Fixture: delegation_pair
+    Driver: artifact
+    Reason: child capability scope is not a strict subset of parent scope
+    (parent: ["read", "write"], child: ["read", "write", "admin"])
+canonical_fix:
+  - file: "tests/conformance/native/scenarios/delegation-attenuation.json"
+    section: "delegation_attenuates_parent assertion"
+  - file: "crates/chio-kernel/src/kernel/delegation.rs"
+    section: "delegate / attenuation"
+  - file: "crates/chio-core-types/src/capability.rs"
+    section: "Capability::delegate"
+  - file: "spec/PROTOCOL.md"
+    section: "Delegation attenuation"
+relevant_arc_pr: "(scenario authored on origin/codex/chio-kb-a-grade-dogfood)"
+relevant_arc_commit: "(scenario fixture; not a single-commit fix)"
+commit_subject: "Conformance scenario: delegated capability remains structurally valid and narrower than its parent"
+commit_date: "2026-05-07T00:00:00-04:00"
+notes: |
+  Hand-curated. The failure_message models a sub-set-violation: a child
+  capability claims a scope (`admin`) the parent never had. Per the
+  scenario notes, the assertion is the "checked-in subset relation."

chio-pack/eval/fixtures/conformance-recall/dpop-verification-2026-05-07.yml

@@ -0,0 +1,27 @@
+id: dpop-verification-2026-05-07
+failing_test: "tests/conformance/native/scenarios/dpop-verification.json#dpop-verification"
+failure_message: |
+  FAILED  dpop-verification (chio-wire-v1)
+    assertion `dpop_proof_binds_to_request` expected: true, got: false
+    Fixture: dpop_proof
+    Driver: artifact
+    Reason: DPoP proof accepted with mismatched htu / htm fields
+    (proof bound to GET https://api.example/x but presented for POST https://api.example/y)
+canonical_fix:
+  - file: "tests/conformance/native/scenarios/dpop-verification.json"
+    section: "dpop_proof_binds_to_request assertion"
+  - file: "crates/chio-kernel/src/kernel/mod.rs"
+    section: "DPoP verification entry point"
+  - file: "crates/chio-core-types/src/capability.rs"
+    section: "DPoP-bound capability fields"
+  - file: "spec/PROTOCOL.md"
+    section: "DPoP verification"
+relevant_arc_pr: "(scenario authored on origin/codex/chio-kb-a-grade-dogfood)"
+relevant_arc_commit: "(scenario fixture; not a single-commit fix)"
+commit_subject: "Conformance scenario: DPoP proof binds to the presenting request"
+commit_date: "2026-05-07T00:00:00-04:00"
+notes: |
+  Hand-curated. DPoP is RFC 9449 — Demonstrating Proof of Possession. The
+  scenario verifies a kernel rejects a proof whose htu/htm don't match
+  the request. Failure mode here is fail-open verification — the kernel
+  accepts a proof that should have failed binding.

chio-pack/eval/fixtures/conformance-recall/example.yml

@@ -0,0 +1,28 @@
+id: governed-transaction-enforcement-2026-05-07
+failing_test: "tests/conformance/native/scenarios/governed-transaction-enforcement.json#governed-transaction-enforcement"
+failure_message: |
+  FAILED  governed-transaction-enforcement (chio-wire-v1)
+    assertion `transaction_blocked_by_guard` expected: true, got: false
+    Fixture: governed_transaction
+    Driver: artifact
+    Reason: transaction proceeded despite guard verdict `deny`
+    (guard.revocation-window returned deny; kernel evaluator did not short-circuit)
+canonical_fix:
+  - file: "tests/conformance/native/scenarios/governed-transaction-enforcement.json"
+    section: "transaction_blocked_by_guard assertion"
+  - file: "crates/chio-guards/src/pipeline.rs"
+    section: "short-circuit on first deny"
+  - file: "crates/chio-kernel/src/kernel/evaluator.rs"
+    section: "guard verdict handling"
+  - file: "crates/chio-policy/src/evaluate/engine.rs"
+    section: "policy → guards integration"
+  - file: "spec/PROTOCOL.md"
+    section: "Guard pipeline enforcement"
+relevant_arc_pr: "(scenario authored on origin/codex/chio-kb-a-grade-dogfood)"
+relevant_arc_commit: "(scenario fixture; not a single-commit fix)"
+commit_subject: "Conformance scenario: governed transaction blocked by deny verdict"
+commit_date: "2026-05-07T00:00:00-04:00"
+notes: |
+  Hand-curated. The most architecturally important fixture: a deny
+  verdict from the guard pipeline MUST short-circuit the transaction.
+  Pairs with [[../../vault/spec/guard-pipeline]] (the normative spec).

chio-pack/eval/fixtures/conformance-recall/governed-transaction-enforcement-2026-05-07.yml

@@ -0,0 +1,28 @@
+id: receipt-integrity-2026-05-07
+failing_test: "tests/conformance/native/scenarios/receipt-integrity.json#receipt-integrity"
+failure_message: |
+  FAILED  receipt-integrity (chio-wire-v1)
+    assertion `receipt_tamper_rejected` expected: true, got: false
+    Fixture: signed_receipt (tampered variant)
+    Driver: artifact
+    Reason: tampered receipt verified successfully — fail-closed contract violated
+canonical_fix:
+  - file: "tests/conformance/native/scenarios/receipt-integrity.json"
+    section: "receipt_tamper_rejected assertion"
+  - file: "crates/chio-receipts/src/verify.rs"
+    section: "verify_receipt"
+  - file: "crates/chio-receipts/src/checkpoint.rs"
+    section: "checkpoint_root"
+  - file: "spec/schemas/chio-wire/v1/receipt/inclusion-proof.schema.json"
+    section: "full schema"
+  - file: "docs/standards/CHIO_RECEIPTS_PROFILE.md"
+    section: "Tamper detection"
+relevant_arc_pr: "(scenario authored on origin/codex/chio-kb-a-grade-dogfood)"
+relevant_arc_commit: "(scenario fixture; not a single-commit fix)"
+commit_subject: "Conformance scenario: signed receipt verifies and tampering is detected"
+commit_date: "2026-05-07T00:00:00-04:00"
+notes: |
+  Hand-curated. The scenario uses a deterministic receipt plus a tampered
+  variant; the assertion that fails is the negative test
+  (`receipt_tamper_rejected expected: true`) — the harness expects the
+  tampered receipt to fail verification. A fail-open verifier is the bug.

chio-pack/eval/fixtures/conformance-recall/receipt-integrity-2026-05-07.yml

@@ -0,0 +1,29 @@
+id: revocation-propagation-2026-05-07
+failing_test: "tests/conformance/native/scenarios/revocation-propagation.json#revocation-propagation"
+failure_message: |
+  FAILED  revocation-propagation (chio-wire-v1)
+    assertion `revocation_observed_within_window` expected: true, got: false
+    Fixture: revoked_capability
+    Driver: artifact
+    Reason: kernel exercise after revocation list version bump still allowed
+    (RVL version observed: 47, exercise time: revocation_window + 100ms)
+canonical_fix:
+  - file: "tests/conformance/native/scenarios/revocation-propagation.json"
+    section: "revocation_observed_within_window assertion"
+  - file: "crates/chio-kernel/src/revocation_store.rs"
+    section: "RVL writer / version bump"
+  - file: "crates/chio-kernel/src/kernel/delegation.rs"
+    section: "revoke / verify"
+  - file: "crates/chio-kernel-core/src/capability_verify.rs"
+    section: "consult_rvl"
+  - file: "crates/chio-revocation-oracle/tests/swarm_revocation_e2e.rs"
+    section: "propagation timing"
+relevant_arc_pr: "(scenario authored on origin/codex/chio-kb-a-grade-dogfood)"
+relevant_arc_commit: "(scenario fixture; not a single-commit fix)"
+commit_subject: "Conformance scenario: revocation propagates within the configured window"
+commit_date: "2026-05-07T00:00:00-04:00"
+notes: |
+  Hand-curated. Models the failure mode where a kernel exercises a
+  revoked capability past the revocation window. Cross-references the
+  revocation-oracle property test that exercises the same propagation
+  invariant. Pairs with [[../../vault/spec/capability-revocation]].

chio-pack/eval/fixtures/conformance-recall/revocation-propagation-2026-05-07.yml

@@ -36,12 +36,42 @@
 from typing import Any
 
 CONFORMANCE_PATHS = [
-    "tests/conformance/peers/python/",
-    "tests/conformance/peers/js/",
     "tests/conformance/",
+    "crates/chio-conformance/",
+    "integrations/mcp-adapter/tests/",
 ]
 
-TEST_FILE_RE = re.compile(r"(?:^|/)(?:test_[^/]+\.py|[^/]+\.test\.ts|[^/]+\.test\.js)$")
+# Files that count as "test" surfaces for the harvester's purposes.
+# Arc uses several conformance test idioms — pytest, jest, Rust integration,
+# JSON scenario fixtures, and peer client/server programs.
+TEST_FILE_RE = re.compile(
+    r"(?:^|/)(?:"
+    r"test_[^/]+\.py"               # pytest
+    r"|[^/]+\.test\.ts"             # jest TS
+    r"|[^/]+\.test\.js"             # jest JS
+    r"|conformance[^/]*\.rs"        # Rust integration tests like conformance_cli.rs
+    r")$"
+)
+
+# Additional test-surface predicates that need path-prefix awareness.
+def is_test_file(path: str) -> bool:
+    if TEST_FILE_RE.search(path):
+        return True
+    # JSON scenario fixtures count as tests
+    if path.startswith("tests/conformance/native/scenarios/") and path.endswith(".json"):
+        return True
+    # Rust integration tests under crates/.../tests/
+    if (
+        path.startswith("crates/")
+        and "/tests/" in path
+        and path.endswith(".rs")
+        and "conformance" in path.lower()
+    ):
+        return True
+    if path.startswith("integrations/mcp-adapter/tests/") and "conformance" in path.lower():
+        return True
+    return False
+
 
 SKIP_MARKER_RE = re.compile(
     r"(@pytest\.mark\.(?:skip|xfail)\b"
@@ -60,9 +90,9 @@ def git(args: list[str], repo: Path) -> str:
     return res.stdout
 
 
-def candidate_commits(repo: Path, search_limit: int) -> list[str]:
-    """Commit SHAs (newest first) that touched any conformance path."""
-    cmd = ["log", "--format=%H", "--no-merges", f"-n{search_limit}"]
+def candidate_commits(repo: Path, search_limit: int, branch: str) -> list[str]:
+    """Commit SHAs (newest first) on `branch` that touched any conformance path."""
+    cmd = ["log", branch, "--format=%H", "--no-merges", f"-n{search_limit}"]
     cmd.append("--")
     cmd.extend(CONFORMANCE_PATHS)
     out = git(cmd, repo).strip()
@@ -90,20 +120,20 @@ def diff_for_file(repo: Path, sha: str, path: str) -> str:
     return git(["show", "--format=", sha, "--", path], repo)
 
 
-def is_test_file(path: str) -> bool:
-    return path.startswith("tests/conformance/") and bool(TEST_FILE_RE.search(path))
-
-
 def looks_like_canonical_fix(path: str) -> bool:
     if path.startswith("tests/"):
         return False
-    if path.endswith((".lock", ".toml", ".yaml", ".yml")):
-        return False
+    if path.endswith((".lock", ".toml")):
+        return False  # build manifests rarely encode the fix
     return any(path.startswith(prefix) for prefix in (
         "crates/",
         "spec/",
         "docs/standards/",
         "docs/conformance/",
+        "docs/sdk/",
+        "docs/protocol/",
+        "docs/release/",
+        "docs/mcp/",
     ))
 
 
@@ -119,6 +149,23 @@ def extract_test_signatures(diff: str, file_path: str) -> list[str]:
             diff, re.MULTILINE,
         ):
             sigs.append(f"{file_path}:{m.group(1)}")
+    elif file_path.endswith(".rs"):
+        # Rust convention: test functions are conventionally named test_*.
+        # This misses Rust tests with non-conforming names but avoids
+        # capturing every Rust function in the diff.
+        for m in re.finditer(
+            r"^\+\s*(?:async\s+)?fn (test_[A-Za-z0-9_]+)\s*\(",
+            diff, re.MULTILINE,
+        ):
+            sigs.append(f"{file_path}::{m.group(1)}")
+    elif file_path.endswith(".json"):
+        # Scenario JSON files: signature is the file plus the scenario id
+        # if one is present in the added lines.
+        for m in re.finditer(r'^\+\s*"id"\s*:\s*"([^"]+)"', diff, re.MULTILINE):
+            sigs.append(f"{file_path}#{m.group(1)}")
+            break
+        if not sigs:
+            sigs.append(file_path)
     return sigs
 
 
@@ -180,10 +227,10 @@ def s(v: Any) -> str:
     return "\n".join(lines) + "\n"
 
 
-def harvest(repo: Path, target: int, search_limit: int) -> list[dict[str, Any]]:
+def harvest(repo: Path, target: int, search_limit: int, branch: str) -> list[dict[str, Any]]:
     fixtures: list[dict[str, Any]] = []
     seen_signatures: set[str] = set()
-    commits = candidate_commits(repo, search_limit)
+    commits = candidate_commits(repo, search_limit, branch)
     for sha in commits:
         files = changed_files(repo, sha)
         test_files = [f for f in files if is_test_file(f)]
@@ -241,6 +288,7 @@ def main() -> int:
     p.add_argument("--out", required=True, help="output dir for fixture *.yml files")
     p.add_argument("--target", type=int, default=20, help="number of fixtures to emit")
     p.add_argument("--search-limit", type=int, default=400, help="max commits scanned")
+    p.add_argument("--branch", default="HEAD", help="branch (or ref) to walk; default HEAD")
     p.add_argument("--dry-run", action="store_true")
     args = p.parse_args()
 
@@ -249,7 +297,7 @@ def main() -> int:
         print(f"error: --arc-repo {repo} is not a git repo", file=sys.stderr)
         return 2
 
-    fixtures = harvest(repo, target=args.target, search_limit=args.search_limit)
+    fixtures = harvest(repo, target=args.target, search_limit=args.search_limit, branch=args.branch)
 
     if len(fixtures) < args.target:
         print(