Harden the DYADT claim verifier against adversarial-review bypasses (#458)

## Context

Follow-up to #457 (Wave 4, merged), which introduced DYADT — the
post-action verifier that checks an agent's *claimed* outcomes actually
happened. Before considering it done, I ran an **adversarial review** of
the reference verifier (3 red-team lenses — bypass, verdict-logic,
spec-gaps — each finding independently re-verified). It found **real
ways to make a false claim pass**, or to make the verifier drop or
mis-judge a claim.

A claim-checker that can be fooled is worse than none. This PR closes
every confirmed hole **fail-safe** (return `unverifiable`, never a
confident wrong verdict) and locks each with a regression assertion
**and** a conformance vector.

## Holes closed (`scripts/verify-claims.sh`)

| Bypass found | Now |
|---|---|
| Unresolvable/empty base ref → confident-wrong
`created`/`modified`/`deleted` | `unverifiable no-base-ref` |
| `created` confirmed any existing file (untracked build output, etc.) |
requires a git-**tracked** file new to the change |
| Missing required field → false-confirm / silent drop | `unverifiable
missing-field`; a block with no `id` still appears (no silent drop) |
| Empty / `.*` `expect` → unconditional confirm | `unverifiable
empty-pattern`; malformed regex → `bad-regex` |
| `target` = absolute / `..` / symlink → evidence redirection |
`unverifiable unsafe-path` |
| `stdout-contains:` matched **stderr** too | matches stdout only
(stderr captured separately) |
| `contains:`/`sha256:` on a directory/unreadable file → `refuted` |
`unverifiable` |
| Licence claim phrased only in `statement` → auto-confirmed | licence
detected in class/target/expect/**statement** → `manual-only` |
| `claims-compose` infinite recursion (cycle / fork bomb) | depth-capped
at 8 |
| `not_before` (stale-evidence) unimplemented | present → `unverifiable`
(reference collects no timestamps) |
| Parser only accepted `key = "v"` | whitespace-tolerant (`key="v"` too)
|

## Spec

`VERIFICATION-PROTOCOL.adoc` gains two normative sections: **Fail-safe
requirements** (the exhaustive list of "cannot collect trustworthy
evidence → unverifiable") and **Command execution & sandboxing**
(`command-transcript` executes `target`; untrusted claims MUST be
sandboxed; the reference impl is trusted-input-only and says so).

## Verification

- `scripts/tests/wave4-dyadt-test.sh`: **14/14** (7 new hardening
assertions — each proves a specific bypass is now closed).
- `spec/conformance/`: **9/9** vectors (added `missing-field`,
`unsafe-path`, `licence-in-statement` so the production verifier must
handle them too).
- Dogfood `CLAIMS.a2ml` still all-confirmed; Waves 0/1/3 tests
unaffected; registry + dashboard in sync.

Method: 3-lens red-team fan-out → independent re-verification of each
finding → fix only the confirmed ones fail-safe. Licence handling stays
`manual-only` end-to-end.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---
_Generated by [Claude
Code](https://claude.ai/code/session_0114ps6mY5jAH4SzbGxeuYjc)_

Co-authored-by: Claude <noreply@anthropic.com>

Author: hyperpolymath

.machine_readable/REGISTRY.a2ml

@@ -234,7 +234,7 @@ name = "DYADT — Did-You-Actually-Do-That"
 stream = "governance"
 home = "did-you-actually-do-that/"
 canonical_doc = "did-you-actually-do-that/README.adoc"
-source_hash = "sha256:445359ddcc92b56dfc8e8a3bdc16062439f1236b5fd0f42099113e7afa86d2e0"
+source_hash = "sha256:2ae635b9ede51e76781cb7c171108f2a4505b0aae9ac97fb05c910915141eb2a"
 route = "post-action agent-claim verification (Tier 4 accountability)"
 
 [[spec]]

did-you-actually-do-that/spec/VERIFICATION-PROTOCOL.adoc

@@ -92,6 +92,45 @@ discharged by `command-transcript`.
   NOT fall back to reading the agent's cited evidence.
 . *Stale evidence is unverifiable.* If `not_before` is set and the primary
   evidence predates it, the verdict is `unverifiable` (reason `stale-evidence`).
+  A verifier that does not collect evidence timestamps MUST return
+  `unverifiable` for any claim carrying `not_before` (it cannot honour the
+  guarantee) rather than verify possibly-stale evidence as fresh.
+
+== Fail-safe requirements (normative)
+
+Every place a verifier *cannot* collect trustworthy primary evidence is
+`unverifiable`, never a confident wrong verdict. A conforming verifier MUST
+return `unverifiable` (not `confirmed`/`refuted`) when:
+
+* the diff base ref is missing or does not resolve (`no-base-ref`) for a
+  `created`/`modified`/`deleted` claim;
+* a required field (`claim_class`, `target`, `expect`, `verifier`) is absent
+  (`missing-field`), or a `[[claim]]` block carries no `id` (it MUST still
+  appear in the output — no silent drop);
+* the `target` of a file verifier is an absolute path, contains `..`, or is a
+  symlink (`unsafe-path` — evidence could be redirected);
+* an `expect` predicate is empty or otherwise always-matching
+  (`empty-pattern`), or its regex is malformed (`bad-regex`);
+* a `contains:`/`sha256:` target is not a readable regular file
+  (`not-a-regular-file` / `unreadable`).
+
+And a licence/SPDX claim — detected in ANY of `claim_class`, `target`,
+`expect`, or `statement` — MUST be `unverifiable` with reason `manual-only`
+(the Manual-Only licence policy; never auto-`confirmed`).
+
+== Command execution & sandboxing (normative)
+
+The `command-transcript` verifier EXECUTES the claim's `target`. Consequences:
+
+* `stdout-contains:` MUST match the command's *stdout only*; stderr MUST be
+  captured separately so a marker printed to stderr cannot false-confirm.
+* Because the claims file is agent-authored, executing its commands is code
+  execution. A verifier running over UNTRUSTED claims MUST sandbox the command
+  (no network, constrained filesystem, resource limits). The reference verifier
+  does NOT sandbox and is for TRUSTED (repo-authored, PR-reviewed) claims only;
+  a conforming production verifier MUST document its execution boundary.
+* `claims-compose` recursion MUST be depth-bounded to defeat self- or
+  mutually-referential claim files (the reference verifier caps depth at 8).
 
 == Multi-verifier corroboration (SHOULD)
 

did-you-actually-do-that/spec/conformance/licence-in-statement.a2ml

@@ -0,0 +1,10 @@
+[claims]
+schema = "dyadt/claim@1"
+actor = "conformance"
+[[claim]]
+id = "C1"
+claim_class = "command-ran"
+statement = "added the SPDX-License-Identifier header"
+target = "true"
+expect = "exit==0"
+verifier = "command-transcript"

did-you-actually-do-that/spec/conformance/licence-in-statement.expected

@@ -0,0 +1 @@
+C1 unverifiable

did-you-actually-do-that/spec/conformance/missing-field.a2ml

@@ -0,0 +1,9 @@
+[claims]
+schema = "dyadt/claim@1"
+actor = "conformance"
+[[claim]]
+id = "C1"
+claim_class = "command-ran"
+statement = "a claim missing its target field"
+expect = "exit==0"
+verifier = "command-transcript"

did-you-actually-do-that/spec/conformance/missing-field.expected

@@ -0,0 +1 @@
+C1 unverifiable

did-you-actually-do-that/spec/conformance/unsafe-path.a2ml

@@ -0,0 +1,10 @@
+[claims]
+schema = "dyadt/claim@1"
+actor = "conformance"
+[[claim]]
+id = "C1"
+claim_class = "file-changed"
+statement = "a path-traversal target"
+target = "../../etc/passwd"
+expect = "created"
+verifier = "git-diff"

did-you-actually-do-that/spec/conformance/unsafe-path.expected

@@ -0,0 +1 @@
+C1 unverifiable

scripts/tests/wave4-dyadt-test.sh

@@ -107,6 +107,76 @@ verifier = "command-transcript"
 EOF
 ( cd "$ROOT" && bash "$V" "$TMP/g.a2ml" >/dev/null 2>&1 ); [ $? -eq 0 ] && ok "all-confirmed file exits 0" || bad "all-confirmed file did not exit 0"
 
+echo "== hardening (adversarial-review fixes) =="
+mk() { printf '%s\n' "$2" > "$TMP/$1"; }
+reason_of() { # file id
+  cd "$ROOT" && DYADT_ALLOW_UNVERIFIABLE=1 bash "$V" "$1" 2>/dev/null \
+    | grep -E "$2|<block" | grep -oE '(confirmed|REFUTED|unverifiable) *\[[^]]*\] *[a-z-]+' | head -1
+}
+# missing required field -> unverifiable, still counted
+mk mf.a2ml '[claims]
+[[claim]]
+id = "C1"
+claim_class = "command-ran"
+expect = "exit==0"
+verifier = "command-transcript"'
+[[ "$(reason_of "$TMP/mf.a2ml" C1)" == unverifiable*missing-field ]] && ok "missing field -> unverifiable" || bad "missing field not caught"
+# claim with no id is NOT silently dropped (appears as a block, unverifiable)
+mk noid.a2ml '[claims]
+[[claim]]
+claim_class = "command-ran"
+target = "true"
+expect = "exit==0"
+verifier = "command-transcript"'
+( cd "$ROOT" && DYADT_ALLOW_UNVERIFIABLE=1 bash "$V" "$TMP/noid.a2ml" 2>/dev/null | grep -q 'no-id' ) && ok "missing id not silently dropped" || bad "missing id was dropped"
+# empty pattern -> unverifiable (not an always-match confirm)
+mk ep.a2ml '[claims]
+[[claim]]
+id = "C1"
+claim_class = "file-changed"
+target = "README.adoc"
+expect = "contains:"
+verifier = "git-diff"'
+[[ "$(reason_of "$TMP/ep.a2ml" C1)" == unverifiable*empty-pattern ]] && ok "empty contains pattern -> unverifiable" || bad "empty pattern not caught"
+# path traversal -> unverifiable
+mk up.a2ml '[claims]
+[[claim]]
+id = "C1"
+claim_class = "file-changed"
+target = "../etc/passwd"
+expect = "created"
+verifier = "git-diff"'
+[[ "$(reason_of "$TMP/up.a2ml" C1)" == unverifiable*unsafe-path ]] && ok "path traversal -> unverifiable" || bad "unsafe path not caught"
+# unresolvable base -> unverifiable (not confident-wrong created)
+mk nb.a2ml '[claims]
+[[claim]]
+id = "C1"
+claim_class = "file-changed"
+target = "README.adoc"
+expect = "created"
+verifier = "git-diff"'
+r="$(cd "$ROOT" && DYADT_BASE=definitely-not-a-ref DYADT_ALLOW_UNVERIFIABLE=1 bash "$V" "$TMP/nb.a2ml" 2>/dev/null | grep -oE 'unverifiable *\[[^]]*\] *[a-z-]+' | head -1)"
+[[ "$r" == unverifiable*no-base-ref ]] && ok "unresolvable base -> unverifiable" || bad "unresolvable base not caught (got: $r)"
+# stderr marker must NOT confirm a stdout-contains claim
+mk se.a2ml '[claims]
+[[claim]]
+id = "C1"
+claim_class = "command-ran"
+target = "echo marker >&2; true"
+expect = "stdout-contains:marker"
+verifier = "command-transcript"'
+[[ "$(reason_of "$TMP/se.a2ml" C1)" == REFUTED* ]] && ok "stderr does not satisfy stdout-contains" || bad "stderr false-confirmed stdout claim"
+# licence claim phrased only in the statement is still manual-only
+mk lic.a2ml '[claims]
+[[claim]]
+id = "C1"
+claim_class = "command-ran"
+statement = "added the SPDX licence header"
+target = "true"
+expect = "exit==0"
+verifier = "command-transcript"'
+[[ "$(reason_of "$TMP/lic.a2ml" C1)" == unverifiable*manual-only ]] && ok "licence-in-statement -> manual-only" || bad "licence-in-statement auto-confirmed"
+
 echo "== conformance suite =="
 bash "$ROOT/did-you-actually-do-that/spec/conformance/run-conformance.sh" >/dev/null 2>&1 && ok "conformance vectors pass" || bad "conformance vectors failed"