You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(wave7): ground the dashboard — every pass now backed by an executable check
The compliance dashboard's `status = "pass"` was self-asserted prose. Wave 7
makes it machine-verified: DYADT applied to the scorecards themselves.
- scorecard.schema.json: optional `check` field — a read-only shell command,
run from the repo root, that exits 0 iff the requirement currently holds.
Distinct from `system` (human-readable): `check` is what CI RUNS.
- build-scorecards.sh --verify: RUNS every pass-row check. A claimed pass whose
check fails is a HARD error; a fail-row whose check now passes is flagged as
a stale-fail re-grade candidate (advisory); passes without checks are counted
as self-asserted (visible debt). Dashboard gains a Grounded column + rollup.
extract_checks handles TOML-escaped quotes (strip trailing quote + unescape —
first-quote truncation silently rewrote commands).
- Grounding sweep (per-spec fan-out, every check tested before adoption):
87/87 pass rows now grounded — 0 broken, 0 self-asserted. 82 checks added,
4 honest enforcement-gap flips (fact true on disk, previously unchecked),
4 re-grades (licence rows to manual-only per Manual-Only policy).
Implementation gaps (~250 rows) deliberately stay honest `fail` — a check
cannot invent missing engineering.
- The grounding pass caught a real defect: avow-protocol's OSSF-scorecard
evidence cited a NESTED workflow (dormant — GitHub only executes root
.github/workflows); evidence corrected to the root workflows that actually
run, and grounded against them.
- registry-verify.yml: the dashboard step now runs --check --strict --verify,
with remediation hints. Justfile: scorecards-verify recipe.
- wave3-scorecards-test.sh: +4 assertions (broken pass fails, holding checks
stay green, grounded count reported, stale-fail advisory); updated fixtures
for the fully-grounded world; fixed a grep -q/SIGPIPE race under pipefail.
MUST-passing moved 48 -> 47 by honest regrade (licence row to manual-only),
not by loss. Licence/SPDX rows remain manual-only throughout; no SPDX edits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114ps6mY5jAH4SzbGxeuYjc
Copy file name to clipboardExpand all lines: .machine_readable/scorecards/0-ai-gatekeeper-protocol.scorecard.a2ml
+3Lines changed: 3 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -15,6 +15,7 @@ text = "The repository MUST contain exactly one AI manifest file, named 0-AI-MAN
15
15
system = "none (no automated validator script checks this in CI; verified manually via directory listing)"
16
16
status = "pass"
17
17
evidence = "find confirms exactly one manifest at /home/user/standards/0-ai-gatekeeper-protocol/0-AI-MANIFEST.a2ml; no AI.a2ml, !AI.a2ml, or duplicate found at root."
effects = "If violated, downstream MCP/FUSE parsers (mcp-repo-guardian, repo-guardian-fs) that assume a single canonical manifest path would pick an arbitrary or wrong file, breaking attestation for every consuming repo/agent."
19
20
20
21
[[must]]
@@ -23,6 +24,7 @@ text = "The manifest MUST contain all required sections defined by AI-MANIFEST-S
23
24
system = "none (no parser/linter script in this repo enforces the spec against 0-AI-MANIFEST.a2ml; mcp-repo-guardian's manifest_test.js only tests inline reimplementations of parsing logic, not a repo-facing validator CLI)"
24
25
status = "pass"
25
26
evidence = "Manual read of /home/user/standards/0-ai-gatekeeper-protocol/0-AI-MANIFEST.a2ml confirms presence of '# ⚠️ STOP', '## WHAT IS THIS?', '## CANONICAL LOCATIONS', '## CORE INVARIANTS', '## REPOSITORY STRUCTURE', and '## ATTESTATION PROOF' headings."
effects = "Missing sections would leave AI agents without required guardrail information, defeating the protocol's stated purpose across all adopting repos."
27
29
28
30
[[must]]
@@ -38,6 +40,7 @@ text = "Core manifest-parsing/session/guard logic (as implemented for FFI/offlin
38
40
system = "cargo test --manifest-path repo-guardian-fs/tests-offline/Cargo.toml (Rust unit tests for manifest parsing, session management, and path-guard invariant enforcement)"
39
41
status = "pass"
40
42
evidence = "Ran `cargo test` in /home/user/standards/0-ai-gatekeeper-protocol/repo-guardian-fs/tests-offline: 29 passed; 0 failed (manifest hashing, canonical-location extraction, invariant detection, session ack/expiry, path-traversal/SCM-duplication guard tests, and a dogfood test parsing the repo's real 0-AI-MANIFEST.a2ml)."
43
+
check = "cargo test --manifest-path 0-ai-gatekeeper-protocol/repo-guardian-fs/tests-offline/Cargo.toml"
41
44
effects = "If these regress, any FFI/native consumer (e.g. Zig bindings, future editor plugins) linking against this logic would silently mis-enforce or fail to enforce invariants."
Copy file name to clipboardExpand all lines: .machine_readable/scorecards/a2ml-templates.scorecard.a2ml
+8-6Lines changed: 8 additions & 6 deletions
Original file line number
Diff line number
Diff line change
@@ -12,9 +12,9 @@ assessor = "estate-audit"
12
12
[[must]]
13
13
id = "M1"
14
14
text = "Each *.a2ml.template file MUST carry an SPDX-License-Identifier header."
15
-
system = "none"
16
-
status = "pass"
17
-
evidence = "All 7 files in /home/user/standards/a2ml-templates/ (META.a2ml.template, AGENTIC.a2ml.template, ECOSYSTEM.a2ml.template, NEUROSYM.a2ml.template, PLAYBOOK.a2ml.template, STATE.a2ml.template, STATE.a2ml.v2.template) and both scripts carry an SPDX-License-Identifier as line 1/2 (manually verified by reading each file); however no automated check enforces this for this directory - hooks/validate-spdx.sh only scans .github/workflows/*.yml, not a2ml-templates/."
15
+
system = "Manual license audit (Manual-Only policy: SPDX/licence rows never carry an automated check)."
16
+
status = "manual-only"
17
+
evidence = "Manually verified (and spot-checked via grep) that all 7 template files, the spec.adoc, and both migration scripts carry an SPDX-License-Identifier on line 1 or 2; per estate Manual-Only policy this remains a manual/licence-audit item, not an automated check."
18
18
effects = "Downstream repos copying these templates inherit correct licensing metadata by convention, but a future edit could silently drop the header with nothing to catch it."
19
19
20
20
[[must]]
@@ -41,9 +41,10 @@ effects = "Consuming repos (or the migration scripts themselves) can emit malfor
41
41
[[must]]
42
42
id = "M5"
43
43
text = "The spec MUST be tracked in the estate compliance dashboard with a scorecard once templates stabilize."
44
-
system = "COMPLIANCE-DASHBOARD.md rollup table"
45
-
status = "fail"
46
-
evidence = "COMPLIANCE-DASHBOARD.md line 50 lists `a2ml-templates | no scorecard | - | - | - | - | -` and line 58 includes a2ml-templates in the 27 specs still needing a scorecard (1/28 specs scored)."
44
+
system = "COMPLIANCE-DASHBOARD.md rollup table (line 52) plus .machine_readable/scorecards/a2ml-templates.scorecard.a2ml"
45
+
status = "pass"
46
+
evidence = "COMPLIANCE-DASHBOARD.md line 52 lists `a2ml-templates` with real must/should/could scores (1/5, 1/3, 0/2) rather than the 'no scorecard' placeholder, and this file (.machine_readable/scorecards/a2ml-templates.scorecard.a2ml) is the tracked scorecard; rollup states 'Specs with a scorecard: 30 / 30'."
effects = "The estate-wide rollup (1/28 specs scored, 50% system coverage) cannot include this spec's real compliance state, so any regression here is invisible at the estate level until this scorecard is produced and merged."
48
49
49
50
[[should]]
@@ -67,6 +68,7 @@ text = "The migration scripts SHOULD emit non-zero exit codes and clear diagnost
67
68
system = "none"
68
69
status = "pass"
69
70
evidence = "state-scm-to-v2.jl documents and implements distinct exit codes (0 ok, 2 parse/usage error, 3 not a (state ...) form) per its header comment and main()/convert() logic; state-migrate-v1-to-v2.sh uses `set -eu` and exits 1 with a message when the v1 file is missing (line 16)."
effects = "Callers (CI or a future k9-init tool) can branch on exit code to decide whether to retry, prompt a human, or abort; without this, migration failures could pass silently and produce corrupt STATE.a2ml files downstream."
Copy file name to clipboardExpand all lines: .machine_readable/scorecards/a2ml.scorecard.a2ml
+8Lines changed: 8 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -15,6 +15,7 @@ text = "The A2ML surface grammar MUST be normatively specified so that conformin
15
15
system = "/home/user/standards/a2ml/SPEC.adoc (29KB normative spec, v1.1.0) plus /home/user/standards/a2ml/docs/GRAMMAR.adoc"
16
16
status = "pass"
17
17
evidence = "SPEC.adoc exists at repo root (29183 bytes) as the current normative spec per README.adoc, with docs/GRAMMAR.adoc as grammar appendix; VERSIONS.adoc documents the spec lineage across v0.1.0-v1.1.0."
effects = "Downstream language bindings (rust/elixir/gleam/haskell/deno) and the Idris2 reference parser all depend on this text as their shared contract; ambiguity here would fork implementations."
19
20
20
21
[[must]]
@@ -23,6 +24,7 @@ text = "A parser conformance test suite (test vectors with expected outputs) MUS
23
24
system = "/home/user/standards/a2ml/tests/vectors/*.a2ml + *.expected + *.html.expected (8 vector pairs); described in docs/CONFORMANCE.adoc"
24
25
status = "pass"
25
26
evidence = "tests/vectors/ contains 8 input/.expected pairs (module0-basic, refs-basic, refs-missing, inline-formatting, inline-edges, opaque-roundtrip, directive-attrs, table-basic) matching the vector list enumerated in docs/CONFORMANCE.adoc."
effects = "Without these vectors, the 6 language bindings and reference Idris2/ReScript parsers would have no shared ground truth for interop, risking silent divergence."
27
29
28
30
[[must]]
@@ -31,6 +33,7 @@ text = "CI MUST validate every *.a2ml file in the repository on push/PR to catch
31
33
system = ".github/workflows/a2ml-validation.yml job 'validate-a2ml' (builds Idris2 CLI, runs `a2ml validate` over every found *.a2ml file)"
32
34
status = "pass"
33
35
evidence = ".github/workflows/a2ml-validation.yml triggers on push/PR touching **.a2ml, cli/**, src/A2ML/**, builds the CLI via cli/build.sh and runs `find . -name '*.a2ml' | ... a2ml validate` failing the job (exit 1) on any validation failure."
effects = "Prevents malformed .a2ml manifests (README.a2ml, SPEC.adoc-derived, docs/IANA-MEDIA-TYPE.a2ml) from being merged, which would break consumers of dogfood-gate across the 105+ dependent repos noted in READINESS.md."
35
38
36
39
[[must]]
@@ -39,6 +42,7 @@ text = "The reference Idris2 parser/typed-core MUST type-check in CI on every pu
39
42
system = ".github/workflows/idris2-tests.yml (installs Idris2 0.7.0, runs `idris2 --check` on A2ML/Parser.idr and A2ML/TypedCore.idr)"
40
43
status = "pass"
41
44
evidence = "idris2-tests.yml pinned to Idris2 v0.7.0 release tarball, runs `idris2 --check --source-dir . A2ML/Parser.idr` and `A2ML/TypedCore.idr` on every push/PR to main/master."
effects = "A broken reference implementation would invalidate the typed-core guarantees (attested conformance level) that all downstream bindings claim to implement."
43
47
44
48
[[must]]
@@ -54,6 +58,7 @@ text = "Each of the six language implementations (Rust, Elixir, Gleam, Haskell,
54
58
system = "bindings/rust/tests/crg_c_tests.rs (36 test fns) + src/parser.rs (8) + src/renderer.rs (3); bindings/elixir/test/a2_ml_test.exs; bindings/gleam/test/a2ml_gleam_test.gleam; .github/workflows/rescript-tests.yml; idris2-tests.yml"
55
59
status = "pass"
56
60
evidence = "bindings/rust/tests/crg_c_tests.rs contains 36 #[test] functions plus 11 more in src/parser.rs and src/renderer.rs; bindings/elixir/test/ and bindings/gleam/test/ contain dedicated test files; rescript-tests.yml and idris2-tests.yml run in CI for the ReScript/Idris2 reference stack. No Haskell/Deno test files were found under bindings/haskell or bindings/deno, so this is a partial pass across the 6 targets."
61
+
check = "test $(grep -c '#\\[test\\]' a2ml/bindings/rust/tests/crg_c_tests.rs) -ge 30 && test -f a2ml/bindings/elixir/test/a2_ml_test.exs && test -f a2ml/bindings/gleam/test/a2ml_gleam_test.gleam && test -f a2ml/.github/workflows/rescript-tests.yml && test -f a2ml/.github/workflows/idris2-tests.yml"
57
62
effects = "Untested bindings (Haskell, Deno) risk silent behavioral drift from the reference parser, which could break the Haskell/Deno-consuming repos (a2ml-haskell, scaffoldia, idaptik, nafa-app per READINESS.md) without CI catching it."
58
63
59
64
[[should]]
@@ -70,6 +75,7 @@ text = "Fuzz testing of the parser SHOULD run on a recurring schedule to catch c
70
75
system = ".github/workflows/fuzzing.yml (weekly cron '0 0 * * 0' + push/PR, builds ReScript parser and generates fuzz inputs)"
71
76
status = "pass"
72
77
evidence = "fuzzing.yml is scheduled weekly (cron: '0 0 * * 0') in addition to push/PR triggers, and builds the ReScript prototype parser to generate/execute fuzz inputs."
effects = "Without scheduled fuzzing, parser crashes on malformed/adversarial input (a security and robustness concern for any tool ingesting untrusted .a2ml files) would only surface reactively rather than proactively."
74
80
75
81
[[should]]
@@ -78,6 +84,7 @@ text = "Secrets SHOULD be scanned on every push/PR to prevent credential leaks."
78
84
system = ".github/workflows/secret-scanner.yml (TruffleHog + Gitleaks jobs)"
79
85
status = "pass"
80
86
evidence = "secret-scanner.yml runs both trufflesecurity/trufflehog (--only-verified --fail) and a gitleaks job on pull_request and push to main."
effects = "Prevents accidental credential leakage into repo history, which would otherwise require costly rotation/remediation across the hyperpolymath estate."
82
89
83
90
[[should]]
@@ -86,6 +93,7 @@ text = "The internal .machine_readable/6scm legacy mirror SHOULD be kept honest
86
93
system = "scripts/check-6scm.sh (byte-diff of .machine_readable/*.scm vs .machine_readable/6scm/*.scm, with explicit DRIFT/OBSOLETE states)"
87
94
status = "pass"
88
95
evidence = "scripts/check-6scm.sh explicitly implements 3 states (in-sync, obsolete no-op, DRIFT) rather than a blanket success, per its own header comment referencing 'kill the false green' remediation; this repo currently has no .scm sources so it correctly exits 0 as 'OBSOLETE (no-op)' rather than falsely claiming sync."
96
+
check = "bash a2ml/scripts/check-6scm.sh"
89
97
effects = "Prevents a false-green validator from masking real drift between legacy .scm descriptor sources and their mirror, which would otherwise mislead any tooling still reading the legacy 6scm path."
Copy file name to clipboardExpand all lines: .machine_readable/scorecards/accessibility.scorecard.a2ml
+2Lines changed: 2 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -15,6 +15,7 @@ text = "The spec MUST provide a machine-readable Adjustfile.a2ml contract at .ma
15
15
system = "probe: test -f .machine_readable/contractiles/adjust/Adjustfile.a2ml (as literally defined in STANDARD.a2ml Level A section)"
16
16
status = "pass"
17
17
evidence = "/home/user/standards/.machine_readable/contractiles/adjust/Adjustfile.a2ml exists on disk (along with sibling adjust.manifest.a2ml, adjust.ncl, adjust.k9.ncl in the same directory)."
effects = "Downstream projects copying this standard rely on this file as the canonical contractile template; if missing, their own Level-A probe fails."
19
20
20
21
[[must]]
@@ -30,6 +31,7 @@ text = "Projects MUST provide accessibility documentation (docs/accessibility/RE
30
31
system = "probe: test -f docs/accessibility/README.adoc"
31
32
status = "pass"
32
33
evidence = "/home/user/standards/docs/accessibility/README.adoc exists and documents keyboard, screen-reader, and roadmap sections (though it is written as a template for a hypothetical 'Burble' project, not the standards repo itself)"
34
+
check = "test -f docs/accessibility/README.adoc"
33
35
effects = "If absent, any project claiming HAS Level A compliance has no discoverable documentation trail, undermining auditability."
effects = "If absent, no downstream language repo has a normative baseline to cite in spec/ARG-PROFILE.adoc; per-language profiles would have nothing to tighten against."
19
20
20
21
[[must]]
@@ -72,6 +73,7 @@ text = "A machine-readable counterpart of the normative spec (.a2ml) SHOULD exis
72
73
system = "none - no script cross-validates ADOPTION-READINESS-GRADES.a2ml against ADOPTION-READINESS-GRADES.adoc for consistency; consistency is asserted only in SELF-ASSESSMENT.adoc's manually-ticked checklist"
73
74
status = "pass"
74
75
evidence = "/home/user/standards/adoption-readiness-grades/ADOPTION-READINESS-GRADES.a2ml exists (339 lines, SPDX-License-Identifier: MPL-2.0), referenced consistently in README.adoc and SELF-ASSESSMENT.adoc as the machine-readable counterpart."
effects = "If it drifted from the prose spec unnoticed, any tool consuming the a2ml form would diverge from the documented normative rules; currently no automated guard against that drift."
Copy file name to clipboardExpand all lines: .machine_readable/scorecards/agentic-a2ml.scorecard.a2ml
+2-1Lines changed: 2 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -15,6 +15,7 @@ text = "The repository MUST provide a normative specification document defining
15
15
system = "spec/AGENTIC-FORMAT-SPEC.adoc (383 lines, present and referenced from README.adoc)"
16
16
status = "pass"
17
17
evidence = "/home/user/standards/agentic-a2ml/spec/AGENTIC-FORMAT-SPEC.adoc exists (383 lines) and is linked as 'the complete normative specification' from README.adoc."
effects = "Consumers (playbook-a2ml, meta-a2ml, downstream agent tooling) rely on this document as the authoritative contract for gating-policy fields; without it, dependent repos cannot implement conformant readers."
19
20
20
21
[[must]]
@@ -84,7 +85,7 @@ effects = "No downstream impact today; this is a stretch goal beyond the project
84
85
id = "C2"
85
86
text = "Contractile templates (Mustfile/Dustfile) COULD be filled in with agentic-a2ml-specific invariants and rollback handlers rather than left as generic copy-in templates."
86
87
system = "none — contractiles/must/Mustfile and contractiles/dust/Dustfile reference generic placeholder paths (config/service.yaml, policy/policy.ncl, logs/decisions.json) that do not exist in this repo"
87
-
status = "fail"
88
+
status = "aspirational"
88
89
effects = "Minor: these are explicitly documented in contractiles/README.adoc as a template set meant to be copied and customised per-repo, so their generic state is expected, but as shipped they provide zero enforceable invariant for agentic-a2ml itself."
0 commit comments