Skip to content

Commit 47cde2c

Browse files
committed
feat(wave5): per-language testing standard v2.0.0 + guide template + AffineScript
The estate's only per-language testing depth was a single Julia guide from 2024 (no MUST/SHOULD, Rust+Julia only) plus a byte-identical duplicate snapshot. The user flagged this: replicate the pattern properly instead of one stale generic. - standards/language-testing-standards.md -> v2.0.0: RFC-2119 conformance requirements R1–R9 (unit/format/lint/coverage/property/bench/audit/contract/ proof) mapped to the CRG test taxonomy, an anti-theatre rule (no continue-on-error on a MUST check; coverage reported-not-asserted), and the required per-language guide set. Rust/SPARK + Julia mapped to R1–R9. - standards/templates/language-testing-guide-TEMPLATE.md: the skeleton every per-language guide follows — a requirement-mapping table (tool or visible `none`), tools, CI pipeline, best practices, and a mandatory honest "Known gaps" section. - standards/affinescript-testing-guide.md: the estate's primary language, which had ZERO testing standard. Authored honestly — most SHOULD rows are tracked gaps (no formatter/coverage/fuzz/bench yet) and R3 notes that affinescript-verify.yml is currently advisory. Canonical SSOT migrates to hyperpolymath/affinescript prospectively (Wave-6 charter). - scripts/check-language-guide.sh + Justfile language-guides-check, wired into `just validate` as a hard gate: a guide missing a required section (e.g. "Known gaps"), the SPDX header, or the R1..R9 mapping fails loudly. - Deleted the byte-identical language-testing-standards-v1.0.0-2024-04-14.md. - scripts/tests/wave5-language-guides-test.sh (7/7): lint accepts a conformant guide and rejects incomplete ones; the standard is v2.0.0/RFC-2119. Zig / Elixir+Gleam / Idris2+Agda guides are Wave-6 charters. Licence rows manual-only (flag-only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114ps6mY5jAH4Sz
1 parent 832b157 commit 47cde2c

7 files changed

Lines changed: 384 additions & 315 deletions

Justfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,12 +85,18 @@ dyadt-conformance:
8585
dyadt-test:
8686
@bash scripts/tests/wave4-dyadt-test.sh
8787

88+
# Structural lint for per-language testing guides (required sections + R1..R9)
89+
language-guides-check:
90+
@bash scripts/check-language-guide.sh
91+
8892
# Aggregate compliance gate: registry drift is the HARD gate (registry-check,
8993
# a hard dep). The RSR self-audit is INFORMATIONAL — a monorepo is not expected
9094
# to score Gold — but a *broken* audit (exit 4 / unexpected) must fail loudly
9195
# rather than pass silently under a blanket `|| true` (Wave-0 false-green fix).
9296
validate: registry-check
9397
@echo "=== validate: registry drift (HARD GATE) — passed as a dependency above ==="
98+
@echo "=== validate: per-language testing guides (structural, HARD GATE) ==="
99+
@bash scripts/check-language-guide.sh
94100
@echo "=== validate: RSR self-audit (INFORMATIONAL grade; errors fail loudly) ==="
95101
@bash scripts/rsr-selfaudit.sh .
96102
@echo "=== validate: done ==="

scripts/check-language-guide.sh

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
#!/usr/bin/env bash
2+
# SPDX-License-Identifier: MPL-2.0
3+
#
4+
# check-language-guide.sh — structural lint for per-language testing guides.
5+
#
6+
# Every guide built from templates/language-testing-guide-TEMPLATE.md MUST carry
7+
# the required sections and the R1–R9 requirement-mapping table. A guide that
8+
# silently omits a section (e.g. "Known gaps") is a false-completeness hole —
9+
# this fails loudly instead.
10+
#
11+
# Usage: check-language-guide.sh [guide.md ...]
12+
# With no args, checks every standards/*-testing-guide.md.
13+
# Exit: 0 all valid · 1 a guide is missing a required section
14+
15+
set -uo pipefail
16+
17+
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
18+
19+
REQUIRED_SECTIONS=(
20+
"## Requirement mapping"
21+
"## Tools"
22+
"## Recommended CI pipeline"
23+
"## Best practices"
24+
"## Known gaps"
25+
"## Resources"
26+
)
27+
28+
check_one() { # file
29+
local f="$1" rc=0 sec
30+
if [ ! -f "$f" ]; then echo "$f: not found"; return 1; fi
31+
for sec in "${REQUIRED_SECTIONS[@]}"; do
32+
grep -Fqx "$sec" "$f" || { echo "$(basename "$f"): missing section '$sec'"; rc=1; }
33+
done
34+
# The requirement mapping MUST reference the R1..R9 rows (at least R1 and R9).
35+
grep -Eq '\bR1\b' "$f" && grep -Eq '\bR9\b' "$f" || { echo "$(basename "$f"): requirement mapping does not reference R1..R9"; rc=1; }
36+
# A SPDX header is required.
37+
head -3 "$f" | grep -q 'SPDX-License-Identifier' || { echo "$(basename "$f"): missing SPDX header"; rc=1; }
38+
[ "$rc" -eq 0 ] && echo "$(basename "$f")"
39+
return $rc
40+
}
41+
42+
if [ "$#" -gt 0 ]; then
43+
files=("$@")
44+
else
45+
mapfile -t files < <(ls "$ROOT"/standards/*-testing-guide.md 2>/dev/null)
46+
fi
47+
48+
if [ "${#files[@]}" -eq 0 ]; then
49+
echo "no language testing guides found (standards/*-testing-guide.md)"; exit 0
50+
fi
51+
52+
rc=0
53+
echo "Language testing guides:"
54+
for f in "${files[@]}"; do check_one "$f" || rc=1; done
55+
exit $rc
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/usr/bin/env bash
2+
# SPDX-License-Identifier: MPL-2.0
3+
set -uo pipefail
4+
#
5+
# Wave-5 regression: the per-language testing guide lint must accept a
6+
# template-conformant guide and reject one missing a required section.
7+
8+
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
9+
CHK="$ROOT/scripts/check-language-guide.sh"
10+
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
11+
12+
pass=0 fail=0
13+
ok() { echo "$1"; pass=$((pass + 1)); }
14+
bad() { echo "$1"; fail=$((fail + 1)); }
15+
16+
echo "== real guides pass =="
17+
bash "$CHK" >/dev/null 2>&1 && ok "estate guides pass structural lint" || bad "estate guides failed lint"
18+
bash "$CHK" "$ROOT/standards/affinescript-testing-guide.md" >/dev/null 2>&1 && ok "affinescript guide valid" || bad "affinescript guide invalid"
19+
20+
echo "== rejects incomplete guides =="
21+
# missing a required section
22+
g="$TMP/foo-testing-guide.md"
23+
printf '<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->\n# Foo\n## Requirement mapping\nR1 ... R9 ...\n## Tools\n## Recommended CI pipeline\n## Best practices\n## Resources\n' > "$g"
24+
bash "$CHK" "$g" >/dev/null 2>&1 && bad "missing 'Known gaps' not caught" || ok "missing section rejected"
25+
# missing SPDX header
26+
g2="$TMP/bar-testing-guide.md"
27+
printf '# Bar\n## Requirement mapping\nR1 R9\n## Tools\n## Recommended CI pipeline\n## Best practices\n## Known gaps\n## Resources\n' > "$g2"
28+
bash "$CHK" "$g2" >/dev/null 2>&1 && bad "missing SPDX not caught" || ok "missing SPDX rejected"
29+
# missing R1..R9 reference
30+
g3="$TMP/baz-testing-guide.md"
31+
printf '<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->\n# Baz\n## Requirement mapping\nno numbers here\n## Tools\n## Recommended CI pipeline\n## Best practices\n## Known gaps\n## Resources\n' > "$g3"
32+
bash "$CHK" "$g3" >/dev/null 2>&1 && bad "missing R1..R9 not caught" || ok "missing R1..R9 rejected"
33+
34+
echo "== the stale duplicate snapshot is gone =="
35+
[ ! -f "$ROOT/standards/language-testing-standards-v1.0.0-2024-04-14.md" ] && ok "duplicate snapshot removed" || bad "duplicate snapshot still present"
36+
echo "== the standard is v2.0.0 with RFC-2119 =="
37+
grep -q 'Version:\*\* 2.0.0' "$ROOT/standards/language-testing-standards.md" && grep -qi 'RFC-2119' "$ROOT/standards/language-testing-standards.md" && ok "standard refreshed to v2.0.0 RFC-2119" || bad "standard not refreshed"
38+
39+
echo
40+
echo "Wave-5 language-guides regression: $pass passed, $fail failed"
41+
[ "$fail" -eq 0 ]
Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
2+
# AffineScript Testing Tools Guide
3+
4+
**Version:** 1.0.0
5+
**Date:** 2026-07-03
6+
**Status:** Active (baseline — honest by construction)
7+
**Parent standard:** `language-testing-standards.md` (R1–R9)
8+
9+
AffineScript is the estate's primary application language (RS/TS/JS →
10+
AffineScript → typed-wasm; affine/linear types, OCaml-based compiler). This
11+
guide is the estate's current best statement of its testing story. AffineScript's
12+
tooling is young, so several rows below are honest **gaps**, not omissions — a
13+
gap here is a tracked piece of work, and this guide names it rather than
14+
pretending coverage exists.
15+
16+
**Canonical SSOT (prospective):** the authoritative home for this guide will be
17+
`hyperpolymath/affinescript` (`spec/` or `docs/testing.adoc`). Until that lands,
18+
this repo carries it; the migration is a Wave-6 charter. Do not let the two
19+
diverge — when the affinescript-repo version ships, this becomes a pointer.
20+
21+
## Requirement mapping
22+
23+
| # | Requirement | Level | Tool | CI invocation | Status |
24+
|---|---|---|---|---|---|
25+
| R1 | Unit test runner | MUST | `affinescript-deno-test` bootstrap runner | `deno task test` (in the AS repo) | partial — bootstrap shim, self-hosting pending |
26+
| R2 | Formatter (check mode) | MUST | `affinescript fmt` (compiler subcommand) | `affinescript fmt --check` | gap — formatter not yet shipped |
27+
| R3 | Linter / static analysis | MUST | compiler diagnostics + `affinescript-verify.yml` | `affinescript compile --check` | partial — the type system IS the primary static check; a dedicated linter is a gap |
28+
| R4 | Coverage | SHOULD | `none` || gap |
29+
| R5 | Property-based / fuzz | SHOULD | `none` || gap (parser/lowering are the priority targets) |
30+
| R6 | Benchmark | SHOULD | wasm bench harness || gap |
31+
| R7 | Security / dependency audit | MUST | Deno (`deno.json` import audit) | `deno task audit` | partial — Deno-managed deps; no AS-native audit |
32+
| R8 | Contract / pre-post | MAY | affine/linear types (compile-time) | compiler | partial — linearity is a compile-time contract |
33+
| R9 | Proof check | MUST\* | Idris2 ABI proofs (for proven-backed modules) | ECHIDNA proof gate | partial — applies to modules using the `proven` library |
34+
35+
`MUST*` = R7 applies (Deno ecosystem); R9 applies only to AS modules that call
36+
proven/Idris2-verified code.
37+
38+
## Tools
39+
40+
### AffineScript compiler (`affinescript`) — R2, R3, R8
41+
- **Purpose:** the type checker is the primary correctness gate. Affine/linear
42+
types reject use-after-move and aliasing at compile time — that is R8
43+
(contract) discharged by construction, and much of R3 (static analysis).
44+
- **Usage:** `affinescript compile <file>.affine` (type-checks + lowers to
45+
typed-wasm); `affinescript compile --check` for check-only.
46+
- **CI:** `.github/workflows/affinescript-verify.yml` clones + builds the
47+
compiler and runs verification. **Note:** that job is currently *advisory*
48+
(`continue-on-error`) while the compiler build stabilises — it does not yet
49+
gate. Promotion to blocking is the unblock condition for R3.
50+
51+
### affinescript-deno-test — R1
52+
- **Purpose:** the bootstrap test runner used until AffineScript self-hosts its
53+
test framework. TS/JS shim (documented carve-out).
54+
- **Usage:** `deno task test` in the AS repo.
55+
- **CI:** runs in the AffineScript repo's CI.
56+
57+
## Recommended CI pipeline
58+
59+
Until the AS-native toolchain matures, the recommended pipeline is:
60+
61+
1. **Type-check (R3/R8, MUST):** build the compiler, `affinescript compile
62+
--check` over all `.affine` sources — SHA-pinned, and **blocking once the
63+
compiler build is reliably green** (today advisory; see `affinescript-verify.yml`).
64+
2. **Unit tests (R1, MUST):** `deno task test` via the bootstrap runner.
65+
3. **Dep audit (R7, MUST):** Deno import audit.
66+
4. SHOULD rows (coverage, property, bench) are tracked gaps — see below.
67+
68+
No `continue-on-error` on a MUST check once its tool is stable; the current
69+
advisory status of `affinescript-verify.yml` is itself a tracked gap, not a
70+
silent pass.
71+
72+
## Best practices
73+
74+
1. Design modules to admit affine/linear typing from the start — the type system
75+
is the cheapest test you have.
76+
2. Prefer compile-time linearity contracts (R8) over runtime assertions where the
77+
type system can express the invariant.
78+
3. For correctness-critical paths, route through `proven`/Idris2 (R9) rather than
79+
hand-rolled checks.
80+
4. Keep `.affine` sources free of TS/JS shims except the documented bootstrap
81+
carve-outs.
82+
83+
## Known gaps
84+
85+
Honest inventory (every gap is real work, not an omission):
86+
87+
- **R2 formatter** — no `affinescript fmt` yet. Charter.
88+
- **R3 dedicated linter** — beyond type diagnostics; and `affinescript-verify.yml`
89+
is advisory (`continue-on-error`), so R3 does not yet *gate*. Charter: flip to
90+
blocking once the compiler build is reliably green.
91+
- **R4 coverage** — no wasm coverage tool. Charter.
92+
- **R5 property/fuzz** — none; parser and canonical-lowering are the priority
93+
targets. Charter.
94+
- **R6 benchmark** — no wasm bench harness. Charter.
95+
- **R1 self-hosting** — the test runner is a TS/JS bootstrap shim, not AS-native.
96+
Unblocks when AffineScript self-hosts the runner.
97+
98+
These gaps are why AffineScript's Toolchain Readiness Grade cannot yet exceed the
99+
lower bands — which is the honest position, and the reason this guide exists.
100+
101+
## Resources
102+
103+
- `language-testing-standards.md` — the parent R1–R9 standard.
104+
- `.github/workflows/affinescript-verify.yml` — the current (advisory) CI check.
105+
- `templates/language-testing-guide-TEMPLATE.md` — the skeleton this follows.
106+
- SSOT (prospective): `hyperpolymath/affinescript`.
107+
108+
**Maintainers:** @hyperpolymath
109+
**Last Updated:** 2026-07-03

standards/language-testing-standards-v1.0.0-2024-04-14.md

Lines changed: 0 additions & 165 deletions
This file was deleted.

0 commit comments

Comments
 (0)