Skip to content

Commit 91f7dc7

Browse files
docs: design-diary entry 0007 + flagship rubric (measuring-stick) (#53)
## What Docs-only. Two additions: - **`docs/design-diary/PEPYS-DESIGN-DIARY.adoc`** — Entry 0007. Records P5 (audited `assail-classifications`, G18), the A2ML dialect collision that failed #51's dogfood gate + its at-source TOML-A2ML fix (#52), and the verified-by-execution Julia-library template + rubric work. - **`docs/flagship-rubric.adoc`** — the measuring-stick every estate Julia library is scored against, with **Axiom.jl as the 100% reference (30/30)**. ## The flagship rubric Positions Axiom.jl as the estate's Julia measuring-stick: - **5 grades** — D (Alpha) / C (Beta) / B (RC) / A (Production) / ★ (Flagship). - **10 scored dimensions** — correctness+tests, Aqua, JET, compat+registry, docs, security+soundness, verification honesty, estate compliance, maintenance, wire-first. Each carries `[auto]` (enforced by `just quality`) or `[review]` (human/agent-judged, e.g. licence classification — never automated). - **Machine-checkable `just quality` subset** — `Pkg.test()` (Aqua+JET inside it), Documenter doctests, `panic-attack assail` 0 unsuppressed critical. - **Holes-first scoring** — 0–30 vector; a hole in any dimension caps the total (a 24 with a 0 in Security is not a 24-quality library). ## Notes Both files are `.adoc` under `docs/` (not `docs/src/`, so Documenter does not build them; estate-rules only flags `.md`), with correct `CC-BY-SA-4.0` SPDX headers. No code changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- _Generated by [Claude Code](https://claude.ai/code/session_01UPFC9YQ7g9gc3VnRox42Q1)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3949370 commit 91f7dc7

2 files changed

Lines changed: 229 additions & 0 deletions

File tree

docs/design-diary/PEPYS-DESIGN-DIARY.adoc

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,3 +271,57 @@ CI-only). Registry Pkg-add round-trip = CI-only (sandbox libgit2 quirk).
271271
*Method held:* holes-first; ran the tools not the docs; no self-grade (opus
272272
independent verify); no licence edits; birth-SPDX on new files; pushed + PR'd
273273
each repo so nothing lives only on local disk.
274+
275+
== Entry 0007 — 2026-07-01 — P5 audit, an A2ML dialect collision, and a *verified* Julia template
276+
277+
*Directive:* "p5, then 2 then 3." Done in order.
278+
279+
*P5 (G18) — audited residuals.* Authored `audits/assail-classifications.a2ml`
280+
+ `audit-ffi-unsafe.adoc` classifying only the 3 genuine FFI/unsafe seams
281+
(crypto/src/lib.rs UnsafeCode; signing.jl + zig_ffi.jl UnsafeFFI). Ground-truthed
282+
by re-running `panic-attack assail`: exactly 3 flip to suppressed, 0 critical.
283+
PR #51 — owner merged.
284+
285+
*The collision (and the lesson, again).* #51 merged with the dogfood-gate
286+
"Validate A2ML manifests" check RED. Cause: the estate `a2ml-validate-action`
287+
scans *every* `.a2ml` and requires TOML-A2ML identity (`name`/`project`);
288+
panic-attack's classification registry is authored in an *S-expression* dialect
289+
with no such header — two A2ML dialects colliding on one file extension. The
290+
honest fix was NOT to game the lenient check but to make the file *genuinely*
291+
TOML-A2ML: a real `[metadata]` header + the S-expression payload carried in a
292+
TOML literal string (`'''…'''`) under `[payload]`. Verified BOTH sides locally —
293+
Julia's `TOML` stdlib parses it (identity present); `panic-attack assail` still
294+
suppresses exactly 3 (its loader reads raw text for `(classification …)` forms,
295+
ignores the header). PR #52 — green in CI, merged. *Lesson reinforced from
296+
#49→#50: run the newly-added CI gate's logic locally, and when a merged PR left a
297+
red check I caused, fix it at source in a fresh-from-main follow-up.*
298+
299+
*Item 2 — the Julia-library template (staged; VERIFIED by running it).* Drafted
300+
`rsr-julia-library-template-DRAFT/` in scratchpad: PORT-MANIFEST (file-by-file
301+
CLONE/PARAM/NEW/DROP provenance + instantiation runbook), GENERIC-VS-JULIA-SPLIT
302+
(the owner-requested README/EXPLAINME split; generic `rsr-template-repo` stays
303+
un-renamed), README/EXPLAINME, and a full parameterised `template/` payload
304+
lifted from Axiom's *verified* scaffolding (Project.toml `[compat]` closure +
305+
weakdeps/extensions; Aqua+JET+runtests; Documenter make.jl; julia-test.yml with
306+
optional native-shim block; registry REGISTER hook). Then — doctrine, ground
307+
truth by running — I materialised a throwaway `Widget.jl` and *executed* it:
308+
`Pkg.test()` 13/13 green (Aqua+JET+smoke, offline from the depot) and
309+
`docs/make.jl` green (doctests). Execution caught THREE latent bugs pure
310+
reasoning missed: (1) docs env listing the unregistered package → premature
311+
registry resolve (fix: Documenter-only docs env, package dev-pathed in make.jl);
312+
(2) git-remote auto-detection dependency (fix: explicit
313+
`repo = Remotes.GitHub(...)`); (3) module docstring missing from the manual under
314+
strict `checkdocs=:all` (fix: include it in the `@docs` block). Green-from-birth
315+
is now *proven*, not asserted. BLOCKED on push: `rsr-julia-library-template-repo`
316+
is not in session scope — owner must create it + add to env config + start a
317+
fresh session.
318+
319+
*Item 3 — the flagship rubric (P8).* `FLAGSHIP-RUBRIC.adoc`: 5 grades
320+
(D/C/B/A/★-Flagship), 10 machine-or-review-scored dimensions, a `just quality`
321+
`[auto]` subset, and a 0–30 scoring vector with Axiom.jl = 30/30 as the reference
322+
every estate Julia library is measured against (holes-first: a 0 in Security
323+
caps the whole score).
324+
325+
*Method held:* ran tools not docs (and ran them on my OWN deliverable);
326+
fixed the A2ML collision at source; no self-grade; faithful reporting of the
327+
BLOCKED push and the 3 template bugs I had to fix.

docs/flagship-rubric.adoc

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
// SPDX-License-Identifier: CC-BY-SA-4.0
2+
= Flagship Rubric — the measuring-stick for hyperpolymath Julia libraries
3+
Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
4+
:revdate: 2026-07-01
5+
:toc: left
6+
:toclevels: 3
7+
8+
The gradable standard every estate Julia library is measured against.
9+
`Axiom.jl` is the reference instance (the flagship), defined as 100% on this
10+
rubric; every other estate Julia library is scored relative to it. The
11+
machine-checkable subset of this rubric is what `just quality` runs.
12+
13+
[.lead]
14+
Design principle: *a claim counts only if a tool can be run to confirm it.*
15+
Doctrine — ground-truth by running the tool, not trusting a status doc; report
16+
faithfully, no overclaim; holes before features.
17+
18+
== Grades
19+
20+
Adapted from the Component Readiness Grade (CRG) ladder, specialised for a
21+
registered Julia library.
22+
23+
[cols="1,2,4"]
24+
|===
25+
| Grade | Name | Bar
26+
27+
| D | Alpha | Loads and runs: `using Pkg; Pkg.test()` completes without erroring
28+
on the LTS floor. At least one *meaningful* assertion (not `@test true`).
29+
30+
| C | Beta | Correct on representative input; `Aqua.test_all` passes; closed
31+
`[compat]`; README.adoc + a building Documenter site.
32+
33+
| B | RC | Edge cases + multiple input types covered; scoped `JET.test_package`
34+
returns zero package-origin reports; doctests pass; `panic-attack assail` = 0
35+
unsuppressed critical; registry name↔UUID/repo-slug agree.
36+
37+
| A | Production | All of B green in CI across `[LTS, stable]`; property/fuzz
38+
tests where the domain admits them; maintenance axes documented; SPDX + licence
39+
correct; A2ML dogfood gate + estate-rules green.
40+
41+
| ★ | Flagship | All of A, *plus*: verification claims formally honest (no
42+
overclaimed proofs; certificates truthfully labelled; FFI residuals audited via
43+
`assail-classifications.a2ml`); zero-warning build; and it is itself a
44+
*reference* others are measured against. `Axiom.jl` holds ★.
45+
|===
46+
47+
An instance's grade is the highest ladder rung for which *every* row below at or
48+
below that rung passes. One failing B-row caps you at C regardless of A-rows.
49+
50+
== The scored dimensions
51+
52+
Each dimension lists its machine check (what `just quality` runs) and the flagship
53+
bar. `[auto]` = enforced by `just quality`; `[review]` = human/agent-judged
54+
(e.g. licence classification — never automated, per doctrine #6).
55+
56+
=== 1. Correctness & tests `[auto]`
57+
* `julia --project=. -e 'using Pkg; Pkg.instantiate(); Pkg.build(); Pkg.precompile(); Pkg.test()'` green.
58+
* Assertions are behavioural, not tautological. Grep guard: fail if the only
59+
assertions are `@test true` / type-only `@test x isa T` (mirrors panic-attack
60+
MutationGap PA025 "all-type-only assertions").
61+
* Flagship bar: green on LTS *and* stable in CI; nightly allow-fail present.
62+
63+
=== 2. Static quality — Aqua `[auto]`
64+
* `Aqua.test_all(<Pkg>)` passes with NO overrides.
65+
* An override is a finding unless it carries an inline written rationale for a
66+
genuine false positive. Flagship bar: zero overrides (fix source, as Axiom did
67+
for the normalization unbound-typeparam bug + the stale-JSON3 dep).
68+
69+
=== 3. Static quality — JET `[auto]`
70+
* `JET.test_package(<Pkg>; target_modules = (<Pkg>,))` returns zero
71+
package-origin reports.
72+
* Scoping uses JET's native `target_modules` (not a file-path hack), and a
73+
written note explains any borderline candidate as non-package-origin.
74+
* Flagship bar: zero, with the honesty note ("count is 0 today, not an
75+
allowlist").
76+
77+
=== 4. Compat closure & registry `[auto]`
78+
* Every `[deps]`/`[weakdeps]` entry + stdlibs + `julia` bounded in `[compat]`.
79+
* `name`/`uuid`/`repo` agree across `Project.toml`, the registry `Package.toml`,
80+
the `Registry.toml` line, and the GitHub slug (registry `ci/validate.jl`).
81+
* Flagship bar: AutoMerge-eligible; upper-bounded compat everywhere (registry
82+
warns on lower-bound-only).
83+
84+
=== 5. Documentation `[auto]` + `[review]`
85+
* `julia --project=docs docs/make.jl` builds; `doctest = true` passes `[auto]`.
86+
* `README.adoc` (AsciiDoc, estate default) + `EXPLAINME.adoc` present `[auto]`.
87+
* API-doc coverage of exported symbols; no dangling doc references `[review]`.
88+
* Flagship bar: doctests green in CI; docs deploy on main.
89+
90+
=== 6. Security & soundness `[auto]`
91+
* `panic-attack assail .` = 0 *unsuppressed* critical.
92+
* FFI/`unsafe` residuals are audited via `audits/assail-classifications.a2ml`
93+
(TOML-A2ML conformant) + a companion `audits/audit-*.adoc` — suppressions are
94+
non-gameable (registry lives separately from the code under scan).
95+
* No MD5/SHA-1 for security (SHA-256+); HTTPS-only; no hardcoded secrets;
96+
SHA-pinned CI actions.
97+
* Flagship bar: every suppression cites a current audit; secret-scanner +
98+
static-analysis-gate green.
99+
100+
=== 7. Verification honesty `[review]` + partial `[auto]`
101+
* No overclaimed proofs: a `@prove`/proof-import path labelled as formal must be
102+
formal; heuristics labelled as heuristics (Axiom G03/G04).
103+
* Certificates truthfully labelled (no fake "authenticated" without a real MAC/
104+
signature; Axiom G01).
105+
* Proof-drift honest: proof escape hatches visible, not hidden (panic-attack
106+
ProofDrift PA021 left un-suppressed unless audited).
107+
* Flagship bar: verification claims survive an adversarial re-read; no
108+
soundness hole outranked by a feature.
109+
110+
=== 8. Estate compliance `[auto]` + `[review]`
111+
* A2ML substrate present and *valid TOML-A2ML* — dogfood gate `Validate A2ML
112+
manifests` green (every `.a2ml` self-identifies; even a panic-attack
113+
S-expression registry is TOML-A2ML-wrapped, per Axiom PR #52).
114+
* `estate-rules` green: no Python; AsciiDoc-by-default under `docs/` (except
115+
`docs/wiki/` + `docs/src/`); no V-lang (Zig is allowed and encouraged).
116+
* Licence correct for the repo's classification `[review]` — MPL-2.0 sole-owner
117+
default; NEVER an automated licence edit.
118+
* SPDX headers on all source files (authored from birth, never bulk-swept).
119+
120+
=== 9. Maintenance & lifecycle `[review]`
121+
* Corrective / adaptive / perfective / preventive maintenance axes documented
122+
(`.machine_readable/policies/MAINTENANCE-AXES.a2ml`) with a checklist.
123+
* `CHANGELOG.md` current; ADRs under `docs/decisions/` for load-bearing choices.
124+
* Flagship bar: a living gap register; every "fixed" reaches "verified" via an
125+
independent adversarial pass before it counts.
126+
127+
=== 10. Wire-first / no-dead-scaffolding `[review]`
128+
* Every claimed feature is reachable and tested; no orphaned modules that the
129+
README advertises (Axiom G06 huggingface lesson). Unwired is not done.
130+
* Placeholders (`{{...}}`) rendered or removed before release.
131+
* Flagship bar: green from birth on instantiation; docs and code agree.
132+
133+
== `just quality` — the machine-checkable bar
134+
135+
The recipe runs every `[auto]` row and fails on the first breach. Sketch (the
136+
template ships the real recipe wired to the instance):
137+
138+
[source,makefile]
139+
----
140+
# Justfile (excerpt) — the flagship rubric's automated subset.
141+
quality:
142+
# 1,2,3 correctness + Aqua + JET (Aqua/JET run inside Pkg.test)
143+
julia --project=. -e 'using Pkg; Pkg.instantiate(); Pkg.build(); Pkg.precompile(); Pkg.test()'
144+
# 5 docs build + doctests
145+
julia --project=docs -e 'using Pkg; Pkg.instantiate()'
146+
julia --project=docs docs/make.jl
147+
# 6 static analysis: 0 unsuppressed critical
148+
panic-attack assail . --output /tmp/assail.json
149+
# 4,8 compat/registry + A2ML validity are enforced by CI gates
150+
# (registry ci/validate.jl; dogfood-gate Validate A2ML manifests)
151+
@echo "quality gates passed"
152+
153+
test:
154+
julia --project=. -e 'using Pkg; Pkg.test()'
155+
----
156+
157+
`just test && just quality` is the golden path; both green is the minimum for a
158+
release claim (release-claim-requires-hard-pass).
159+
160+
== Scoring (relative to the flagship)
161+
162+
For cross-library comparison, score each of the 10 dimensions 0–3 (0 = absent,
163+
1 = partial, 2 = meets grade-A bar, 3 = meets flagship bar). Max 30.
164+
165+
* `Axiom.jl` = 30/30 by definition (the reference).
166+
* A library is *flagship-track* at ≥ 27 with no dimension below 2.
167+
* Publish the vector, not just the total — a 24 with a 0 in "Security &
168+
soundness" is not a 24-quality library; the hole caps it (holes-first).
169+
170+
== Provenance
171+
172+
Every bar here was met, and the tool run, on `Axiom.jl` during the flagship
173+
initiative (gaps G01–G18 verified). This rubric is the generalisation of that
174+
work into a reusable standard; the template's `just quality` is its executable
175+
form.

0 commit comments

Comments
 (0)