Skip to content

Commit 514146d

Browse files
fix: retire the last @test_broken (main is red); land DB-6 docs + STATE refresh (#87)
## Summary **`main` is red right now** and this fixes it, plus lands two documentation changes that missed the #85/#86 merges (both PRs were merged before these commits were pushed). ### 1. The last `@test_broken` retires — main is red until it does KnotTheory.jl#52 merged the link canonicalisation, so the BR-5 crossing-order marker now reports **"Unexpected Pass"** — a hard error, and exactly the forcing signal the marker existed to produce. Measured against KnotTheory `main` after #52: **0 mismatches over 200 trials**, links and knots alike (was 11/200, every one a multi-component link). Validated at CI's 50 trials — all four BR-5 testsets pass with no Broken column. **quandledb now carries no `@test_broken` markers at all.** Every marker this campaign added was retired by the upstream fix it was tracking, which is the mechanism working as intended. ### 2. `docs/db-6-explain-strategy.md` — reconciling two surfaces under one name The document said "Phase A landed 2026-06-01" while issue #34's Phase A had never been built. Both statements were true of **different surfaces**, and conflating them is what made #34 look closable: | Surface | What | Landed | |---|---|---| | SQL-side | `EXPLAIN QUERY PLAN` over read-path SQL, HTTP endpoint | 2026-06-01 | | KRL-side | `explain` keyword + structured plan with `indexed`/selectivity | 2026-07-28 (#86) | Now names both explicitly and documents the KRL surface: syntax, plan shape, why `explain` parses to its own statement type, why selectivity is a labelled stub guarded by `plan_is_costed()`, why `INDEXED_COLUMNS` is deliberately short, and what Phase B actually needs (DB-3, #33). ### 3. `.machine_readable/6a2/STATE.a2ml` — the file agents read said nothing Last touched 2026-02-13, with `# No milestones recorded`, `# No blockers recorded`, `# No actions recorded`. Now carries measured state: phase `testing`, the DB-6/DB-3 milestone chain, the four live issues (Guix gate, ruleset `--admin`, doc drift, ungated Idris2/Zig/AffineScript tiers) each with an evidence path, ordered next actions with owners, and `zero-known-broken-markers = true`. `last-result` is deliberately **`warn`, not `pass`**: every gate the repo owns is green, but the estate-wide Guix policy check is red and is not green-able here in good faith (see `dev-notes/guix-policy-gate-advice-2026-07-28.md` — the gate is presence-only, so a stub `guix.scm` would buy green with a lie). Reported rather than suppressed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 9c8d705 commit 514146d

3 files changed

Lines changed: 125 additions & 29 deletions

File tree

.machine_readable/6a2/STATE.a2ml

Lines changed: 31 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,34 +5,56 @@
55
[metadata]
66
project = "quandledb"
77
version = "0.1.0"
8-
last-updated = "2026-02-13"
8+
last-updated = "2026-07-28"
99
status = "active"
10-
session = "converted from scheme — 2026-04-11"
10+
session = "CI/correctness campaign — 2026-07-21 → 2026-07-28"
1111

1212
[project-context]
1313
name = "Quandledb"
1414
purpose = """Knot-theory database application — the invariant/equivalence + semantic-identity face of the KRL stack (quandle fingerprints, equivalence/identity), as a read-only HTTP API + AffineScript UI wrapping the Skein.jl engine. Hosts the server-side KRL parser (server/krl/)."""
15-
completion-percentage = 40
15+
completion-percentage = 55
1616

1717
[position]
18-
phase = "initial" # design | implementation | testing | maintenance | archived
18+
phase = "testing" # design | implementation | testing | maintenance | archived
1919
maturity = "experimental" # experimental | alpha | beta | production | lts
2020

2121
[route-to-mvp]
2222
milestones = [
23-
# No milestones recorded
23+
{ id = "M-CI", name = "Mathematical core gated in CI", status = "done", landed = "2026-07-21", note = "PR #72: quandle axioms, Reidemeister invariance, BR-5 fuzz, EXPLAIN and TLA+ model check all run on every push; both new gate types negative-tested (a real failure exits non-zero)." },
24+
{ id = "DB-6A-SQL", name = "DB-6 Phase A, SQL surface", status = "done", landed = "2026-06-01", note = "SQLite EXPLAIN QUERY PLAN over read-path SQL, mutation-guarded HTTP endpoint." },
25+
{ id = "DB-6A-KRL", name = "DB-6 Phase A, KRL surface", status = "done", landed = "2026-07-28", note = "PR #86: `explain` keyword, KRLExplainStmt, structured plan with indexed + selectivity. Selectivity is an honest stub; plan_is_costed() is false until DB-3." },
26+
{ id = "DB-3", name = "Secondary indexes + column histograms", status = "not-started", note = "Issue #33. Blocks DB-6 Phase B: real selectivity cannot exist without it." },
27+
{ id = "DB-6B", name = "DB-6 Phase B, cost-based reordering", status = "blocked", blocked-by = "DB-3", note = "Deliberately not attempted. Reordering against uniform stub estimates would be arbitrary while appearing principled." },
2428
]
2529

2630
[blockers-and-issues]
2731
issues = [
28-
# No blockers recorded
32+
{ id = "GUIX-GATE", severity = "high", scope = "estate", summary = "governance / Guix primary / Nix fallback policy fails: repo has flake.nix and no Guix artefact. NOT locally fixable in good faith — the gate is presence-only (checks a filename exists, never reads it), so a stub guix.scm would buy green with a lie. Measured estate-wide: 163 stub / 43 placeholder / 40 wrong-identity guix.scm files all pass today. Recommendation is to fix the GATE first.", evidence = "dev-notes/guix-policy-gate-advice-2026-07-28.md" },
33+
{ id = "RULESET-ADMIN", severity = "medium", scope = "repo", summary = "Every merge to main requires --admin: the ruleset carries code_coverage minimum 95 on a repo with no coverage tooling (unsatisfiable), a code_scanning requirement on Scorecard (which never runs on a PR head, so the context is phantom), and copilot_code_review. Routine --admin bypass is how a genuinely failing gate stops being noticed.", remedy = "dev-notes/quandledb-ruleset-fix-2026-07-27.md + quandledb-ruleset-fixed.json (one gh api -X PUT)" },
34+
{ id = "DOC-DRIFT", severity = "low", scope = "repo", summary = "READINESS.md (grade D, assessed 2026-04-05) cites src/api/*.v — no .v files and no src/api/ exist; it also lists EXPLAINME / TEST-NEEDS / PROOF-NEEDS as missing, all of which now exist. ARCHITECTURE.md is scaffold boilerplate describing directories this repo does not have.", tracked = "issue #74" },
35+
{ id = "UNGATED-TIERS", severity = "medium", scope = "repo", summary = "Idris2 (src/abi), Zig (src/ffi) and AffineScript (frontend) have no CI gate at all. The Julia, BEAM, Agda and TLA+ tiers do.", tracked = "issue #73 covers the BEAM half, now closed" },
2936
]
3037

3138
[critical-next-actions]
3239
actions = [
33-
# No actions recorded
40+
{ order = 1, action = "Merge open PRs in order: #85 (unblocks red main) then #86 (DB-6 KRL surface).", owner = "human", note = "Both require --admin until RULESET-ADMIN is fixed." },
41+
{ order = 2, action = "Apply the ruleset fix so merges stop requiring --admin.", owner = "human", command = "gh api -X PUT repos/hyperpolymath/quandledb/rulesets/18110611 --input dev-notes/quandledb-ruleset-fixed.json" },
42+
{ order = 3, action = "Decide the Guix gate question at estate level before touching this repo's packaging.", owner = "human", note = "Writing a stub guix.scm here would pass CI and be a lie. See GUIX-GATE." },
43+
{ order = 4, action = "DB-3 (issue #33): secondary indexes + column histograms.", owner = "either", note = "The single highest-value next build: it unblocks DB-6 Phase B and turns plan_is_costed() true." },
44+
{ order = 5, action = "Re-assess READINESS.md against the tree as it now stands.", owner = "either", tracked = "issue #74" },
3445
]
3546

3647
[maintenance-status]
37-
last-run-utc = "2026-02-13T00:00:00Z"
38-
last-result = "unknown" # unknown | pass | warn | fail
48+
last-run-utc = "2026-07-28T19:45:00Z"
49+
last-result = "warn" # unknown | pass | warn | fail
50+
# warn, not pass: every gate the repo owns is green (KRL Verification incl.
51+
# the full Julia suite, Agda --safe, TLC, QD-12, CodeQL, Hypatia, gitleaks).
52+
# The single red check is the estate-wide Guix policy gate — see GUIX-GATE
53+
# above. It is not green-able here in good faith, so it is reported as a warn
54+
# rather than suppressed.
55+
zero-known-broken-markers = true
56+
# As of 2026-07-28 the test suites carry NO @test_broken markers. The last one
57+
# (BR-5 quandle_key crossing-order invariance, 11/200 mismatches) cleared when
58+
# KnotTheory.jl#52 fixed link normalisation. Every marker this campaign added
59+
# was removed by the upstream fix it was tracking, which is the mechanism
60+
# working as intended: a marker that survives is a lie waiting to be told.

docs/db-6-explain-strategy.md

Lines changed: 85 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,25 @@ Copyright (c) Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
44
-->
55
# DB-6: EXPLAIN strategy for read-only API queries
66

7-
Status: **Phase A landed 2026-06-01.** Phases B + C deferred.
7+
Status: **both Phase A surfaces landed.** SQL-side 2026-06-01; KRL-side
8+
2026-07-28 (issue #34). Phases B + C deferred.
9+
10+
> **Two surfaces, one issue number — read this first.**
11+
>
12+
> "DB-6 Phase A" has meant two different things in this repo, which caused a
13+
> real mis-assessment (the issue was believed closable when half of it had
14+
> never been built):
15+
>
16+
> | Surface | What it is | Where | Landed |
17+
> |---|---|---|---|
18+
> | **SQL-side** | SQLite `EXPLAIN QUERY PLAN` over the read-path SQL, exposed on an HTTP endpoint | `server/query_explain.jl`, `handle_explain` in `server/serve.jl` | 2026-06-01 (this document's original scope) |
19+
> | **KRL-side** | `explain` as a KRL keyword returning a structured plan with `indexed` and selectivity per filter | `server/krl/ExplainPlan.jl`, `KRLExplainStmt` | 2026-07-28 |
20+
>
21+
> Issue #34's acceptance criteria describe the **KRL-side** surface. Until
22+
> 2026-07-28 only the SQL-side existed: `explain` appeared in neither
23+
> `server/krl/Lexer.jl` nor `server/krl/Parser.jl`, and there was no
24+
> selectivity model anywhere — not even the stub the criteria permit.
25+
> Both now exist and are independently gated in CI.
826
927
## Goal
1028

@@ -126,3 +144,69 @@ Once that lands, the wrapper for `handle_knots` becomes one-line.
126144
- [hyperpolymath/quandledb#34](https://github.com/hyperpolymath/quandledb/issues/34) — DB-6 parent
127145
- [hyperpolymath/quandledb#33](https://github.com/hyperpolymath/quandledb/issues/33) — DB-3 Phase B + C parent (Skein.jl gap is shared)
128146
- [hyperpolymath/quandledb#42](https://github.com/hyperpolymath/quandledb/pull/42) — DB-3 Phase A (indexes that Phase A test 4 exercises)
147+
148+
149+
---
150+
151+
## KRL-side EXPLAIN (landed 2026-07-28)
152+
153+
### Syntax
154+
155+
`explain` prefixes a pipeline query. It returns the **plan**; it does not run
156+
the query.
157+
158+
```krl
159+
explain from knots | filter colouring_count_3 = 9 and crossing < 8 | take 5
160+
```
161+
162+
### Plan shape
163+
164+
```
165+
{"op" => "scan", "table" => "knots"}
166+
{"op" => "filter", "column" => "colouring_count_3", "comparator" => "=", "value" => 9,
167+
"indexed" => true, "selectivity_estimate" => 0.5, "selectivity_source" => "stub"}
168+
{"op" => "filter", "column" => "crossing", "comparator" => "<", "value" => 8,
169+
"indexed" => false, "selectivity_estimate" => 0.5, "selectivity_source" => "stub"}
170+
{"op" => "take", "n" => 5}
171+
```
172+
173+
A conjunction becomes **separate** filter operations. That is deliberate: it is
174+
what makes the Phase B reorder meaningful, since a reorder can only permute
175+
operations the plan actually distinguishes.
176+
177+
### Design decisions worth knowing
178+
179+
**`explain` parses to its own statement type.** `KRLExplainStmt` is distinct
180+
from `KRLQueryStmt` rather than a flag on it, so every existing consumer of
181+
`KRLQueryStmt` still means "this query will actually execute" and cannot be
182+
handed a plan where it expected rows.
183+
184+
**Selectivity is a stub, and says so.** There are no column histograms yet —
185+
that is DB-3 (#33), and the DB-6 criteria explicitly permit a stub until it
186+
lands. Every estimate is `SELECTIVITY_STUB` (0.5) carrying
187+
`"selectivity_source" => "stub"`, and `plan_is_costed(plan)` returns `false`
188+
while any stub remains. **That predicate is the guard that stops Phase B
189+
optimising against invented numbers** — a cost-based reorder driven by
190+
uniform 0.5 estimates would be arbitrary while looking principled.
191+
192+
**`indexed` is driven by a short explicit list.** `INDEXED_COLUMNS` names only
193+
columns that genuinely carry a secondary index. An over-claiming list would
194+
make `indexed` a fake signal, which is worse than reporting `false`. Extend it
195+
as DB-3 lands real indexes — not before.
196+
197+
**Predicates the plan cannot describe are not mis-described.** Column-to-column
198+
comparisons, nested booleans and calls report `"predicate" => "<expression>"`
199+
with no `column` key, rather than being given a fabricated column.
200+
201+
### What is deliberately NOT done
202+
203+
Phase B (cost-based reordering) is not attempted. The plan is returned in
204+
**execution order as written**. Phase B needs real selectivity, i.e. DB-3.
205+
206+
### Tests
207+
208+
`server/krl/test/explain_test.jl` — 25 assertions, in the dependency-free CI
209+
step (so it runs before `Pkg.instantiate` and cannot be masked by a dependency
210+
breakage). Two of its testsets assert the *honesty* properties above rather
211+
than the happy path: that selectivity is labelled a stub with
212+
`plan_is_costed == false`, and that an undescribable predicate stays opaque.

server/test_br5_fuzz.jl

Lines changed: 9 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -120,23 +120,13 @@ end
120120

121121
@testset "BR-5: crossing-order invariance on random braid words" begin
122122
rng = MersenneTwister(BR5_SEED + 1)
123-
# KNOWN-BROKEN (upstream KnotTheory.jl#51): the polynomial segments of
124-
# quandle_key are crossing-order sensitive on multi-component LINKS.
125-
# Measured over this 200-trial corpus, splitting by Delta(1):
126-
# links (Delta(1)=0): 11 mismatch, 87 agree
127-
# knots (Delta(1)=±1): 0 mismatch, 102 agree
128-
# Knots became fully order-invariant when KnotTheory#48 landed the
129-
# Delta(1)=+1 normalisation; that fix cannot apply to links, where
130-
# Delta(1)=0 and the code falls back to an order-sensitive rule. Worse,
131-
# conway_polynomial reconstructs from an assumed symmetry about
132-
# exponent 0, which no link satisfies (even span ⟹ half-integer
133-
# centre), so it returns genuinely DIFFERENT polynomials for the same
134-
# link (e.g. 1 + z² vs 1) — not merely a different unit.
135-
# NOTE: this count did NOT move when #48 merged. The earlier
136-
# attribution of these mismatches to the alexander/conway sign
137-
# convention was wrong; the sign fix was real but orthogonal.
138-
# The layers this repo owns (presentation_hash, colouring counts) are
139-
# asserted hard below and hold on every trial.
123+
# Crossing-order invariance of quandle_key, including its polynomial
124+
# segments. This was @test_broken at 11/200: every mismatch was a
125+
# multi-component LINK (knots were already invariant at 0/102). Upstream
126+
# KnotTheory.jl#51/#52 fixed both causes — an exponent window that could
127+
# not be canonicalised for odd spans, and a Conway expansion that emitted
128+
# even powers of z for links that require odd ones. Measured after that
129+
# merge: 0 mismatches over 200 trials, links and knots alike.
140130
quandle_key_mismatches = 0
141131
for trial in 1:BR5_TRIALS
142132
nstrands = rand(rng, 3:4)
@@ -156,8 +146,8 @@ end
156146
end
157147
end
158148
quandle_key_mismatches > 0 &&
159-
@info "BR-5 quandle_key order-sensitivity (known-broken)" quandle_key_mismatches BR5_TRIALS
160-
@test_broken quandle_key_mismatches == 0
149+
@info "BR-5 quandle_key order-sensitivity" quandle_key_mismatches BR5_TRIALS
150+
@test quandle_key_mismatches == 0
161151
end
162152

163153
# ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)