Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions .claude/board/AGENT_LOG.md

Large diffs are not rendered by default.

28 changes: 28 additions & 0 deletions .claude/board/TECH_DEBT.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,34 @@

## Open Debt

### TD-NDARRAY-SIMD-POPCNT-NATIVE — `extract_rules` SIGILLs under `-C target-cpu=native` on larger RowMasks (2026-06-14)

**Surfaced by** the `invariance_witness_probe` (lance-graph-arm-discovery, `--features ndarray-simd`).
Reproducible: `RUSTFLAGS="-C target-cpu=native"` → the probe prints its header, then dies with
**signal 4 (SIGILL, exit 132), no Rust panic**, inside `extract_rules` (the per-candidate count loop,
which under `ndarray-simd` routes RowMasks AND+popcount through `ndarray::simd::U64x8`).

**Characterized:**
- Crashes at n = 24 000 rows (375 popcount-words) AND n = 24 576 (384 words, no 8-word tail) → **not**
a tail-remainder bug; it scales with mask size.
- The `meta_awareness_probe` at n = 4 096 rows (64 words) runs FINE under the same `native` flags →
the faulting kernel is only reached for larger masks.
- The **default codegen path (no `target-cpu`) and CI's `x86-64-v3` (AVX2, no AVX-512) are unaffected**;
the probe result is bit-identical there (INV1 ρ=+0.894 etc.). Both probes are documented to run
WITHOUT `native`.

**Most likely cause (unconfirmed):** compile-time `native` on this sandbox enables an AVX-512 feature
(suspect VPOPCNTQ / AVX-512-VPOPCNTDQ from the U64x8 popcount, possibly AMX) that the **virtualized
runtime CPU does not actually implement** → illegal instruction. A build-config × sandbox-CPU mismatch,
not necessarily a logic bug. NOTE: `lance-graph-arm-discovery/Cargo.toml` *recommends* `target-cpu=native`
/ `x86-64-v4` in a comment — that is the trap a consumer would copy.

**Owed:** simd-savant / sentinel-qa triage to (a) pin the exact faulting instruction (objdump the
U64x8 popcount kernel + the AMX re-exports), (b) decide whether ndarray's `U64x8` should runtime-detect
VPOPCNTDQ/AMX rather than trust compile-time `target_feature`, and (c) correct the Cargo.toml comment to
warn that `native` can SIGILL on virtualized hosts lacking the detected AVX-512 subset. Until then:
run the `ndarray-simd` probes on the default path. **Status: Open.**

### TD-CI-COVERAGE-MOLD-1 — `test-with-coverage` job lacks the mold linker the `test` job has (2026-06-12)

**2026-06-12 local-repro addendum (same PR, later commit) — diagnosis CONFIRMED
Expand Down
28 changes: 26 additions & 2 deletions .claude/knowledge/bf16-hhtl-terrain.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
## READ BY: truth-architect, savant-research, family-codec-smith,
## palette-engineer, integration-lead, container-architect

## STATUS: 5 corrections absorbed, 0 probes run. All claims are CONJECTURES
## until the probe queue drains.
## STATUS: 6 corrections absorbed, 0 probes run. All claims are CONJECTURES
## until the probe queue drains. Correction 6 (2026-06-14) reclassifies
## the whole float-reconstruction framing as a category error for
## byte-encoded models — see the Correction Chain.
Comment on lines +6 to +9

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Update the status banner to match the measured correction.

The header still says "0 probes run" and "all claims are conjectures," but Iteration 6 now cites measured results and a certification artifact. Split the banner so only the earlier iterations stay conjectural, or update the status once this evidence is promoted.

📝 Suggested edit
-## STATUS: 6 corrections absorbed, 0 probes run. All claims are CONJECTURES
-##         until the probe queue drains. Correction 6 (2026-06-14) reclassifies
-##         the whole float-reconstruction framing as a category error for
-##         byte-encoded models — see the Correction Chain.
+## STATUS: 6 corrections absorbed. Earlier corrections remain conjectural
+##         until the probe queue drains; Correction 6 (2026-06-14) is
+##         evidence-backed and reclassifies the whole float-reconstruction
+##         framing as a category error for byte-encoded models — see the
+##         Correction Chain.

Also applies to: 48-68

🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 7-7: Multiple spaces after hash on atx style heading

(MD019, no-multiple-space-atx)


[warning] 8-8: Multiple spaces after hash on atx style heading

(MD019, no-multiple-space-atx)


[warning] 9-9: Multiple spaces after hash on atx style heading

(MD019, no-multiple-space-atx)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/knowledge/bf16-hhtl-terrain.md around lines 6 - 9, Update the STATUS
banner to reflect that Iteration 6 has measured results and certification
artifacts, not just conjectures. Instead of claiming "0 probes run" and "all
claims are CONJECTURES," distinguish between the earlier iterations (which
remain conjectural) and Iteration 6 (which now has measured evidence and
certification). Rewrite the banner to clarify that only the pre-Iteration-6 work
remains conjectural, while Iteration 6 onwards is based on measured results.


---

Expand Down Expand Up @@ -42,6 +44,28 @@ Iteration 5: Slot D = CLAM tree path (bucketing > resolution)
→ Correct: Slot D is PRIMARY (bucket address), Slot V is REFINEMENT (exact value)
→ Correct: 3-level 16-way CLAM = 12 bits → 16 → 256 → 4096 alignment
→ UNVERIFIED: requires Probe M1 (CLAM tree fit on 256 Jina centroids)

Iteration 6 (operator, 2026-06-14): float codec on byte-encoded model = CATEGORY ERROR
→ Measured: bgz-hhtl-d certified on real Qwen3-TTS-1.7B FAILS reconstruction on
all 16 roles (palette/hhtld cos ~0.05–0.32, Base17 cos ~0.01–0.37 at the first
fold, matvec rel-err ~1.0, fisher-z ρ 0.03–0.68) — cert at
.claude/knowledge/certification/hhtld_qwen3tts17b.json. The scalar Slot V (BF16
residual) adds ≈0 over palette-only (palette_cos ≈ hhtld_cos).
→ Root insight: this is NOT just a weak residual. It is a category error. Qwen's
information lives in a BYTE-QUANTIZED REGISTER (GGUF / byte-BPE / bgz7), not a
continuous float field. Scoring float-RECONSTRUCTION fidelity (Base17 fold +
cosine/L2) on a discrete/byte representation measures the wrong category. Same
failure mode as I-VSA-IDENTITIES, one layer down: decoding a quantized register
back to f32 to score cosine IS the register-loss the iron rule forbids.
→ Correction: for byte-encoded models the success metric is discrete CODE-AGREEMENT
/ bucket-ROUTING (the rolling floor), NOT reconstruction. The codec should never
decode to f32 to score cosine. Reconstruction is the wrong question.
→ Evidence the right-category path works (ndarray PR #218 probe family):
rolling_floor_probe (coarse code routed via a self-calibrating μ+3σ floor,
ρ=1.0 tracking, reject stable ~0.1% under SD drift), helix_bitdepth_probe
(direction as a discrete golden index, 24-bit lossless vs ≤f16), and
morton_perturbation_probe (discrete frame drives a field on-demand). None
decode-to-f32-to-score-cosine.
```

## The Five Hard Constraints
Expand Down
46 changes: 38 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions crates/lance-graph-arm-discovery/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,23 @@ optional = true

[lib]
doctest = true

# Meta-awareness probe: real Aerial+ extraction → entropy ladder + Granger meta,
# witness-as-pointer (meta derived, never stored). Needs the ndarray meta surface,
# so it rides the `ndarray-simd` feature (the only path that pulls ndarray here).
[[example]]
name = "meta_awareness_probe"
required-features = ["ndarray-simd"]

# Invariance-witness probe: basins as ICP environments — a confounded pair that entropy
# trusts (low H) but cross-basin invariance refutes (the MIT causal-discovery third axis).
[[example]]
name = "invariance_witness_probe"
required-features = ["ndarray-simd"]

# Coreference rung-separation probe: relative-pronoun resolution over the REAL 144-cell
# verb_table (VerbFamily × Tense → SlotPrior) as a syntax/Semantik/pragmatic split.
# Needs the contract (verb_table, via `landing`) AND ndarray (reliability, via `ndarray-simd`).
[[example]]
name = "coreference_rung_probe"
required-features = ["ndarray-simd", "landing"]
Loading
Loading