Skip to content

Commit c3dddfc

Browse files
authored
Merge pull request #511 from AdaWorldAPI/claude/perturbation-sim-calibrate-soa
perturbation-sim: substrate calibration (study as ground truth) + calibrated SoA member spec
2 parents 0e6452c + 35041fd commit c3dddfc

9 files changed

Lines changed: 886 additions & 0 deletions

File tree

.claude/board/TECH_DEBT.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,38 @@ timing-race hypothesis (read the actual `cargo llvm-cov` log with a scoped token
147147
Cross-ref: `.github/workflows/rust-test.yml` (test job mold step vs coverage job);
148148
`bindspace-singleton-to-mailbox-soa-v1` (the migration this is NOT).
149149

150+
**2026-06-16 addendum — the `test` job now hits the SAME cliff; fix extended to
151+
it (branch `claude/ci-test-job-debuginfo0`).** The cliff this entry called out as
152+
*"a 2/50 intermittent"* on the coverage job has now surfaced on the **plain
153+
`test`** job: `ld terminated with signal 7 [Bus error]` + an LLVM crash dump at
154+
the `cargo test --no-run` link step of `test_sql_query` / `intervene_counterfactual`.
155+
Root-caused to a **link-footprint growth, not a logic break** (a layout break would
156+
fail an assertion, not SIGBUS at link): **PR #507** (`0c6ef02c`, +4055/−1048 across
157+
`causal-edge` edge.rs/layout.rs — the ce64-v2 layout — and `cognitive-shader-driver`
158+
mailbox_soa.rs/driver.rs/planner_bridge.rs — MailboxSoaOwner + SurrealMailboxView,
159+
D-PG-6) grew the object-file set linked by the lance-graph integration tests enough
160+
to tip the previously-marginal `test`-job link over the same disk/RSS ceiling. It
161+
surfaced on the first full-workspace CI run *after* #507 (the two PRs between, #509
162+
and the perturbation-sim #511, are root-`exclude`d so their CI never linked the
163+
post-#507 tree — which is why this is "the first failing PR" yet not its fault).
164+
**This is a FENCE, not a root reduction:** it does not shrink #507's legitimate
165+
codegen; it removes the dead `debuginfo=1` weight (CI never opens a debugger) to
166+
buy headroom — exactly the b56bb2cd lever, now applied to the `test` job. **Fix:**
167+
job-level `RUSTFLAGS: "-C debuginfo=0 -C target-cpu=x86-64-v3"` on `test` (parity
168+
with `test-with-coverage`; mold already installed). Side effect: the `test` job
169+
gets its own Swatinem cache key (first run repopulates). **Confirm** on the next
170+
green `test` run. **Residual debt if it recurs after this:** the footprint is on a
171+
secular upward trend (every cognitive-layer PR adds codegen) — the durable fix is a
172+
bigger runner or splitting the integration-test link set, not repeatedly shaving
173+
flags. Separately, #507 left `intervene_counterfactual.rs:133/165` calling the
174+
**deprecated** `CausalEdge64::inference_type()` (the consumer-migration commit
175+
`8131c480` lives on the unmerged `claude/continue-ndarray-x0Oaw`) — that WARNS, does
176+
not fail (v1 default routes through the canonical mapping per I-LEGACY-API-FEATURE-
177+
GATED); tracked here as a separate latent item, not fixed on this CI branch.
178+
Cross-ref: `.github/workflows/rust-test.yml` (now both jobs at `debuginfo=0`); PR
179+
#507 (`0c6ef02c`); `claude/continue-ndarray-x0Oaw` (the pending ce64-v2 consumer
180+
migration).
181+
150182
### TD-UNBUNDLE-FROM-1 — `unbundle_from` is NOT the inverse of `bundle_into` (2026-06-07)
151183

152184
**Open.** `crates/lance-graph-planner/src/cache/kv_bundle.rs``unbundle_from`

.github/workflows/rust-test.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,22 @@ jobs:
2929
test:
3030
runs-on: ubuntu-24.04
3131
timeout-minutes: 30
32+
env:
33+
# Override the workflow-level debuginfo=1 for this job too (parity with
34+
# test-with-coverage, TD-CI-COVERAGE-MOLD-1). The `test` job links the
35+
# full lance+datafusion integration-test set at the SAME disk/RSS cliff
36+
# the coverage job hit — and #507 (+4055 lines across causal-edge +
37+
# cognitive-shader-driver: ce64-v2 layout + MailboxSoaOwner/
38+
# SurrealMailboxView) grew that link footprint enough to tip the
39+
# previously-marginal link into a hard `ld` SIGBUS (signal 7 = object
40+
# file truncated when the runner partition fills mid-link). debuginfo=1
41+
# carried no value here (CI never opens a debugger); dropping it cut the
42+
# coverage job's per-binary link from ~930 MB to ~252 MB (-73%, measured
43+
# in b56bb2cd) and relieves BOTH ceilings (mold/GNU-ld RSS + disk). mold
44+
# is already installed below. Note: a job-level RUSTFLAGS gives this job
45+
# its own Swatinem cache key — the first run after this change
46+
# repopulates the test cache.
47+
RUSTFLAGS: "-C debuginfo=0 -C target-cpu=x86-64-v3"
3248
defaults:
3349
run:
3450
working-directory: lance-graph
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Framing the resilience study as CLAM (hierarchy) + CHAODA (anomaly ensemble)
2+
3+
*The resilience study is not a bespoke method — it is a CLAM cluster tree with a
4+
CHAODA-style anomaly ensemble, on the electrical-distance manifold. Both already
5+
exist in `ndarray::hpc::clam` (cited below); this doc maps the correspondence and
6+
states honestly what is grounded vs what is a conceptual mapping not yet wired.*
7+
8+
> Operator prompt (2026-06-16): "you could even try to frame it — CLAM
9+
> (resilience) / CHAODA". Companion to `PAPER.md`, `COUNTRY_STUDY.md`,
10+
> `src/columns.rs`.
11+
12+
## The correspondence
13+
14+
| resilience-study object | CLAM / CHAODA construct | ndarray reference |
15+
|---|---|---|
16+
| recursive Cheeger/HHTL bisection into basins | **CLAM cluster tree** (`ClamTree::build`) | `hpc/clam.rs` `ClamTree` |
17+
| a basin (compartment) | a **`Cluster`** | `hpc/clam.rs:106` `Cluster { radius, cardinality, lfd }` |
18+
| basin algebraic connectivity λ₂ / mean R | cluster **radius** / spread | `Cluster::radius` |
19+
| basin node count | cluster **cardinality** | `Cluster::cardinality` |
20+
| how fragmented/space-filling a basin is | **local fractal dimension** `Lfd` | `hpc/clam.rs:81` `Lfd::compute(count_r, count_half_r)` |
21+
| fail-first / exposure ranking | **CHAODA anomaly score** | `hpc/clam.rs:1517` `ClamTree::anomaly_scores() -> Vec<AnomalyScore>` |
22+
| "this compartment can't wait" flag | CHAODA **flag threshold** (≥ 0.75) | `hpc/clam.rs` anomaly-flag test |
23+
24+
So the study's machinery is CLAM's machinery on a different metric: instead of a
25+
Hamming/embedding distance, the manifold is the **electrical distance** (effective
26+
resistance `R_ij = (e_i−e_j)ᵀ L⁺ (e_i−e_j)`, the self-inverse `L⁺` reference). The
27+
HHTL tiers ARE the CLAM tree depth; the weakest compartment IS the cluster CHAODA
28+
would score as the outlier.
29+
30+
## Why the three axes ARE a CHAODA ensemble (the load-bearing match)
31+
32+
CHAODA's thesis: **no single graph-anomaly method wins; ensemble several *diverse*
33+
detectors** (relative cardinality, parent/child cardinality ratio, graph
34+
neighbourhood, stationary distribution, …) and the gain comes from their
35+
*non-redundancy*. The resilience study's three axes are exactly such an ensemble:
36+
37+
- **topology** (λ₂ / Kirchhoff) — the connectivity detector,
38+
- **buffer** (inertia storage) — the transient detector,
39+
- **policy** (feed-in / dispatch) — the operational detector,
40+
41+
ensembled into the **exposure** score. And the study's measured **low / negative
42+
Cronbach α** (the axes are distinct facets, `Spearman ≈ 0` between them) is not a
43+
defect — it is *precisely CHAODA's design goal*: low inter-detector correlation is
44+
what makes the ensemble add information rather than restate it. The discriminant
45+
finding and the CHAODA non-redundancy principle are the same statement.
46+
47+
This also re-frames the §4.11 confound cleanly: the modifier `Weyl × (1/Fiedler)`
48+
failed as an independent axis because `1/λ₂` is the dominant Kirchhoff term — i.e.
49+
it was a **redundant detector**, the CHAODA anti-pattern. The buffer axis is the
50+
*orthogonal* detector the ensemble actually needed.
51+
52+
## Honest scope
53+
54+
- **Grounded [G]:** `CLAM`, `Cluster{radius,cardinality,lfd}`, `Lfd`, and
55+
`ClamTree::anomaly_scores` all exist in `ndarray::hpc::clam` (cited). The
56+
structural correspondence is exact, not metaphor.
57+
- **Conceptual [H]:** `perturbation-sim` is zero-dep and is **NOT wired** to
58+
`ndarray::hpc::clam`. The mapping above is read off the APIs, not run. No code
59+
here calls `ClamTree` or `anomaly_scores`.
60+
- **The falsifiable probe** that would promote [H][G]: build a `ClamTree` over the
61+
contingency factor vectors (or the per-basin `(λ₂, Kf, buffer)` rows), run
62+
`anomaly_scores`, and correlate the CHAODA ranking against the study's exposure
63+
ranking (ICC/Spearman, Jirak rate). If they agree, the study *is* CHAODA on the
64+
electrical manifold; if not, the framing is rhyme and gets retracted. This is the
65+
gated bridge (crosses perturbation-sim's zero-dep boundary into `ndarray`,
66+
behind a feature flag) — analogous to the calibration harness, not yet built.
67+
68+
## Tie-in to the calibrated columns (`src/columns.rs`)
69+
70+
CLAM gives two more value members for free, hung off the same HHTL-OGAR key:
71+
`radius` (basin spread) and `lfd` (local fractal dimension). They are helix-residue
72+
value members like the rest — orthogonal to topology by the key/value split — and
73+
the CHAODA `anomaly_score` is the *read* over the column set, the same way
74+
`exposure` is. The substrate that carries the study is therefore literally a CLAM
75+
tree of HHTL-keyed helix value members with a CHAODA read.

crates/perturbation-sim/Cargo.toml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,3 +92,11 @@ path = "examples/explore.rs"
9292
[[example]]
9393
name = "scorecard"
9494
path = "examples/scorecard.rs"
95+
96+
[[example]]
97+
name = "calibrate"
98+
path = "examples/calibrate.rs"
99+
100+
[[example]]
101+
name = "hhtl_grid"
102+
path = "examples/hhtl_grid.rs"

0 commit comments

Comments
 (0)