Skip to content

Commit d0a7690

Browse files
authored
Merge pull request #1024 from ruvnet/feat/v2-beyond-sota-sweep-m5
Beyond-SOTA sweep M5–M6 (ADR-159/160): appliance + edge-skill honesty + crates.io publish
2 parents 427c568 + 8487192 commit d0a7690

94 files changed

Lines changed: 2028 additions & 798 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

PROOF.md

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# PROOF — reproduce every claim, or find the one we can't yet
2+
3+
This project (RuView / wifi-densepose) has been publicly called "AI slop" and
4+
"fake." This document is the answer: **a skeptic can clone the repo, run one
5+
script, and have every headline claim either verified on their own machine or
6+
shown — explicitly — as "CLAIMED, not yet reproduced (here's exactly what it
7+
needs)."** Nothing below is asserted without a command you can run.
8+
9+
```bash
10+
git clone https://github.com/ruvnet/RuView && cd RuView
11+
bash scripts/prove.sh # core gate + the anti-slop assertion tests
12+
bash scripts/prove.sh --full # also attempt the feature-gated subset
13+
```
14+
15+
`prove.sh` exits 0 only if every **non-gated** claim passes. Gated claims never
16+
fail the run; they print the prerequisite (a GPU, a dataset, real hardware, a
17+
trained checkpoint) so you can reproduce them yourself.
18+
19+
## Grading
20+
21+
- **MEASURED** — reproduced on our hardware, with the exact command recorded, and
22+
pinned by a test that *fails on the pre-fix code*. `prove.sh` re-runs these.
23+
- **CLAIMED** — cited from a source, or measured by the source, but not
24+
reproduced in this repo's automated harness.
25+
- **DATA-GATED / HARDWARE-GATED** — the *code path* is real and tested, but the
26+
*accuracy/throughput claim* needs data or hardware we don't ship. We never
27+
fabricate the number; the code carries a typed error or a `weights_trained`/
28+
provenance flag instead.
29+
30+
## The hard gate (run on any machine with Rust + Python)
31+
32+
| Claim | Grade | Reproduce |
33+
|---|---|---|
34+
| Rust workspace: 3,128 tests, 0 failed | **MEASURED** | `cd v2 && cargo test --workspace --no-default-features` |
35+
| Deterministic CSI pipeline proof (bit-exact SHA-256) | **MEASURED** | `python archive/v1/data/proof/verify.py``VERDICT: PASS` |
36+
37+
## Anti-slop assertion tests (each fails on the pre-fix code)
38+
39+
| Claim | Grade | Test (run via `cargo test -p <crate> <name>`) |
40+
|---|---|---|
41+
| Fusion crafted-input DoS panics are closed (ADR-156 §2.2) | **MEASURED** | `wifi-densepose-ruvector :: triangulation_out_of_range_index_returns_none_no_panic` |
42+
| **The "Soul Signature" identity claim, honestly bounded:** on WiFi-only cardiac+respiratory channels two people are **not separable** (gap ≈ 0.0005) | **MEASURED** | `wifi-densepose-bfld :: cardiac_alone_cannot_separate_identity_matches_audit` |
43+
| OccWorld `predict()` is real (input-dependent), not random noise | **MEASURED** | `wifi-densepose-occworld-candle :: predict_is_deterministic_for_same_input` |
44+
| Pose runtime emits frames under its own default config (ADR-159 A1) | **MEASURED** | `cog-pose-estimation :: default_config_emits_frames_with_real_model` |
45+
| Person-count flags untrained classes — no count inflation (ADR-159 A2) | **MEASURED** | `cog-person-count :: untrained_class_argmax_is_flagged_low_confidence` |
46+
| Medical edge skills carry a "not a medical device" disclaimer (ADR-160 A1) | **MEASURED** | `wifi-densepose-wasm-edge :: a1_med_modules_have_clinical_disclaimer` (`--features std`) |
47+
| Survivor dedup 3→1, count-inflation killed (ADR-158 §2) | **MEASURED** | `wifi-densepose-mat :: test_identical_vitals_no_location_dedup_to_one` (`--features mat`) |
48+
49+
## Measured performance (criterion; reproduce on your machine)
50+
51+
| Claim | Grade | Reproduce |
52+
|---|---|---|
53+
| PSD FFT-planner cache 2.0–3.1×, DTW band 2.4–4.1× (ADR-154) | **MEASURED** | `cd v2 && cargo bench -p wifi-densepose-signal` |
54+
| fuse() double-clone removed ~2.17× marshalling (ADR-156) | **MEASURED** | `cd v2 && cargo bench -p wifi-densepose-ruvector --bench fusion_bench` |
55+
| zero-copy ORT input ~1.48× (ADR-155) | **MEASURED** | `cd v2 && cargo bench -p wifi-densepose-nn --features onnx --bench onnx_bench` |
56+
| pointcloud splats 9→2 passes ~1.24× (ADR-160 research) | **MEASURED** | `cd v2 && cargo bench -p wifi-densepose-pointcloud --bench splats_bench` |
57+
| native wlanapi multi-BSSID scan 9.74 Hz (vs netsh ~2 Hz) | **MEASURED (Windows)** | `cd v2 && cargo test -p wifi-densepose-wifiscan -- --ignored measure_native_scan_rate` |
58+
59+
## What we do NOT claim (the honest negatives — the strongest anti-slop signal)
60+
61+
| Capability | Status |
62+
|---|---|
63+
| **Named person-identity from WiFi** | **NOT achieved, and measured why.** The §3.6 matcher is real, but identity does not lock on WiFi-only channels (gap 0.0005). DATA-GATED on a real enrollment feeding the AETHER/body-resonance channel — never done. No named-identity claim is made. |
64+
| WiFlow-STD ~96% PCK@20 | **CLAIMED-reproduced** on our RTX 5080 (`benchmarks/wiflow-std/RESULTS.md`); HARDWARE-GATED for you (needs an NVIDIA GPU + the MM-Fi dataset). The upstream *shipped checkpoint* was **REFUTED** (0.08% PCK) — we publish that. |
65+
| OccWorld trajectory accuracy | DATA-GATED on a trained checkpoint; `predict()` carries `weights_trained=false` until one is loaded — never silently faked. |
66+
| Edge-skill detection accuracy (seizure, weapon, affect, …) | UNVALIDATED — every such module is now disclaimer-gated as experimental/research; the DSP is real, the accuracy is not claimed. |
67+
| 802.11bf-2025 OTA conformance | No commodity silicon ships a conformant interface as of 2026; ours is a simulation-tested forward-compat protocol model, not a certified implementation. |
68+
69+
## Provenance
70+
71+
Every claim above traces to a committed ADR (`docs/adr/ADR-154``ADR-160`), a
72+
test, a criterion bench, or `benchmarks/wiflow-std/RESULTS.md`. The history
73+
includes published **retractions** (the 92.9% PCK retraction; the WiFlow-STD
74+
shipped-checkpoint refutation; the NV-diamond BOM reality check) — a faker hides
75+
failures; we commit them.
Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
# ADR-159: Cognitum Appliance Cluster — Beyond-SOTA Sweep, Anti-"AI-Slop" Hardening
2+
3+
- **Status**: accepted
4+
- **Date**: 2026-06-11
5+
- **Deciders**: ruv
6+
- **Tags**: cognitum, cogs, person-count, pose-estimation, ha-matter, drone-swarm, remote-id, manifest, prove-everything
7+
8+
## Context
9+
10+
This ADR records the beyond-SOTA sweep over the Cognitum appliance cluster
11+
(`cog-person-count`, `cog-pose-estimation`, `cog-ha-matter`, `ruview-swarm`),
12+
executed under the project's **prove-everything / anti-"AI-slop"** directive: the
13+
claim surface every cog presents (manifests, descriptions, runtime events,
14+
broadcast fields) must match what the code and the shipped weights actually do.
15+
16+
### Headline — the "never identified anyone" accusation is REFUTED
17+
18+
A read-only audit raised the worst-class accusation: that these cogs are slop that
19+
"never identified anyone." That accusation is **refuted by byte-level evidence**:
20+
21+
- `cog-pose-estimation` and `cog-person-count` ship **real, trained Candle models**
22+
(`pose_v1.safetensors`, `count_v1.safetensors`), not placeholders. The forward
23+
passes (`PoseNet`, `CountNet`) mirror the training scripts exactly and run on
24+
real CSI bytes.
25+
- The artifacts are **SHA-pinned and Ed25519-signed**: the on-disk
26+
`manifests/x86_64/manifest.json` carries a real `binary_sha256`
27+
(`051614ce…388b3` for person-count, `a434739a…71fa` for pose), a real
28+
`weights_sha256`, and a `binary_signature` over `sig_algo: Ed25519`.
29+
- The manifests are **brutally honest about accuracy**: person-count's
30+
`build_metadata` ships `training_class1_accuracy = 0.343` and a candid
31+
`training_caveat`; pose ships `training_pck20 = 3.0` / `training_pck50 = 18.5`.
32+
Nothing is inflated. That honesty *is* the anti-slop win — the models are weak
33+
in the field, and the manifests say so.
34+
35+
So the cogs **do** run real trained inference and **do** disclose how weak it is.
36+
What the audit correctly found were not fabrications but **claim-surface
37+
overclaims** — four places where the surface said more than the weights deliver.
38+
This ADR tightens those four (A1–A4) and cites the already-correct subsystems as
39+
NO-ACTION positives.
40+
41+
Grading vocabulary follows ADR-152 / ADR-158:
42+
- **MEASURED** — reproduced in this worktree, command + failing-on-old test recorded.
43+
- **DATA-GATED** — real code path present; honestly flagged where data/hardware is absent.
44+
- **NO-ACTION (already-SOTA)** — audited, found correct, cited as a positive.
45+
- **ACCEPTED-FUTURE** — deliberately deferred, nothing dropped.
46+
47+
## Graded SOTA Landscape
48+
49+
| Capability | Grade | Note |
50+
|------------|-------|------|
51+
| CSI person counting (`cog-person-count`) | **DATA-GATED** | Real Candle count head + Bayesian fusion; weights trained only on classes 0/1 (presence). Multi-occupant accuracy is genuinely unproven and is **not fabricated** — counts above the trained range are now flagged `low_confidence` and clamped. |
52+
| CSI pose estimation (`cog-pose-estimation`) | **DATA-GATED** | Real Candle encoder + 17-keypoint head; field accuracy honestly weak (PCK@50 = 18.5%, disclosed in the manifest). The default-install gate bug (A1) is fixed so it actually emits frames. |
53+
| Signed cog manifests (Ed25519 + SHA-256) | **NO-ACTION (already-SOTA)** | On-disk manifests are real, signed, SHA-pinned, and honest about accuracy. The CLI now emits them verbatim (A4). |
54+
| HA bridge (`cog-ha-matter`) MQTT + witness | **NO-ACTION (already-SOTA)** | Real Ed25519 hash-chain witness, mDNS, embedded broker. Matter commissioning is honestly deferred to v0.8 (TLS off, LAN-only) — description softened to stop claiming Matter (honest-absence). |
55+
| Drone-swarm MARL (`ruview-swarm`) | **DATA-GATED / honest** | `candle_ppo.rs` is real autodiff PPO; it is **untrained at runtime** (random init) by design — the swarm must be trained before deploy, which the code does not hide. |
56+
| ASTM F3411 Remote ID | **MEASURED (A3)** | Basic ID message is real; the Location/Vector message is honestly *not* implemented (NED metres are no longer mislabelled as WGS84 lat/lon). |
57+
58+
## Decision — Fixes Landed (MEASURED)
59+
60+
### §A1 Pose runtime emitted ZERO frames under default config (HIGH)
61+
62+
**Overclaim (silent correctness bug):** `inference.rs` hardcoded
63+
`confidence: 0.185` for every inference, `config.rs default_min_confidence()`
64+
returned `0.3`, and `runtime.rs` gated emission on `confidence >= min_confidence`.
65+
A default install therefore **never emitted a single `pose.frame`** while
66+
`health` reported healthy — the cog *claimed* to be a running pose estimator but
67+
silently produced nothing.
68+
69+
**Real fix:** `pose_v1` has **no confidence head** (the head emits 34 keypoint
70+
coordinates only), so a real per-frame confidence is genuinely unavailable. We
71+
took the disclosed "ok" path rather than silently lowering the threshold:
72+
- Introduced `inference::MODEL_TYPICAL_CONFIDENCE = 0.185` (the validation PCK@50)
73+
as the single published per-frame confidence, used by both `infer()` and the
74+
config default.
75+
- Pinned `default_min_confidence()` to `MODEL_TYPICAL_CONFIDENCE` so a default
76+
install clears its own gate and emits.
77+
- Documented the trade-off in the config field doc, the JSON schema
78+
(`default` 0.3 → 0.185, with a description), **and** added a `run.started`
79+
warning in `main.rs` that fires when an operator raises `min_confidence` above
80+
the model's typical confidence — so a deliberately-high threshold is loud, not
81+
silent.
82+
83+
**Failing-on-old test:** `cog_pose_estimation` smoke
84+
`default_config_emits_frames_with_real_model` — parses a default config and
85+
asserts `min_confidence <= MODEL_TYPICAL_CONFIDENCE` (and, with the real model
86+
loaded, that `infer().confidence >= min_confidence`). **Proven to fail** on the
87+
old `default_min_confidence()=0.3`:
88+
`default min_confidence 0.3 exceeds model typical confidence 0.185 — a default
89+
install would emit zero pose.frame events`.
90+
91+
**Grade: MEASURED.**
92+
93+
### §A2 8-class count head on a 2-class-trained model (MEDIUM)
94+
95+
**Overclaim:** `inference.rs COUNT_CLASSES = 8` with argmax over {0..7}, but
96+
`count_train_results.json` has support only for classes 0 and 1 (`per_class_accuracy`
97+
keys `"0"`/`"1"`). The model is a **presence detector**, not a calibrated
98+
multi-occupant counter; an argmax on classes 2..=7 is out-of-distribution, yet the
99+
cog would emit it as a confident headcount. The Cargo.toml billed it as a
100+
"learned multi-person counter."
101+
102+
**Real fix (no network change — DATA-GATED, accuracy not fabricated):**
103+
- Added `inference::MAX_TRAINED_CLASS = 1`, plus `CountPrediction::is_low_confidence()`
104+
(argmax beyond the trained ceiling) and `clamped_count()` (report clamped to the
105+
trained range, raw argmax kept for audit).
106+
- `person.count` events now carry `low_confidence` + `raw_count`, and downgrade to
107+
`level: "warn"` when out-of-distribution; the reported `count` is clamped so we
108+
never emit a fabricated headcount the weights can't back.
109+
- `run.started` discloses `count_max_trained_class` and `count_classes`.
110+
- Cargo.toml description changed from "learned multi-person counter" to
111+
"presence detector + (data-gated) person count".
112+
113+
**Failing-on-old test:** `cog_person_count` smoke
114+
`untrained_class_argmax_is_flagged_low_confidence` — a prediction whose argmax is
115+
class 5 is asserted `is_low_confidence() == true` and `clamped_count() ==
116+
MAX_TRAINED_CLASS`; a class-1 prediction is asserted *not* flagged. Fails on old
117+
code (no such methods/flag existed).
118+
119+
**Grade: MEASURED (mechanism); multi-occupant accuracy DATA-GATED.**
120+
121+
### §A3 Remote ID broadcast NED metres as WGS84 lat/lon (MEDIUM — safety/compliance)
122+
123+
**Overclaim (compliance hazard):** `security/remote_id.rs update()` stored
124+
`state.position.x/.y` (NED **metres**) into `drone_lat`/`drone_lon`, so the Remote
125+
ID broadcast would carry physically-impossible coordinates (e.g. "latitude =
126+
37.5 m"). The module doc claimed a "Basic ID + Location/Vector message," but only
127+
`encode_basic_id()` exists.
128+
129+
**Real fix (honest naming — never broadcast impossible coordinates):**
130+
- Renamed `drone_lat`/`drone_lon``drone_north_m`/`drone_east_m` (NED metres
131+
relative to the operator/takeoff datum), with field docs stating they are *not*
132+
geodetic. `operator_lat`/`operator_lon` remain true WGS84 (from the operator's
133+
GNSS).
134+
- Corrected the module doc to claim **Basic ID only**; the Location/Vector encoder
135+
is explicitly deferred until a datum-anchored NED→WGS84 transform lands
136+
(ACCEPTED-FUTURE), rather than removing a real feature.
137+
138+
**Failing-on-old test:** `security::remote_id::tests::test_ned_offset_stored_as_metres_not_latlon`
139+
— a 37.5 m north / −12.0 m east NED offset is asserted to land in
140+
`drone_north_m`/`drone_east_m`; the operator's real WGS84 fix stays in range. Fails
141+
on old code, where these values were stored into `drone_lat`/`drone_lon`.
142+
143+
**Grade: MEASURED.**
144+
145+
### §A4 Hollow CLI manifest (LOW)
146+
147+
**Overclaim:** `cog-person-count main.rs cmd_manifest` emitted a null skeleton
148+
(`binary_sha256: null`, no training metadata), making the CLI look unsigned even
149+
though the **real signed manifest** existed at
150+
`cog/artifacts/manifests/x86_64/manifest.json`.
151+
152+
**Real fix:** new `cog_person_count::manifest` module `include_str!`-embeds the
153+
real signed manifests (x86_64 + arm), selected by build target arch.
154+
`cmd_manifest` now parses-then-emits the embedded signed manifest — exactly the
155+
pattern `cog-pose-estimation`'s `manifest_roundtrips` test demonstrates. The CLI
156+
now reports the real `binary_sha256`, `weights_sha256`, Ed25519 signature, and
157+
honest `build_metadata` (`training_class1_accuracy = 0.343`).
158+
159+
**Failing-on-old test:** `manifest::tests::embedded_manifest_has_non_null_binary_sha256`
160+
asserts a 64-hex-char `binary_sha256`; companions assert the embedded manifest is
161+
signed (`sig_algo == Ed25519`) and `id == COG_ID`. End-to-end verified:
162+
`cog-person-count manifest` prints `binary_sha256:
163+
051614ce6ba63df704fae848a67ad095df4bb88862fdff05ef3c0419cc8388b3`.
164+
165+
**Grade: MEASURED.**
166+
167+
### §A5 cog-ha-matter description claimed Matter before it exists (LOW — honest-labeling)
168+
169+
**Overclaim:** the Cargo.toml description said "Home Assistant + Matter
170+
integration," but Matter commissioning is deferred to v0.8 (`TlsConfig::Off`,
171+
LAN-only, asserted by `runtime.rs tls_defaults_to_off_for_v1_lan_only`).
172+
173+
**Real fix (no code change):** softened the description to "Home Assistant (MQTT)
174+
integration … LAN-only (no TLS); Matter Bridge commissioning is deferred to v0.8
175+
and not yet implemented." Mirrors ADR-158 §6 honest-absence: state what isn't
176+
there rather than implying it is.
177+
178+
**Grade: MEASURED (label).**
179+
180+
## Negative Results (Confirmed — NO-ACTION positives)
181+
182+
Audited and found genuinely correct; cited as positives, not edited:
183+
184+
- **`cog-ha-matter` witness chain** (`witness.rs` / `witness_signing.rs`) — real
185+
Ed25519 hash-chained witness log. Already-SOTA.
186+
- **`cog-person-count` fusion** (`fusion.rs`) — real Bayesian product-of-experts
187+
multi-node fusion (Stoer-Wagner-bounded clip), not a heuristic. Already-SOTA.
188+
- **`ruview-swarm` PPO** (`marl/candle_ppo.rs`) — real Candle autodiff PPO with a
189+
genuine policy-gradient update; its `randn` uses (init, action sampling,
190+
exploration) are all legitimate, not fake-output substitutes. Untrained at
191+
runtime by design (the swarm must be trained before deploy), which the code
192+
does not hide. Already-SOTA / honest.
193+
194+
## Deferred Backlog (Nothing Dropped)
195+
196+
- **Multi-occupant count accuracy** — DATA-GATED on labelled multi-occupant CSI.
197+
The `low_confidence` flag + clamp (§A2) is the honest stand-in until then.
198+
- **Remote ID Location/Vector message** — ACCEPTED-FUTURE; requires a
199+
datum-anchored local-tangent-plane NED→WGS84 transform with an operator datum.
200+
Basic ID ships today.
201+
- **Matter Bridge commissioning** — ACCEPTED-FUTURE (v0.8); LAN-only MQTT ships today.
202+
- **Criterion benches** for cog inference latency and `mesh_guard` — ACCEPTED-FUTURE
203+
(cold-start timings are recorded in the manifests' `build_metadata`, not yet a
204+
regression bench).
205+
- **`wasm-edge` skill accuracy** — unvalidated; **now honestly labelled, not
206+
claimed** (done in ADR-160: medical/affect/security/exotic claim surfaces
207+
disclaimed, renamed, and feature-gated; per-skill accuracy remains DATA-GATED).
208+
209+
## Consequences
210+
211+
- A default pose-estimation install now actually emits `pose.frame` events;
212+
raising the threshold above the model's reach is a loud `run.started` warning,
213+
not a silent dropout.
214+
- A person-count reading on an untrained class is flagged `low_confidence`,
215+
clamped, and downgraded to `warn` — no fabricated headcounts.
216+
- The Remote ID broadcast can never carry physically-impossible coordinates; NED
217+
metres live in honestly-named metre fields.
218+
- `cog-person-count manifest` now reports the real signed manifest instead of a
219+
hollow null skeleton.
220+
- No cog Cargo.toml description claims a capability (multi-person counting, Matter)
221+
the code/weights don't yet deliver.
222+
223+
## Reproduction (MEASURED)
224+
225+
```bash
226+
cd v2
227+
cargo test -p cog-person-count -p cog-pose-estimation -p cog-ha-matter -p ruview-swarm \
228+
--no-default-features
229+
# ruview-swarm train path compiles (PPO autodiff)
230+
cargo check -p ruview-swarm --features train
231+
# A4 end-to-end — real signed manifest, non-null binary_sha256
232+
cargo run -q -p cog-person-count --no-default-features -- manifest
233+
```
234+
235+
Result at time of writing (all 0 failed):
236+
- `cog-person-count`**19 passed** (lib 10 incl. 3 manifest; smoke 9)
237+
- `cog-pose-estimation`**8 passed** (smoke)
238+
- `cog-ha-matter`**64 passed** (unchanged; description-only edit)
239+
- `ruview-swarm`**117 passed** (default features); `--features train` compiles clean.
240+
241+
Scope was limited to the four named crates. NO-ACTION positives (witness chain,
242+
fusion, PPO + randn audit) were verified by inspection and left untouched.

0 commit comments

Comments
 (0)