|
| 1 | +# ADR-159: Cognitum Appliance Cluster — Beyond-SOTA Sweep, Anti-"AI-Slop" Hardening |
| 2 | + |
| 3 | +- **Status**: accepted |
| 4 | +- **Date**: 2026-06-11 |
| 5 | +- **Deciders**: ruv |
| 6 | +- **Tags**: cognitum, cogs, person-count, pose-estimation, ha-matter, drone-swarm, remote-id, manifest, prove-everything |
| 7 | + |
| 8 | +## Context |
| 9 | + |
| 10 | +This ADR records the beyond-SOTA sweep over the Cognitum appliance cluster |
| 11 | +(`cog-person-count`, `cog-pose-estimation`, `cog-ha-matter`, `ruview-swarm`), |
| 12 | +executed under the project's **prove-everything / anti-"AI-slop"** directive: the |
| 13 | +claim surface every cog presents (manifests, descriptions, runtime events, |
| 14 | +broadcast fields) must match what the code and the shipped weights actually do. |
| 15 | + |
| 16 | +### Headline — the "never identified anyone" accusation is REFUTED |
| 17 | + |
| 18 | +A read-only audit raised the worst-class accusation: that these cogs are slop that |
| 19 | +"never identified anyone." That accusation is **refuted by byte-level evidence**: |
| 20 | + |
| 21 | +- `cog-pose-estimation` and `cog-person-count` ship **real, trained Candle models** |
| 22 | + (`pose_v1.safetensors`, `count_v1.safetensors`), not placeholders. The forward |
| 23 | + passes (`PoseNet`, `CountNet`) mirror the training scripts exactly and run on |
| 24 | + real CSI bytes. |
| 25 | +- The artifacts are **SHA-pinned and Ed25519-signed**: the on-disk |
| 26 | + `manifests/x86_64/manifest.json` carries a real `binary_sha256` |
| 27 | + (`051614ce…388b3` for person-count, `a434739a…71fa` for pose), a real |
| 28 | + `weights_sha256`, and a `binary_signature` over `sig_algo: Ed25519`. |
| 29 | +- The manifests are **brutally honest about accuracy**: person-count's |
| 30 | + `build_metadata` ships `training_class1_accuracy = 0.343` and a candid |
| 31 | + `training_caveat`; pose ships `training_pck20 = 3.0` / `training_pck50 = 18.5`. |
| 32 | + Nothing is inflated. That honesty *is* the anti-slop win — the models are weak |
| 33 | + in the field, and the manifests say so. |
| 34 | + |
| 35 | +So the cogs **do** run real trained inference and **do** disclose how weak it is. |
| 36 | +What the audit correctly found were not fabrications but **claim-surface |
| 37 | +overclaims** — four places where the surface said more than the weights deliver. |
| 38 | +This ADR tightens those four (A1–A4) and cites the already-correct subsystems as |
| 39 | +NO-ACTION positives. |
| 40 | + |
| 41 | +Grading vocabulary follows ADR-152 / ADR-158: |
| 42 | +- **MEASURED** — reproduced in this worktree, command + failing-on-old test recorded. |
| 43 | +- **DATA-GATED** — real code path present; honestly flagged where data/hardware is absent. |
| 44 | +- **NO-ACTION (already-SOTA)** — audited, found correct, cited as a positive. |
| 45 | +- **ACCEPTED-FUTURE** — deliberately deferred, nothing dropped. |
| 46 | + |
| 47 | +## Graded SOTA Landscape |
| 48 | + |
| 49 | +| Capability | Grade | Note | |
| 50 | +|------------|-------|------| |
| 51 | +| CSI person counting (`cog-person-count`) | **DATA-GATED** | Real Candle count head + Bayesian fusion; weights trained only on classes 0/1 (presence). Multi-occupant accuracy is genuinely unproven and is **not fabricated** — counts above the trained range are now flagged `low_confidence` and clamped. | |
| 52 | +| CSI pose estimation (`cog-pose-estimation`) | **DATA-GATED** | Real Candle encoder + 17-keypoint head; field accuracy honestly weak (PCK@50 = 18.5%, disclosed in the manifest). The default-install gate bug (A1) is fixed so it actually emits frames. | |
| 53 | +| Signed cog manifests (Ed25519 + SHA-256) | **NO-ACTION (already-SOTA)** | On-disk manifests are real, signed, SHA-pinned, and honest about accuracy. The CLI now emits them verbatim (A4). | |
| 54 | +| HA bridge (`cog-ha-matter`) MQTT + witness | **NO-ACTION (already-SOTA)** | Real Ed25519 hash-chain witness, mDNS, embedded broker. Matter commissioning is honestly deferred to v0.8 (TLS off, LAN-only) — description softened to stop claiming Matter (honest-absence). | |
| 55 | +| Drone-swarm MARL (`ruview-swarm`) | **DATA-GATED / honest** | `candle_ppo.rs` is real autodiff PPO; it is **untrained at runtime** (random init) by design — the swarm must be trained before deploy, which the code does not hide. | |
| 56 | +| ASTM F3411 Remote ID | **MEASURED (A3)** | Basic ID message is real; the Location/Vector message is honestly *not* implemented (NED metres are no longer mislabelled as WGS84 lat/lon). | |
| 57 | + |
| 58 | +## Decision — Fixes Landed (MEASURED) |
| 59 | + |
| 60 | +### §A1 Pose runtime emitted ZERO frames under default config (HIGH) |
| 61 | + |
| 62 | +**Overclaim (silent correctness bug):** `inference.rs` hardcoded |
| 63 | +`confidence: 0.185` for every inference, `config.rs default_min_confidence()` |
| 64 | +returned `0.3`, and `runtime.rs` gated emission on `confidence >= min_confidence`. |
| 65 | +A default install therefore **never emitted a single `pose.frame`** while |
| 66 | +`health` reported healthy — the cog *claimed* to be a running pose estimator but |
| 67 | +silently produced nothing. |
| 68 | + |
| 69 | +**Real fix:** `pose_v1` has **no confidence head** (the head emits 34 keypoint |
| 70 | +coordinates only), so a real per-frame confidence is genuinely unavailable. We |
| 71 | +took the disclosed "ok" path rather than silently lowering the threshold: |
| 72 | +- Introduced `inference::MODEL_TYPICAL_CONFIDENCE = 0.185` (the validation PCK@50) |
| 73 | + as the single published per-frame confidence, used by both `infer()` and the |
| 74 | + config default. |
| 75 | +- Pinned `default_min_confidence()` to `MODEL_TYPICAL_CONFIDENCE` so a default |
| 76 | + install clears its own gate and emits. |
| 77 | +- Documented the trade-off in the config field doc, the JSON schema |
| 78 | + (`default` 0.3 → 0.185, with a description), **and** added a `run.started` |
| 79 | + warning in `main.rs` that fires when an operator raises `min_confidence` above |
| 80 | + the model's typical confidence — so a deliberately-high threshold is loud, not |
| 81 | + silent. |
| 82 | + |
| 83 | +**Failing-on-old test:** `cog_pose_estimation` smoke |
| 84 | +`default_config_emits_frames_with_real_model` — parses a default config and |
| 85 | +asserts `min_confidence <= MODEL_TYPICAL_CONFIDENCE` (and, with the real model |
| 86 | +loaded, that `infer().confidence >= min_confidence`). **Proven to fail** on the |
| 87 | +old `default_min_confidence()=0.3`: |
| 88 | +`default min_confidence 0.3 exceeds model typical confidence 0.185 — a default |
| 89 | +install would emit zero pose.frame events`. |
| 90 | + |
| 91 | +**Grade: MEASURED.** |
| 92 | + |
| 93 | +### §A2 8-class count head on a 2-class-trained model (MEDIUM) |
| 94 | + |
| 95 | +**Overclaim:** `inference.rs COUNT_CLASSES = 8` with argmax over {0..7}, but |
| 96 | +`count_train_results.json` has support only for classes 0 and 1 (`per_class_accuracy` |
| 97 | +keys `"0"`/`"1"`). The model is a **presence detector**, not a calibrated |
| 98 | +multi-occupant counter; an argmax on classes 2..=7 is out-of-distribution, yet the |
| 99 | +cog would emit it as a confident headcount. The Cargo.toml billed it as a |
| 100 | +"learned multi-person counter." |
| 101 | + |
| 102 | +**Real fix (no network change — DATA-GATED, accuracy not fabricated):** |
| 103 | +- Added `inference::MAX_TRAINED_CLASS = 1`, plus `CountPrediction::is_low_confidence()` |
| 104 | + (argmax beyond the trained ceiling) and `clamped_count()` (report clamped to the |
| 105 | + trained range, raw argmax kept for audit). |
| 106 | +- `person.count` events now carry `low_confidence` + `raw_count`, and downgrade to |
| 107 | + `level: "warn"` when out-of-distribution; the reported `count` is clamped so we |
| 108 | + never emit a fabricated headcount the weights can't back. |
| 109 | +- `run.started` discloses `count_max_trained_class` and `count_classes`. |
| 110 | +- Cargo.toml description changed from "learned multi-person counter" to |
| 111 | + "presence detector + (data-gated) person count". |
| 112 | + |
| 113 | +**Failing-on-old test:** `cog_person_count` smoke |
| 114 | +`untrained_class_argmax_is_flagged_low_confidence` — a prediction whose argmax is |
| 115 | +class 5 is asserted `is_low_confidence() == true` and `clamped_count() == |
| 116 | +MAX_TRAINED_CLASS`; a class-1 prediction is asserted *not* flagged. Fails on old |
| 117 | +code (no such methods/flag existed). |
| 118 | + |
| 119 | +**Grade: MEASURED (mechanism); multi-occupant accuracy DATA-GATED.** |
| 120 | + |
| 121 | +### §A3 Remote ID broadcast NED metres as WGS84 lat/lon (MEDIUM — safety/compliance) |
| 122 | + |
| 123 | +**Overclaim (compliance hazard):** `security/remote_id.rs update()` stored |
| 124 | +`state.position.x/.y` (NED **metres**) into `drone_lat`/`drone_lon`, so the Remote |
| 125 | +ID broadcast would carry physically-impossible coordinates (e.g. "latitude = |
| 126 | +37.5 m"). The module doc claimed a "Basic ID + Location/Vector message," but only |
| 127 | +`encode_basic_id()` exists. |
| 128 | + |
| 129 | +**Real fix (honest naming — never broadcast impossible coordinates):** |
| 130 | +- Renamed `drone_lat`/`drone_lon` → `drone_north_m`/`drone_east_m` (NED metres |
| 131 | + relative to the operator/takeoff datum), with field docs stating they are *not* |
| 132 | + geodetic. `operator_lat`/`operator_lon` remain true WGS84 (from the operator's |
| 133 | + GNSS). |
| 134 | +- Corrected the module doc to claim **Basic ID only**; the Location/Vector encoder |
| 135 | + is explicitly deferred until a datum-anchored NED→WGS84 transform lands |
| 136 | + (ACCEPTED-FUTURE), rather than removing a real feature. |
| 137 | + |
| 138 | +**Failing-on-old test:** `security::remote_id::tests::test_ned_offset_stored_as_metres_not_latlon` |
| 139 | +— a 37.5 m north / −12.0 m east NED offset is asserted to land in |
| 140 | +`drone_north_m`/`drone_east_m`; the operator's real WGS84 fix stays in range. Fails |
| 141 | +on old code, where these values were stored into `drone_lat`/`drone_lon`. |
| 142 | + |
| 143 | +**Grade: MEASURED.** |
| 144 | + |
| 145 | +### §A4 Hollow CLI manifest (LOW) |
| 146 | + |
| 147 | +**Overclaim:** `cog-person-count main.rs cmd_manifest` emitted a null skeleton |
| 148 | +(`binary_sha256: null`, no training metadata), making the CLI look unsigned even |
| 149 | +though the **real signed manifest** existed at |
| 150 | +`cog/artifacts/manifests/x86_64/manifest.json`. |
| 151 | + |
| 152 | +**Real fix:** new `cog_person_count::manifest` module `include_str!`-embeds the |
| 153 | +real signed manifests (x86_64 + arm), selected by build target arch. |
| 154 | +`cmd_manifest` now parses-then-emits the embedded signed manifest — exactly the |
| 155 | +pattern `cog-pose-estimation`'s `manifest_roundtrips` test demonstrates. The CLI |
| 156 | +now reports the real `binary_sha256`, `weights_sha256`, Ed25519 signature, and |
| 157 | +honest `build_metadata` (`training_class1_accuracy = 0.343`). |
| 158 | + |
| 159 | +**Failing-on-old test:** `manifest::tests::embedded_manifest_has_non_null_binary_sha256` |
| 160 | +asserts a 64-hex-char `binary_sha256`; companions assert the embedded manifest is |
| 161 | +signed (`sig_algo == Ed25519`) and `id == COG_ID`. End-to-end verified: |
| 162 | +`cog-person-count manifest` prints `binary_sha256: |
| 163 | +051614ce6ba63df704fae848a67ad095df4bb88862fdff05ef3c0419cc8388b3`. |
| 164 | + |
| 165 | +**Grade: MEASURED.** |
| 166 | + |
| 167 | +### §A5 cog-ha-matter description claimed Matter before it exists (LOW — honest-labeling) |
| 168 | + |
| 169 | +**Overclaim:** the Cargo.toml description said "Home Assistant + Matter |
| 170 | +integration," but Matter commissioning is deferred to v0.8 (`TlsConfig::Off`, |
| 171 | +LAN-only, asserted by `runtime.rs tls_defaults_to_off_for_v1_lan_only`). |
| 172 | + |
| 173 | +**Real fix (no code change):** softened the description to "Home Assistant (MQTT) |
| 174 | +integration … LAN-only (no TLS); Matter Bridge commissioning is deferred to v0.8 |
| 175 | +and not yet implemented." Mirrors ADR-158 §6 honest-absence: state what isn't |
| 176 | +there rather than implying it is. |
| 177 | + |
| 178 | +**Grade: MEASURED (label).** |
| 179 | + |
| 180 | +## Negative Results (Confirmed — NO-ACTION positives) |
| 181 | + |
| 182 | +Audited and found genuinely correct; cited as positives, not edited: |
| 183 | + |
| 184 | +- **`cog-ha-matter` witness chain** (`witness.rs` / `witness_signing.rs`) — real |
| 185 | + Ed25519 hash-chained witness log. Already-SOTA. |
| 186 | +- **`cog-person-count` fusion** (`fusion.rs`) — real Bayesian product-of-experts |
| 187 | + multi-node fusion (Stoer-Wagner-bounded clip), not a heuristic. Already-SOTA. |
| 188 | +- **`ruview-swarm` PPO** (`marl/candle_ppo.rs`) — real Candle autodiff PPO with a |
| 189 | + genuine policy-gradient update; its `randn` uses (init, action sampling, |
| 190 | + exploration) are all legitimate, not fake-output substitutes. Untrained at |
| 191 | + runtime by design (the swarm must be trained before deploy), which the code |
| 192 | + does not hide. Already-SOTA / honest. |
| 193 | + |
| 194 | +## Deferred Backlog (Nothing Dropped) |
| 195 | + |
| 196 | +- **Multi-occupant count accuracy** — DATA-GATED on labelled multi-occupant CSI. |
| 197 | + The `low_confidence` flag + clamp (§A2) is the honest stand-in until then. |
| 198 | +- **Remote ID Location/Vector message** — ACCEPTED-FUTURE; requires a |
| 199 | + datum-anchored local-tangent-plane NED→WGS84 transform with an operator datum. |
| 200 | + Basic ID ships today. |
| 201 | +- **Matter Bridge commissioning** — ACCEPTED-FUTURE (v0.8); LAN-only MQTT ships today. |
| 202 | +- **Criterion benches** for cog inference latency and `mesh_guard` — ACCEPTED-FUTURE |
| 203 | + (cold-start timings are recorded in the manifests' `build_metadata`, not yet a |
| 204 | + regression bench). |
| 205 | +- **`wasm-edge` skill accuracy** — unvalidated; **now honestly labelled, not |
| 206 | + claimed** (done in ADR-160: medical/affect/security/exotic claim surfaces |
| 207 | + disclaimed, renamed, and feature-gated; per-skill accuracy remains DATA-GATED). |
| 208 | + |
| 209 | +## Consequences |
| 210 | + |
| 211 | +- A default pose-estimation install now actually emits `pose.frame` events; |
| 212 | + raising the threshold above the model's reach is a loud `run.started` warning, |
| 213 | + not a silent dropout. |
| 214 | +- A person-count reading on an untrained class is flagged `low_confidence`, |
| 215 | + clamped, and downgraded to `warn` — no fabricated headcounts. |
| 216 | +- The Remote ID broadcast can never carry physically-impossible coordinates; NED |
| 217 | + metres live in honestly-named metre fields. |
| 218 | +- `cog-person-count manifest` now reports the real signed manifest instead of a |
| 219 | + hollow null skeleton. |
| 220 | +- No cog Cargo.toml description claims a capability (multi-person counting, Matter) |
| 221 | + the code/weights don't yet deliver. |
| 222 | + |
| 223 | +## Reproduction (MEASURED) |
| 224 | + |
| 225 | +```bash |
| 226 | +cd v2 |
| 227 | +cargo test -p cog-person-count -p cog-pose-estimation -p cog-ha-matter -p ruview-swarm \ |
| 228 | + --no-default-features |
| 229 | +# ruview-swarm train path compiles (PPO autodiff) |
| 230 | +cargo check -p ruview-swarm --features train |
| 231 | +# A4 end-to-end — real signed manifest, non-null binary_sha256 |
| 232 | +cargo run -q -p cog-person-count --no-default-features -- manifest |
| 233 | +``` |
| 234 | + |
| 235 | +Result at time of writing (all 0 failed): |
| 236 | +- `cog-person-count` — **19 passed** (lib 10 incl. 3 manifest; smoke 9) |
| 237 | +- `cog-pose-estimation` — **8 passed** (smoke) |
| 238 | +- `cog-ha-matter` — **64 passed** (unchanged; description-only edit) |
| 239 | +- `ruview-swarm` — **117 passed** (default features); `--features train` compiles clean. |
| 240 | + |
| 241 | +Scope was limited to the four named crates. NO-ACTION positives (witness chain, |
| 242 | +fusion, PPO + randn audit) were verified by inspection and left untouched. |
0 commit comments