@@ -208,8 +208,10 @@ Packet framing: `[SYNC:4][TYPE:1][LEN:4 LE][PAYLOAD]`
208208| ---| ---| ---|
209209| ` TYPE_WAL ` | ` 0x03 ` | host → device |
210210| ` TYPE_SEARCH ` | ` 0x04 ` | host → device |
211+ | ` TYPE_INFER ` | ` 0x06 ` | host → device |
211212| ` TYPE_PROOF ` | ` 0x01 ` | device → host |
212213| ` TYPE_SEARCH_RESULT ` | ` 0x05 ` | device → host |
214+ | ` TYPE_INFER_RESULT ` | ` 0x07 ` | device → host |
213215| ` TYPE_ERR ` | ` 0xEE ` | device → host |
214216| Sync word | ` 0x55 0xAA 0x55 0xAA ` | both directions |
215217
@@ -247,6 +249,61 @@ Total: `3 + 128×4 = 515 bytes`
247249[state_hash: 32 bytes] BLAKE3 — verify against /v1/proof
248250```
249251
252+ ### Inference packet payload (host → device, TYPE_INFER)
253+
254+ ```
255+ [gen_len: u8] number of tokens to generate (1–32)
256+ [prompt_len: u8] number of prompt token IDs
257+ [tokens: prompt_len × u8] token IDs in [0, VOCAB)
258+ ```
259+
260+ ### Inference result payload (device → host, TYPE_INFER_RESULT)
261+
262+ ```
263+ [ok: u8] 1 = success, 0 = error
264+ [out_len: u8] number of generated tokens
265+ [tokens: out_len × u8]
266+ [receipt: 32 bytes] BLAKE3(model_hash | prompt | output)
267+ [record_id: u32 LE] RecordId assigned in KernelState
268+ [version: u64 LE] KernelState version after insert
269+ [state_hash: 32 bytes] Valori BLAKE3 state hash after insert
270+ ```
271+
272+ The ` receipt ` is computed as ` BLAKE3(model_hash || prompt_tokens || output_tokens) ` .
273+ A verifier can replay the same prompt through the same model binary, recompute the
274+ receipt, and confirm it appears at this position in the Valori audit chain.
275+
276+ The ` state_hash ` returned here matches the cloud node's ` /v1/proof ` hash (assuming
277+ both have received the same event log), enabling end-to-end proof across devices.
278+
279+ ---
280+
281+ ## On-device RAG (inference.rs)
282+
283+ ` inference.rs ` wires INT's ` QGPTModel ` into Valori's audit chain:
284+
285+ 1 . ** Model baked into flash** — ` tiny_transformer_int8.bin ` is embedded via ` include_bytes! ` .
286+ The model BLAKE3 hash becomes part of every inference receipt.
287+ 2 . ** Greedy decode** — prefill + autoregressive decode, all Q8 integer math, no f32 on the hot path.
288+ 3 . ** Logit fingerprint stored in Valori** — the final-step logit distribution is converted to
289+ Q16.16 ` FxpVector ` and inserted into ` KernelState ` via ` KernelEvent::InsertRecord ` .
290+ The 32-byte BLAKE3 receipt goes into the ` metadata ` field.
291+ 4 . ** On-device RAG** — ` handle_with_rag() ` searches Valori for K nearest past inferences,
292+ prepends their tokens as context, then runs inference. The MCU retrieves from its own
293+ memory — no server needed.
294+
295+ ** Memory budget (STM32F407, 192 KB RAM):**
296+ ```
297+ QGPTModel<61,64,64,256,4,3> ≈ 172 KB (heap)
298+ KV caches ≈ 24 KB (stack per request)
299+ KernelState + WAL buffers ≈ 16 KB
300+ ─────────────────────────────────────
301+ Total peak ≈ 212 KB ← fits on STM32F407 with HEAP_MEM tuned
302+ ```
303+
304+ For RP2040 (264 KB) or nRF52840 (256 KB) this fits comfortably. For a smaller model
305+ (2 layers, DIM=32), RAM footprint drops to ~ 56 KB, leaving plenty for the vector store.
306+
250307---
251308
252309## QEMU smoke test
@@ -295,6 +352,7 @@ cargo test -p valori-embedded
295352| ` src/recovery.rs ` | Boot recovery: checkpoint → hash verify → snapshot restore |
296353| ` src/proof.rs ` | ` EmbeddedProof ` — ` snapshot_hash ` + ` kernel_state_hash ` → hex JSON |
297354| ` src/search.rs ` | Parse search request, call ` search_l2_ns ` , emit verifiable result |
355+ | ` src/inference.rs ` | INT ` QGPTModel ` integration — greedy decode, BLAKE3 receipt, on-device RAG |
298356| ` tests/cross_platform_hash.rs ` | Host-side CI tests proving determinism claim |
299357| ` scripts/qemu_test.sh ` | QEMU build + smoke test script |
300358
0 commit comments