Skip to content

Commit 2b2fcef

Browse files
Oliver Baerclaude
andcommitted
chore: add drafts and memory letters
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c564257 commit 2b2fcef

13 files changed

Lines changed: 1697 additions & 0 deletions

.memory/letter_20260206_0001.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Letter to Myself (Session Handoff)
2+
3+
**Date:** 2026-02-06 02:30 CET
4+
5+
## 1. Executive Summary
6+
* **Goal:** Migrate Aurus Voice Intelligence from desktop-only to cross-platform mobile-ready architecture (Tauri v2 + Next.js 14), now entering Phase 5 (Yjs + WebRTC sync).
7+
* **Current Status:** Phases 1-6 complete and committed. Mobile build compatibility (rustls-tls, desktop-only global shortcut, Android scaffolding) committed as `d68738a`. Ready to begin Phase 5: Desktop <-> Mobile real-time sync.
8+
9+
## 2. The "Done" List (Context Anchor)
10+
11+
### This Session
12+
* Committed mobile build compatibility changes as `d68738a`:
13+
- `src-tauri/src/lib.rs` - Global shortcut plugin wrapped in `#[cfg(desktop)]`
14+
- `src-tauri/Cargo.toml` - Moved `tauri-plugin-global-shortcut` to desktop-only deps (`[target.'cfg(not(any(target_os = "ios", target_os = "android")))'.dependencies]`)
15+
- `src-tauri/Cargo.toml` - Switched `reqwest` from native-tls to `rustls-tls` (cross-platform)
16+
- `src-tauri/Cargo.toml` - Switched `async-tungstenite` from `tokio-native-tls` to `tokio-rustls-native-certs`
17+
- Added Android project scaffolding (`src-tauri/gen/android/`)
18+
- Added `src-tauri/.cargo/config.toml` for Android NDK cross-compilation targets
19+
- Added `.claudeignore`, `.claude/settings.json`, `claude.toml`
20+
* Verified: `cargo check` clean (0 warnings), 53/53 frontend tests passing
21+
22+
### Previous Sessions (already committed)
23+
* Phase 1: Platform abstraction layer (`src-tauri/src/platform/`), SecureStorage trait
24+
* Phase 2: secrets.rs rewritten to use platform abstraction
25+
* Phase 3: AudioCapture trait, CPAL desktop, mobile stub, useWebAudioCapture hook
26+
* Phase 4: ai-worker.ts (Transformers.js Web Worker), useLocalAI hook
27+
* Phase 6: 19 platform tests (`__tests__/platform.test.tsx`)
28+
* `2ce5e60` - Offline Whisper transcription fallback for mobile
29+
* `8b7a2c2` - Mobile recording integration, CI/CD pipeline, cleanup
30+
* `16ce596` - Cross-platform mobile architecture (Phases 1-6)
31+
32+
## 3. The "Pain" Log (CRITICAL)
33+
* **Tried:** Using `native-tls` features for reqwest/async-tungstenite on mobile targets
34+
* **Failed:** native-tls requires platform-specific OpenSSL or Security.framework linking that differs per mobile target
35+
* **Workaround:** Switched to `rustls-tls` / `tokio-rustls-native-certs` which is pure Rust and works on all platforms
36+
* *Note:* Always use rustls for cross-platform Tauri apps targeting mobile
37+
38+
## 4. Active Variable State
39+
* Working directory: `/Users/oliverbaer/Projects/aurus-voiceintelligence`
40+
* Build status: `cargo check` clean (0 warnings), `pnpm build` passes
41+
* Test status: 53/53 frontend (3 test files), 7/7 Rust tests
42+
* Git: `d68738a` on `main`, 4 commits ahead of origin (not pushed)
43+
* Unstaged: `.claude/settings.local.json`, `.memory copy/`, `drafts/`, `licenses/`, `ndk/`, `src-tauri/.github/`, `src-tauri/.memory/`
44+
* No Xcode (full app) installed - only Command Line Tools. No Android SDK fully configured.
45+
46+
## 5. Immediate Next Steps
47+
1. [ ] Phase 5: Design and implement Yjs + WebRTC sync architecture for Desktop <-> Mobile
48+
2. [ ] Wire `useLocalAI` hook into UI for on-device transcription fallback
49+
3. [ ] Install Xcode (full app) and run `pnpm tauri ios init` with Apple Team ID
50+
4. [ ] Test useWebAudioCapture in mobile simulator (requires Xcode)
51+
5. [ ] Create GitHub Actions workflow for mobile CI/CD
52+
6. [ ] Test Transformers.js worker initialization in browser dev mode

.memory/letter_20260206_0002.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Letter to Myself (Session Handoff)
2+
3+
**Date:** 2026-02-06 03:15 CET
4+
5+
## 1. Executive Summary
6+
* **Goal:** Design Phase 5 — Privacy-first ephemeral device sync for Aurus Voice Intelligence (Desktop <-> Mobile).
7+
* **Current Status:** Architecture plan complete and written to `/Users/oliverbaer/.claude/plans/declarative-baking-whistle.md`. Ready for user approval to begin implementation. Committed mobile build changes as `d68738a`.
8+
9+
## 2. The "Done" List (Context Anchor)
10+
11+
### This Session
12+
* Committed `d68738a` — mobile build compatibility (rustls-tls, desktop-only global shortcut, Android scaffolding)
13+
* Saved checkpoint `letter_20260206_0001.md`
14+
* Explored full codebase architecture via 3 subagents:
15+
- Zustand store data model (7 agents, streaming state, preferences)
16+
- Platform abstraction patterns (traits, cfg, platform-specific impls)
17+
- WebSocket/channel patterns from `transcription.rs`
18+
- Type definitions matching Rust ↔ TypeScript
19+
* Created comprehensive Gemini research prompt for privacy-first sync architecture
20+
* Received Gemini recommendations: webrtc-rs, yrs, SPAKE2+, double-layer E2E, in-memory only
21+
* Wrote full Phase 5 implementation plan with 4 sub-phases (5a/5b/5c/5d)
22+
23+
### Architecture Decisions Made (via Gemini security analysis)
24+
* **Transport:** Rust-native WebRTC (`webrtc-rs`) + mDNS local discovery
25+
* **Pairing:** Magic Wormhole / SPAKE2+ (short human-readable codes like "4-purple-castle")
26+
* **Encryption:** Double-layer E2E (WebRTC DTLS + application-layer via `ring` AES-256-GCM)
27+
* **CRDT:** `yrs` crate (Rust port of Yjs), in-memory Y.Doc only — zero persistence
28+
* **Signaling:** Opaque relay (encrypted blobs, server can't read content)
29+
* **Session lifecycle:** Dead man's switch, 4h max timeout, key rotation every 30min
30+
31+
### Plan File Location
32+
`/Users/oliverbaer/.claude/plans/declarative-baking-whistle.md`
33+
34+
## 3. The "Pain" Log (CRITICAL)
35+
* **Tried:** Initially explored JS-side Yjs + y-webrtc approach
36+
* **Pivoted:** Gemini recommended Rust-native approach (yrs + webrtc-rs) for security — keeps all sync logic in Rust where memory can be securely wiped
37+
* **Key insight:** "Don't store any data" constraint eliminates y-indexeddb, localStorage, and any persistence layer. In-memory yrs Y.Doc in Rust process is the only safe option.
38+
* **Risk identified:** Cross-network sync requires a signaling server. Even as opaque relay, it's an infrastructure dependency. Local-network-first approach mitigates this.
39+
* *Note:* webrtc-rs `v0.11` is tokio-native and matches existing async patterns. The newer `rtc` crate (Sans-IO) is more flexible but adds complexity — stick with `webrtc-rs` for MVP.
40+
41+
## 4. Active Variable State
42+
* Working directory: `/Users/oliverbaer/Projects/aurus-voiceintelligence`
43+
* Build status: `cargo check` clean (0 warnings), `pnpm build` passes
44+
* Test status: 53/53 frontend (3 test files), 7/7 Rust tests
45+
* Git: `d68738a` on `main`, 4 commits ahead of origin (not pushed)
46+
* Plan mode: WAS active, user requested checkpoint before approval
47+
* Plan file: `/Users/oliverbaer/.claude/plans/declarative-baking-whistle.md`
48+
49+
## 5. Immediate Next Steps
50+
1. [ ] **Approve plan** — re-enter plan mode, review `declarative-baking-whistle.md`, approve to begin implementation
51+
2. [ ] **Phase 5a:** Add yrs, uuid, ring, rand to Cargo.toml; create `src-tauri/src/sync/mod.rs`, `document.rs`, `encryption.rs`
52+
3. [ ] **Phase 5b:** Add mdns-sd, spake2; create `discovery.rs`, `transport.rs`, `pairing.rs`; build pairing UI
53+
4. [ ] **Phase 5d:** Dead man's switch, session timeout, forward secrecy
54+
5. [ ] **Phase 5c:** Add webrtc crate; create `webrtc.rs`, `signaling.rs`; deploy signaling relay
55+
56+
### New Cargo Dependencies Needed (7)
57+
```toml
58+
yrs = "0.21"
59+
uuid = { version = "1", features = ["v4", "serde"] }
60+
ring = "0.17"
61+
rand = "0.8"
62+
mdns-sd = "0.11"
63+
spake2 = "0.4"
64+
webrtc = "0.11" # Phase 5c only
65+
```
66+
67+
### New Files to Create (11)
68+
**Rust (8):** sync/mod.rs, document.rs, encryption.rs, discovery.rs, transport.rs, pairing.rs, webrtc.rs, signaling.rs
69+
**Frontend (3):** hooks/useSync.ts, components/SyncPairing.tsx, components/SyncStatus.tsx

.memory/letter_20260206_0003.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Letter to Myself (Session Handoff)
2+
3+
**Date:** 2026-02-06 03:20 CET
4+
5+
## 1. Executive Summary
6+
* **Goal:** Design and begin implementing Phase 5 — Privacy-first ephemeral device sync for Aurus Voice Intelligence (Desktop <-> Mobile).
7+
* **Current Status:** Plan approved by user. Ready to begin Phase 5a implementation (yrs CRDT core + encryption scaffolding). No code written yet for sync — next session starts building.
8+
9+
## 2. The "Done" List (Context Anchor)
10+
11+
### This Session
12+
* Committed `d68738a` — mobile build compatibility:
13+
- `src-tauri/src/lib.rs`: Global shortcut plugin wrapped in `#[cfg(desktop)]`
14+
- `src-tauri/Cargo.toml`: Moved `tauri-plugin-global-shortcut` to desktop-only deps, switched reqwest to `rustls-tls`, async-tungstenite to `tokio-rustls-native-certs`
15+
- Added Android project scaffolding (`src-tauri/gen/android/`)
16+
- Added `.claudeignore`, `.claude/settings.json`, `claude.toml`, `src-tauri/.cargo/config.toml`
17+
* Explored codebase via subagents (voiceStore data model, platform traits, transcription.rs patterns, TypeScript types)
18+
* Created comprehensive Gemini research prompt for privacy-first sync security architecture
19+
* Received and synthesized Gemini security recommendations
20+
* Wrote full Phase 5 implementation plan at `/Users/oliverbaer/.claude/plans/declarative-baking-whistle.md`
21+
* Plan approved — ready for implementation
22+
23+
### Architecture Decisions (Finalized)
24+
* **Transport:** Rust-native WebRTC (`webrtc-rs` v0.11) + mDNS local discovery (`mdns-sd`)
25+
* **Pairing:** SPAKE2+ via `spake2` crate (short human-readable codes like "4-purple-castle")
26+
* **Encryption:** Double-layer E2E — WebRTC DTLS (transport) + `ring` AES-256-GCM (application layer)
27+
* **CRDT:** `yrs` v0.21 (Rust Yjs), in-memory Y.Doc only — ZERO persistence
28+
* **Signaling:** Opaque relay (encrypted blobs only, server can't read content)
29+
* **Session lifecycle:** Dead man's switch (15s timeout), 4h max session, key rotation every 30min
30+
31+
## 3. The "Pain" Log (CRITICAL)
32+
* **Tried:** Initially explored JS-side Yjs + y-webrtc approach
33+
* **Pivoted:** Gemini recommended Rust-native approach (yrs + webrtc-rs) — keeps all sync logic in Rust where memory can be securely wiped on Drop
34+
* **Key insight:** "Don't store any data" eliminates y-indexeddb, localStorage, any persistence. In-memory yrs Y.Doc in Rust is the only safe option
35+
* **Risk:** Cross-network sync needs a signaling server (infrastructure dependency). Mitigated by local-network-first approach
36+
* *Note:* Use `webrtc-rs` v0.11 (tokio-native), NOT the newer `rtc` crate (Sans-IO adds unnecessary complexity for MVP)
37+
38+
## 4. Active Variable State
39+
* Working directory: `/Users/oliverbaer/Projects/aurus-voiceintelligence`
40+
* Build: `cargo check` clean (0 warnings), 53/53 frontend tests, 7/7 Rust tests
41+
* Git: `d68738a` on `main`, 4 commits ahead of origin (not pushed)
42+
* Plan file: `/Users/oliverbaer/.claude/plans/declarative-baking-whistle.md` (APPROVED)
43+
44+
## 5. Immediate Next Steps
45+
1. [ ] **Phase 5a — Step 1:** Add `yrs`, `uuid`, `ring`, `rand` to `src-tauri/Cargo.toml` and verify `cargo check`
46+
2. [ ] **Phase 5a — Step 2:** Create `src-tauri/src/sync/mod.rs` — SyncManager, SyncState, SyncError, Tauri commands (create_sync_session, join_sync_session, leave_sync_session, get_sync_status, get_pairing_code)
47+
3. [ ] **Phase 5a — Step 3:** Create `src-tauri/src/sync/document.rs` — yrs Y.Doc setup mapping voiceStore structure (session map, agent result maps, preferences map), observe/apply update functions
48+
4. [ ] **Phase 5a — Step 4:** Create `src-tauri/src/sync/encryption.rs` — ECDH key agreement, AES-256-GCM encrypt/decrypt for yrs update vectors, nonce management
49+
5. [ ] **Phase 5a — Step 5:** Register sync module in `src-tauri/src/lib.rs` (SyncManager state + invoke_handler commands)
50+
6. [ ] **Phase 5a — Step 6:** Create frontend hooks/components — `app/hooks/useSync.ts`, `app/components/SyncStatus.tsx`, add sync fields to `app/store/voiceStore.ts`
51+
7. [ ] **Phase 5a — Step 7:** Write unit tests for document mapping + encryption round-trip

.memory/letter_20260206_0004.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Letter to Myself (Session Handoff)
2+
3+
**Date:** 2026-02-06 03:45 CET
4+
5+
## 1. Executive Summary
6+
* **Goal:** Implement Phase 5a — Core sync infrastructure (yrs CRDT, E2E encryption, Tauri commands, frontend state).
7+
* **Current Status:** Phase 5a COMPLETE. All code written, 24/24 Rust tests passing (17 new sync tests), 53/53 frontend tests passing. Ready to commit and begin Phase 5b (local network transport).
8+
9+
## 2. The "Done" List (Context Anchor)
10+
11+
### Phase 5a Files Created (Rust)
12+
* `src-tauri/src/sync/mod.rs` — SyncManager (Arc<Mutex<SyncState>>), SyncState, SyncError enum, SyncStatus enum, PeerInfo struct, 5 Tauri commands (create_sync_session, join_sync_session, leave_sync_session, get_sync_status, get_pairing_code), generate_pairing_code() with word lists, Drop impl that zeros sensitive data
13+
* `src-tauri/src/sync/document.rs` — SyncDocument wrapping yrs::Doc, Y.Map for each voiceStore section (session, 7 agents, preferences), typed helpers (set_transcript, set_agent_result, get_*/set_* for all fields), encode_state_as_update/apply_update/encode_diff for CRDT sync, snapshot() for emitting full state to frontend, extract_string helper using yrs::Out::Any(Any::String(...))
14+
* `src-tauri/src/sync/encryption.rs` — SessionEncryption with AES-256-GCM via ring crate, HKDF-SHA256 key derivation from shared secret, counter-based nonces (96-bit), EncryptedEnvelope struct, encrypt/decrypt methods, rotate_key() with HKDF ratchet, Drop impl that zeros all key material
15+
16+
### Phase 5a Files Modified (Rust)
17+
* `src-tauri/Cargo.toml` — Added yrs = "0.21", uuid (v4, serde), ring = "0.17", rand = "0.8"
18+
* `src-tauri/src/lib.rs` — Added `pub mod sync;`, SyncManager state initialization in setup(), 5 sync commands in invoke_handler
19+
20+
### Phase 5a Files Created (Frontend)
21+
* `app/hooks/useSync.ts` — Listens to sync-status-changed and sync-state-updated Tauri events, applies remote state to voiceStore, exposes createSession/joinSession/leaveSession actions
22+
* `app/components/SyncStatus.tsx` — Connection indicator component (shows pairing code, connected device name, status dot)
23+
24+
### Phase 5a Files Modified (Frontend)
25+
* `app/store/voiceStore.ts` — Added SyncStatus type, PeerInfo interface, sync fields (syncStatus, pairingCode, pairedDeviceName, syncPeer), sync setters
26+
27+
### Test Results
28+
* Rust: 24/24 (7 existing + 17 new: 3 sync/mod, 7 sync/document, 7 sync/encryption)
29+
* Frontend: 53/53 (unchanged, all existing tests still pass)
30+
31+
## 3. The "Pain" Log (CRITICAL)
32+
* **Tried:** Using `yrs::Value` enum for extracting string values from Y.Map
33+
* **Failed:** `Value` is deprecated in yrs 0.21 — compiler warns "Use `yrs::Out` instead"
34+
* **Fix:** Changed to `Out::Any(Any::String(s))` pattern matching
35+
* **Tried:** Using `txn.get_map()` for read-only access to maps
36+
* **Failed:** `get_map()` doesn't exist on `Transaction` (ReadTxn) in yrs 0.21
37+
* **Fix:** Use `transact_mut()` everywhere and `txn.get_or_insert_map()` (requires `WriteTxn` trait import). MapRef is Copy so the mutable borrow is released immediately.
38+
* **Tried:** Using `v.to_string(&txn)` on Value/Out
39+
* **Failed:** `to_string` doesn't exist as a method on Out — it's not Display-like
40+
* **Fix:** Pattern match on `Out::Any(Any::String(s))` directly
41+
* *Note:* Always import `yrs::WriteTxn` when working with yrs 0.21. Use `Out` not `Value`. Use `transact_mut()` for both reads and writes when accessing named maps.
42+
43+
## 4. Active Variable State
44+
* Working directory: `/Users/oliverbaer/Projects/aurus-voiceintelligence`
45+
* Build: `cargo check` clean (0 warnings), `pnpm build` passes
46+
* Tests: 24/24 Rust, 53/53 frontend
47+
* Git: `d68738a` on `main`, 4 commits ahead of origin + uncommitted Phase 5a changes
48+
* Uncommitted files: sync/mod.rs, sync/document.rs, sync/encryption.rs, lib.rs, Cargo.toml, Cargo.lock, voiceStore.ts, useSync.ts, SyncStatus.tsx
49+
50+
## 5. Immediate Next Steps
51+
1. [ ] Commit Phase 5a changes
52+
2. [ ] Phase 5b: Add `mdns-sd` and `spake2` to Cargo.toml
53+
3. [ ] Phase 5b: Create `sync/discovery.rs` — mDNS service announcement + browsing (_aurus-sync._tcp.local)
54+
4. [ ] Phase 5b: Create `sync/transport.rs` — SyncTransport trait + LocalTransport (WebSocket on LAN)
55+
5. [ ] Phase 5b: Create `sync/pairing.rs` — SPAKE2+ key exchange with human-readable codes
56+
6. [ ] Phase 5b: Create `app/components/SyncPairing.tsx` — Pairing UI (code display + input)
57+
7. [ ] Phase 5b: Wire transport to SyncDocument — send yrs updates through encrypted channel

.memory/letter_20260206_0006.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Letter to Myself (Session Handoff)
2+
3+
**Date:** 2026-02-06 ~08:35 UTC
4+
5+
## 1. Executive Summary
6+
* **Goal:** Implement Phase 5d — security hardening for the sync transport (heartbeat, session timeout, forward secrecy)
7+
* **Current Status:** Phase 5d complete and committed as `140b81a`. All 34 Rust + 53 frontend tests passing. Next: Phase 5c (cross-network WebRTC with signaling relay).
8+
9+
## 2. The "Done" List (Context Anchor)
10+
* Rewrote `src-tauri/src/sync/transport.rs` with security hardening:
11+
- Added constants: `HEARTBEAT_INTERVAL` (5s), `PEER_TIMEOUT` (15s), `SESSION_MAX_DURATION` (4h), `SESSION_WARNING_BEFORE` (15m), `KEY_ROTATION_INTERVAL` (30m)
12+
- Added `KeyRotate { epoch: u64 }` variant to `SyncMessage` enum
13+
- Changed `run_sync_loop` encryption param from `Arc<SessionEncryption>` to `mut SessionEncryption` (owned, mutable for rotation)
14+
- Dead man's switch: tracks `last_peer_activity`, disconnects if > 15s silent
15+
- Session timeout: 4h max, emits `sync-session-warning` at 3h45m, `sync-session-timeout` at 4h
16+
- Forward secrecy: HKDF key ratchet every 30 minutes, `KeyRotate` message sent before rotation (WebSocket ordering guarantees)
17+
- Added `send_msg` helper for cleaner message sending
18+
- 2 new tests: `test_key_rotate_serialization`, `test_constants_sanity` (6 total transport tests)
19+
* Updated `app/hooks/useSync.ts` with 3 new security event listeners:
20+
- `sync-heartbeat-timeout` — resets sync state on peer timeout
21+
- `sync-session-warning` — stores warning message in store
22+
- `sync-session-timeout` — resets sync state on session expiry
23+
* Updated `app/store/voiceStore.ts`:
24+
- Added `syncWarning: string | null` field + `setSyncWarning` setter
25+
26+
## 3. The "Pain" Log (CRITICAL)
27+
* **Warning:** `let mut encryption = creator.finish(...)` had unused `mut` — the binding doesn't need `mut` because `run_sync_loop` declares its parameter as `mut encryption: SessionEncryption`. Removed the `mut` on the binding.
28+
* No major issues in Phase 5d — the security code compiled cleanly on first attempt.
29+
30+
## 4. Active Variable State
31+
* Git: Phase 5d committed as `140b81a` on main. Branch is 7 ahead of origin.
32+
* Test counts: 34 Rust, 53 frontend = 87 total
33+
* Commits: 5a (`031c7df`) → 5b (`be74ebd`) → 5d (`140b81a`)
34+
* Key rotation protocol: sender sends `KeyRotate { epoch }` then calls `encryption.rotate_key()`. Receiver receives `KeyRotate`, checks `epoch > key_rotation_epoch`, then rotates. Both arrive at same key via deterministic HKDF ratchet.
35+
36+
## 5. Immediate Next Steps
37+
1. [ ] Phase 5c: Cross-network WebRTC with signaling relay
38+
- [ ] `src-tauri/src/sync/signaling.rs` — opaque relay client (WebSocket to signaling server)
39+
- [ ] `src-tauri/src/sync/webrtc.rs` — WebRTC data channel transport
40+
- [ ] Transport auto-selection: try mDNS local first, fall back to WebRTC
41+
- [ ] Lightweight signaling relay server (Node.js or Rust)
42+
2. [ ] Push commits to origin when ready
43+
3. [ ] Manual integration test: pair two devices on same WiFi

0 commit comments

Comments
 (0)