Skip to content

Commit 6ec6e70

Browse files
intendednullclaude
andauthored
docs(specs): add verified Prior Art sections to 17 specs + coverage survey (#667)
Only 1 of 29 specs carried the `## Prior Art` table convention (the per-author Merkle-DAG spec). This adds one to the 17 specs whose design decisions have rich, citable external lineage, matching that format and placement (after the problem/goals framing, before the design body). Each section pairs a real external work with what Willow borrowed or how it diverged. Every citation was adversarially fact-checked: 148 external identifiers (RFC/NIP/MSC/BIP/BEP/XEP/CIP numbers, paper authors/years, library facts) web-verified, and 188 Willow-side claims checked against the code. All issues found inside the added sections were fixed — notably a fabricated type name (JoinLinkPayload → JoinToken), an overclaim about iroh types (BlobHash wrapper), a wrong dag.rs line range, and a false Matrix v1.10 attribution. The remaining 12 specs are process/testing/docs or purely-internal with no external lineage worth citing; see the survey report for the full breakdown and review outcome. Adds docs/reports/2026-05-29-prior-art-coverage-survey.md and indexes it in docs/README.md. The report's "Out of scope" note tracks ~14 pre-existing stale file:line refs in three spec bodies as a separate doc-hygiene follow-up. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent b3a3b68 commit 6ec6e70

19 files changed

Lines changed: 342 additions & 9 deletions

docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,7 @@ See also: [`plans/STATUS.md`](plans/STATUS.md) — point-in-time audit of which
173173
**Reports**
174174

175175
- [Test audit](reports/2026-04-13-test-audit.md) — audit of 14 crates finding coverage gaps at client, relay, and UI/state bridge layers. `[active]`
176+
- [Prior-art coverage survey](reports/2026-05-29-prior-art-coverage-survey.md) — surveyed all 29 specs, added verified `## Prior Art` sections to 17 of them; records the adversarial citation review. `[landed]`
176177

177178
### Process & Tooling
178179

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Prior-Art Coverage Survey — Willow Specs
2+
**Date:** 2026-05-29
3+
**Scope:** All 29 design specs in `docs/specs/`
4+
**Method:** Per-spec survey of design decisions and external lineage, followed by adversarial citation fact-checking (web-verified identifiers + code-verified Willow-side claims)
5+
6+
---
7+
8+
## Summary
9+
10+
Willow's prior-art convention is a `## Prior Art` table — `| System | Key idea adopted / how Willow diverges |` — tying each design decision to the external work it draws from or rejects. Before this survey only **1 of 29 specs** (the per-author Merkle-DAG spec) carried one.
11+
12+
This survey scored every spec, identified **17 strong candidates** whose decisions have rich, citable external lineage, and added a verified `## Prior Art` section to each. The remaining 12 are process/testing/docs or purely-internal specs with no meaningful external prior art (listed below).
13+
14+
After drafting, all 17 sections went through an adversarial review: **148 external citations** web-verified (RFC/NIP/MSC/BIP/BEP/XEP/CIP numbers, paper authors/venues/years, library facts) and **188 Willow-side claims** checked against the worktree code. Findings — 5 major, 24 minor, 4 nit — were triaged; every issue *inside an added Prior Art section* was fixed before merge (see "Review outcome").
15+
16+
## Specs that received a Prior Art section
17+
18+
| Spec | Anchor topic of the prior art |
19+
|---|---|
20+
| `2026-03-24-async-client-ui-refactor-design.md` | TEA / Redux-Flux / CQRS / actor-handle / SolidJS signals |
21+
| `2026-03-26-screen-sharing-call-page-design.md` | WebRTC, perfect negotiation, full-mesh vs SFU (Jitsi/LiveKit/Element Call) |
22+
| `2026-03-27-shareable-join-links-design.md` | matrix.to locators, NIP-21/19, Magic Wormhole, capability URLs |
23+
| `2026-03-27-worker-nodes-design.md` | Nostr relays, SSB pubs, IPFS pinning, Dynamo anti-entropy |
24+
| `2026-03-29-agentic-peer-api-design.md` | MCP, JSON-RPC, Matrix appservices, OAuth scopes, least privilege |
25+
| `2026-03-29-iroh-migration-design.md` | iroh/n0, libp2p, HyParView+Plumtree, QUIC, BLAKE3, pkarr |
26+
| `2026-03-31-actor-system-library-design.md` | Actor model (Hewitt 1973), Erlang/OTP, actix/ractor/kameo/xtra |
27+
| `2026-04-12-state-authority-and-mutations.md` | Object-capabilities, Macaroons, SPKI/SDSI, Certificate Transparency |
28+
| `2026-04-24-bech32-identifiers.md` | BIP-173/350, NIP-19, Base58Check, multiformats, StrKey, CIP-19 |
29+
| `2026-04-24-epoch-key-rotation.md` | MLS/TreeKEM, HPKE, HKDF, Double Ratchet, Sender Keys, Megolm |
30+
| `2026-04-24-error-prefixes.md` | NIP-01/42, Matrix errors, gRPC status, HTTP 429, WebSocket close codes |
31+
| `2026-04-24-history-sync-eose.md` | Nostr EOSE, Matrix /sync, WebDAV sync-token, IMAP CONDSTORE, git fetch |
32+
| `2026-04-24-negentropy-sync.md` | Negentropy/RBSR, SSB, Automerge, git pack, version-vector anti-entropy |
33+
| `2026-04-24-outbox-relay-discovery.md` | NIP-65, pkarr/BEP44, Matrix well-known, RFC 8615, did:plc |
34+
| `2026-04-24-relay-capability-doc.md` | NIP-11, Matrix versions/capabilities, XEP-0115/0390, NodeInfo, OIDC discovery |
35+
| `2026-04-25-llm-agent-ux-spec-design.md` | Discord/Slack/Matrix bots, MCP, SSE streaming, protobuf/Cap'n Proto evolution |
36+
| `2026-04-26-state-management-model-design.md` | Actor model, CSP/Hoare, Erlang gen_server, Akka, Ryhl "Actors with Tokio" |
37+
38+
## Not candidates (no Prior Art section added)
39+
40+
Process, testing, docs-infrastructure, and purely-internal specs with no external lineage worth citing:
41+
42+
- `2026-03-24-multi-peer-e2e-tests-design.md` — internal test harness design.
43+
- `2026-03-25-ux-navigation-improvements-design.md` — product UX, no external protocol lineage.
44+
- `2026-04-01-per-author-merkle-dag-state-design.md`**already has** a Prior Art section (the convention's origin).
45+
- `2026-04-12-willow-channel-removal.md` — internal refactor (crate deletion + type consolidation).
46+
- `2026-04-13-test-architecture.md` — internal test-tier policy.
47+
- `2026-04-21-e2e-test-architecture-design.md` — internal test-tier policy.
48+
- `2026-04-27-event-based-waits-design.md` — internal async-test helper.
49+
- `2026-05-07-docs-organization-design.md` — this docs system itself.
50+
- `2026-05-21-pinned-message-metadata-design.md` — small internal metadata addition.
51+
52+
## Review outcome
53+
54+
The adversarial pass confirmed the citations are overwhelmingly accurate. Issues found and resolved in the added sections:
55+
56+
**Major (4 in-scope, all fixed):**
57+
- `shareable-join-links`: fabricated type name `JoinLinkPayload` → corrected to the real `JoinToken` (`crates/client/src/ops.rs:28`).
58+
- `iroh-migration`: overclaimed "no Willow-invented abstraction; uses iroh's `Hash`" → corrected to note the `BlobHash` wrapper (`crates/network/src/traits.rs:14-20`, needed because `iroh_blobs` is not WASM-compatible) and that `Bytes` is `bytes::Bytes`.
59+
- `negentropy-sync`: wrong code ref `dag.rs:146-158``dag.rs:197-218` (the actual seq + prev-hash checks); matching body refs corrected for consistency.
60+
- `relay-capability-doc`: false "v1.10" attribution for Matrix's `/versions` vs `/capabilities` split (both endpoints exist since r0.5.0, ~2019) → corrected in both the table row and the Future-work bullet (the latter was a pre-existing error in the spec body).
61+
62+
**Minor / nit (in-scope, fixed):** Discord expiry floor (30 min, not 1h); SSB pubs-vs-rooms conflation; Meyer RBSR date (arXiv 2212.13567, Dec 2022 / SRDS 2023); Akka "Lightbend" company-vs-framework wording; kameo "supervision" qualified (Willow keeps single-actor `RestartPolicy`); MLS forward-secrecy ("full FS", reserving "weak FS" for Willow); TreeKEM "key schedule" → "ratchet tree" (RFC 9420 §7); SSB Pubs "superseded" → "complemented" by Rooms; CIP-19 HRPs defined in CIP-5; SSB-EBT authorship disambiguation.
63+
64+
**Out of scope (not fixed here):** The review also flagged ~14 stale `file:line` references in the **bodies** of `history-sync-eose`, `relay-capability-doc`, and `llm-agent-ux-spec` (code drifted since those specs were written). These predate this work and are tracked as a follow-up doc-hygiene pass, not folded into this prior-art change.

docs/specs/2026-03-24-async-client-ui-refactor-design.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,22 @@ Additionally, the Leptos `App` component is an 800-line monolith that creates 30
1717
- **Out of scope:** Bevy native app (`crates/app/`). The refactored Client API will break the Bevy integration. This is acceptable since Bevy is disabled. It can be re-adapted later if needed.
1818
- **Out of scope:** Yew frontend. This design makes the client framework-agnostic, which will make a future Yew frontend straightforward, but building it is not part of this work.
1919

20+
## Prior Art
21+
22+
This refactor adapts established UI and concurrency architecture patterns to a single-threaded WASM client:
23+
24+
| Prior art | Relevance to this design |
25+
|---|---|
26+
| **The Elm Architecture (TEA)** (Evan Czaplicki, Elm; 2012) | Model/Update/View with strict unidirectional flow: messages drive state, state drives the view. Mirrored here by `NetworkEvent` -> `ClientEventLoop` -> `ClientEvent` -> signal updater -> Leptos view, with no view-to-state back-channel. |
27+
| **Redux / Flux** (Dan Abramov & Andrew Clark, 2015; Facebook Flux, 2014) | A read-only store mutated only via dispatched actions through one processing path. Maps to the read/write signal split: `ReadSignal` halves live in `AppState` context (read-only to components); `WriteSignal` halves (`AppWriteSignals`) are held solely by the event-processing layer. |
28+
| **CQRS** (Greg Young, ~2010; building on Bertrand Meyer's Command-Query Separation, Eiffel) | Separate the command/write model from the read/query model. Mirrored by `ClientHandle` (cloneable command + read interface, optimistic local writes) vs. the single non-cloneable `ClientEventLoop` (exclusive async event processor), and again by the `ReadSignal`/`WriteSignal` split. |
29+
| **Actor handle/inbox pattern** (actix `Addr<A>`; ractor; kameo) | A cheaply-cloneable address that enqueues messages to exactly one owning message loop. Directly parallels the cloneable `ClientHandle` (sends `NetworkCommand`s over an `UnboundedSender`) plus the single owning `ClientEventLoop` that drains the inbox and is the only writer of shared state. |
30+
| **iced / relm4** (Elm-inspired Rust GUIs; iced on `futures`, relm4 on gtk4-rs) | Demonstrate TEA's `Message`/`update`/`view` in Rust; iced's first-class async actions turn `futures` into the message source — the substitute for polling that this design also adopts (`select!` over `futures::channel::mpsc` instead of a 16ms/50ms interval poll). |
31+
| **SolidJS fine-grained signals** (the acknowledged inspiration for Leptos) | `createSignal` returns a `[getter, setter]` tuple, making read/write segregation first-class and enabling targeted DOM updates with no virtual DOM. This design leans on the same split (`ReadSignal`/`WriteSignal`) and propagates the read halves via `provide_context` rather than threading 30 signals as props. |
32+
| **Stream `select!` event loops** (`futures`, `tokio`) | Backpressure-free async channels (`futures::channel::mpsc::unbounded`) consumed via `select!` replace timer-driven `std::sync::mpsc` polling, so commands and events are delivered the instant they are produced. |
33+
34+
Out of scope but informed by the same lineage: **Yew / Seed** (Elm-style Rust/WASM frontends) are named future targets — the framework-agnostic `ClientHandle`/`ClientEventLoop` split is intended to make such a frontend straightforward later.
35+
2036
## Design
2137

2238
### 1. Async Channels

docs/specs/2026-03-26-screen-sharing-call-page-design.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,21 @@ Willow's voice chat is audio-only with minimal UI — just mute/deafen/disconnec
7171
- **In scope:** Call page (main view), camera video, screen sharing via WebRTC, speaking detection, grid/focus layout switching, participant tiles with video and speaking indicators.
7272
- **Out of scope:** SFU/MCU topology changes (stays full mesh), recording, virtual backgrounds, simultaneous camera + screen share (mutually exclusive — one video track per direction).
7373

74+
## Prior Art
75+
76+
This design builds on WebRTC standards and on the topologies and patterns proven by existing conferencing systems:
77+
78+
| System / Spec | Relevance to Willow |
79+
|---|---|
80+
| **WebRTC: Real-Time Communication in Browsers** (W3C Recommendation, 2021-01-26; re-published 2024-10-08) | Defines the `RTCPeerConnection` API Willow renegotiates over: `addTrack()`/`removeTrack()` on a live connection fires `negotiationneeded` and triggers a fresh offer/answer round. Willow adopts this in-place renegotiation model rather than tearing down and rebuilding peer connections when a peer starts/stops video. |
81+
| **WebRTC "Perfect Negotiation" pattern** (W3C `webrtc-pc` example added by Jan-Ivar Bruaroey, 2019; documented on MDN / webrtc.org) | Source of Willow's glare handling: polite/impolite roles plus a `makingOffer` flag and `setLocalDescription({type:"rollback"})` to resolve simultaneous offers. Willow assigns roles deterministically by lexicographic peer-ID comparison (lower ID = polite) instead of "first to connect," and — per the realised state — relies on the `making_offer` flag alone rather than polling `signalingState`. |
82+
| **W3C Media Capture and Streams** (`getUserMedia`, CR Draft) and **W3C Screen Capture** (`getDisplayMedia`, WD) | Define camera/mic and screen acquisition. Willow feeds both into a single outbound video track, switching the active source (camera ↔ screen) rather than capturing both at once — and must call `getDisplayMedia()` synchronously in the click handler to preserve transient activation. |
83+
| **Jitsi Meet** (peer-to-peer full-mesh mode vs. Jitsi Videobridge SFU) | Closest architectural analog: Jitsi runs direct full-mesh P2P for small calls and escalates to the Videobridge SFU as participants grow. Willow deliberately stops at the full-mesh half of that spectrum — SFU/MCU explicitly out of scope — accepting the O(n²) upload cost the bridge exists to avoid. |
84+
| **LiveKit** and **mediasoup** (open-source WebRTC SFU servers) | Canonical selective-forwarding-unit implementations that scale calls by routing all media through a server. Cited as the rejected alternative: Willow forgoes their scalability to keep media strictly peer-to-peer, never routing call media through an operator-run server. |
85+
| **Discord voice/video** (custom C++ WebRTC SFU) | The product Willow replaces routes all voice/video/screen-share through Discord-operated selective-forwarding servers that hold the stream keys. Willow's full-mesh choice is the direct decentralization/privacy counterpoint — no operator server ever sees or relays call media. |
86+
| **Matrix / Element Call** (full-mesh MSC3401 → MatrixRTC + LiveKit SFU, MSC4143/MSC4195) | Element Call's evolution makes Willow's tradeoff explicit: it began full-mesh (MSC3401) and later moved to a LiveKit-based SFU precisely because full-mesh does not scale past small groups — the same scaling ceiling Willow knowingly accepts to stay serverless-for-media. |
87+
| **W3C Web Audio API** (`AnalyserNode.getByteFrequencyData`) | Provides the frequency/amplitude analysis Willow's `VoiceManager`/`SpeakingDetector` polls (~60 ms `setInterval`) for client-side speaking detection, instead of inferring activity from server-side RTP or a dedicated voice-activity-detection service — fitting Willow's no-media-server model. |
88+
7489
## Design
7590

7691
### 1. Call Page (Main View)

docs/specs/2026-03-27-shareable-join-links-design.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,20 @@ Replace the multi-step invite flow (share PeerId, generate per-recipient invite,
8383
- Works in a browser without app installation.
8484
- Bevy desktop app is out of scope — web-only for this iteration.
8585

86+
## Prior Art
87+
88+
The design borrows from invite-link, peer-locator, and live-rendezvous systems, and deliberately diverges where the pointer-not-credential and inviter-online constraints demand it:
89+
90+
| System | Relevance / how Willow diverges |
91+
|---|---|
92+
| **Discord invite links** | The shareable-URL UX target: one copy-pasteable link with optional max-uses (1/5/10/25/.../no-limit) and expiry (30 min–7d/never) plus revoke. Willow keeps the affordance and the same "shareable links are open to whoever holds them" tradeoff, but inverts the security model — Discord's link *is* the credential against a hosted guild, whereas Willow's link is only a pointer to a live, inviter-gated P2P key exchange. |
93+
| **Matrix `matrix.to` permalinks + `?via=` routing** (MSC1704) | Precedent for a share link that is a *locator*, not a secret: `matrix.to/#/!room:server?via=hint` carries only the room ID plus homeserver routing hints, behind a `#/` fragment computed client-side so the redirector never sees the target. Mirrors Willow's `JoinToken` (server_id + inviter PeerId as a locate/route hint, display-only names, zero trust material) and its fragment-encoded token. |
94+
| **Nostr `nostr:` URIs (NIP-21) over bech32 entities (NIP-19)** | The canonical "shareable identifier that is a pointer, not a secret" pattern. NIP-21's `nostr:` scheme admits every NIP-19 entity *except* `nsec` — i.e. `npub`/`nprofile`/`nevent` (public keys / events, often carrying relay hints for locating them) are shared while the secret key is structurally excluded. Directly parallels Willow putting inviter PeerId + server_id in the link and keeping the X25519 channel key out of it. |
95+
| **Magic Wormhole — SPAKE2 PAKE over a rendezvous/mailbox server** (Warner; SPAKE2 due to Abdalla & Pointcheval) | The live-online-handshake model: a single-use, forward-secure code that is worthless alone and only completes via an interactive online rendezvous, deriving a fresh shared key per exchange (one guess per code, no offline attack). Willow adopts the same "both parties online, ephemeral exchange, no reusable joinable secret" stance — X25519 key-exchange at click time over gossipsub — rather than a stored/relay-cached credential. Willow's locator is an HTTPS link to a known peer instead of a spoken short code, so it skips the PAKE entropy-stretch step. |
96+
| **Signal group invite links** (server-mediated join requests + Reset Link) | Closest analog to Willow's gating + revocation: a shareable group link where admins can require approval of incoming join requests and can *reset* the link to invalidate the old one. Willow matches approve-or-auto-approve plus "revoke = invalidate the link_id," but performs the gating peer-to-peer with no group server, enforcing max-uses/expiry in the inviter's local persisted state. |
97+
| **pkarr (Public-Key Addressable Resource Records) + iroh node discovery** | The "resolve a public key to where-to-reach-it" idiom underpinning Willow's stack: pkarr publishes signed records keyed by an Ed25519 pubkey to the BitTorrent Mainline DHT (BEP44), and iroh uses it for global NodeID-based discovery. Willow's link plays the same locate-the-peer role — carry the inviter PeerId, resolve to a live endpoint — but defers the trust exchange to the subsequent handshake rather than encoding any grant. |
98+
| **Capability URLs / "secret link" pattern** (W3C TAG, *Good Practices for Capability URLs*, 2014) | The explicit counter-example Willow rejects: bearer-secret-in-URL designs leak via history, Referer, logs, and third-party scripts, and need rate-limiting against enumeration. Willow borrows only the fragment trick the TAG endorses (secret-bearing data in the fragment, never sent to the server) while refusing to make the URL a bearer token, and adds silent-drop anti-enumeration so unknown link_ids cannot be probed (a strictly stronger stance than the TAG's rate-limiting advice). |
99+
86100
## Security Model
87101

88102
The link is a **pointer, not a credential**. It contains no secrets — only enough information to find the inviter on the P2P network. All sensitive data (channel keys) flows over the existing signed gossipsub channel during a live exchange.

0 commit comments

Comments
 (0)