feat(pubsub): stateless space-scoped pub/sub over any-sync#726
Open
requilence wants to merge 6 commits into
Open
feat(pubsub): stateless space-scoped pub/sub over any-sync#726requilence wants to merge 6 commits into
requilence wants to merge 6 commits into
Conversation
Add an ephemeral, fire-and-forget, at-most-once publish/subscribe channel scoped to a space, carried over a dedicated PubSub DRPC stream that is fully isolated from the sync engine. commonspace/pubsub: - Wire protocol (pubsubproto): PubSubStream bidi RPC with Subscribe/ Unsubscribe/Publish/Status frames. - Flat + NATS-style wildcard topics (* one segment, > tail) matched by a per-space sublist-shaped trie; reserved acc/.../<accId> self-owned namespace. - ACL-gated subscribe and publish; per-message account-key signatures (length-prefixed, domain-separated) verified by receivers; payloads encrypted with the space ReadKey via a pluggable Crypto. - Node relay with the one-hop rule (client-originated msgs forwarded once to other responsible nodes, relayed msgs never re-forwarded); LAN peers symmetric; echo and duplicate-path suppression via a bounded msgId ring; per-peer publish token bucket. - Serving-side interest keyed by streamId (trie refcounts subscribing streams) so a reconnecting peer's fresh stream keeps its interest and a mid-subscribe close cannot orphan state. - Reconnect watcher (periodic + on-close resync) and peer TTL so a pure subscriber survives stream drops and idle pool GC. - CloseSpace teardown and EvictMember (active drop on ACL removal, strips tags so delivery stops even on an open stream). - Receive path runs cheap filters (interest, membership, ownership, timestamp staleness) before the Ed25519 verify and records dedup only after verify, shedding forged-signature floods and closing a replay window. net/streampool: - NewStreamPool standalone constructor with WithStreamCloseHook (carries streamId) and WithMetric options; CtxStreamId accessor; RemoveTagsById; cross-tag stream dedup in Broadcast. docs/stateless-pubsub: research and design docs.
Add Service.RevalidateMembers(spaceId, isMember) so a relay node can evict every subscriber whose account no longer passes the membership predicate in a single pass on an ACL change, refactoring EvictMember to share the core eviction path.
- WithMetric no longer registers the private pool into the shared single-slot sync metric (RegisterStreamPoolSyncMetric): that slot belongs to the app-level sync streampool, so a second pool there overwrote its OutgoingMsg telemetry and nulled it on Close. WithMetric now only registers the namespaced prometheus gauges — independent observability, no clobber. (F1, medium) - handleSubscribe cap rejection reports only the rejected patterns in the TooManyTopics status, not the whole subscribe batch. (F4) - removeStream always clones tags for its debug log. (F5) - drop the dead streamInterest.peerId field. (F6) - doc: dedup is a FIFO ring (not LRU); rate limiter is per-peer; note that Close blocks on in-flight handlers and that resync coverage is bounded by DialQueueSize. Adds TestCapExceededReportsOnlyRejected. The remaining review items were verified correct (lock ordering, interest+tag rollback, receive-path membership-before-verify, no double-decrement) — no change needed.
Coverage provided by https://github.com/seriousben/go-patch-cover-action |
…eference
Replace docs/stateless-pubsub/{DESIGN,RESEARCH}.md with a single as-built
docs/stateless-pubsub.md: present-tense reference for the shipped v1 plus a
compact background/rationale distilled from the research. Corrects drift
against the code (SyncInterest/RevalidateMembers, NewStreamPool, Unexpected=0
zero value, receive-path ordering) and drops the now-resolved open items.
800 now belongs to filesyncv2 (commonfile/fileproto/fileprotov2) on the v0.13.x base; the any-sync core <1000 offset range is fully allocated, so pubsub takes the next free offset, 1100. Hand-edited the generated descriptor (the pinned protoc-gen plugins are Linux binaries and won't run locally; regenerating with local plugins introduced unrelated drpc/vtproto version drift). Verified the rawDesc varint round-trips to 1100 -> "ErrorOffset". Record the allocation in docs/rpc-error-offsets.md (brought in from v0.13.x) and reconcile its "core < 1000" rule with the now-exhausted core range.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an ephemeral, fire-and-forget, at-most-once publish/subscribe channel scoped to a space, carried over a dedicated
PubSubDRPC stream that is fully isolated from the sync engine. Design and research docs are indocs/stateless-pubsub/.Use cases this unlocks (built downstream in heart): typing indicators, presence, live cursors, read-position signals — anything ephemeral that shouldn't touch the DAG/KV persistence layer.
How it works
commonspace/pubsub/pubsubproto): one bidiPubSubStreammultiplexing all spaces/topics, withSubscribe/Unsubscribe/Publish/Statusframes.*one segment,>tail), matched by a per-space sublist-shaped trie. Reservedacc/…/<accId>namespace is publishable only by that account.ReadKeyvia a pluggableCrypto. Removed members are cut off actively viaEvictMember/RevalidateMembersand passively via key rotation.relayedmessages never re-forwarded); LAN peers symmetric; echo and duplicate-path suppression via a bounded msgId ring.net/streampoolgains a standaloneNewStreamPoolconstructor (WithStreamCloseHookcarrying streamId,WithMetric),RemoveTagsById,CtxStreamId, and cross-tag dedup inBroadcast— all additive/backward-compatible.Testing
Unit tests for validation/trie/dedup/signing plus a multi-peer in-memory fixture (2 relay nodes, 3 clients over real DRPC) covering fan-out with wildcards, cross-node relay + one-hop, membership rejection,
acc/ownership at all three enforcement layers, echo/duplicate suppression, reconnect interest survival,CloseSpace/EvictMember/RevalidateMembers, and stale-message rejection. Race-clean; full repo suite green.Review
Went through six independent review lenses (concurrency, security, spec-conformance, resource-bounds, API, plus a follow-up) — findings on interest keying, the reconnect watcher, and receive-path ordering were fixed; see
docs/stateless-pubsub/DESIGN.md§12 for what's implemented vs deferred.Downstream
The node relay lives in anyproto/any-sync-node#(pending). Client wiring in anytype-heart is the remaining piece.