feat(pds): emit sync 1.1 firehose events#171
Merged
Merged
Conversation
Brings the firehose payload up to the current spec so the bsky.network relay's strict-validation warnings (notably "missing prevData field") stop firing against Cirrus hosts, and so account/sync state actually propagates without polling. - #commit gets prevData + ops[].prev, tooBig pinned to false, and the CAR slice now carries the MST covering proof (newBlocks + relevantBlocks) needed for inductive verification. - Writes go through repo.formatCommit + repo.applyCommit instead of applyWrites so the CommitData is available; drops the SQL-by-rev block reconstruction. - New #sync and #account events; activate/deactivate now emit them (plus #identity on activate). - #identity.handle is optional; rpcEmitIdentityEvent routes through the sequencer instead of writing empty payloads. - #info OutdatedCursor sent when a cursor predates the retention window; the stream continues from the earliest available event. - applyWrites caps at 200 ops per spec.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
atproto-pds | 1c8824a | May 24 2026, 06:11 AM |
commit: |
Merged
This was referenced May 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings Cirrus's firehose up to AT Protocol sync 1.1 so the bsky.network relay's strict-validation warnings (notably
missing prevData field, logged againstwisp.mk.ggand other Cirrus hosts) stop firing, and so account/sync state propagates to relays without polling. Diagnosed from public relay log charts shared by @calabro.io on this thread; cross-referenced against the upstream PDS reference impl atbluesky-social/atproto/packages/pds/src/sequencer/events.ts.What changed on the wire
#commitprevData(the prior commit's MST root CID) so consumers can run the inductive MST-inversion verification.newBlocksandrelevantBlocks(the covering proof). Previously Cirrus shipped only the new-this-rev blocks viaSELECT bytes FROM blocks WHERE rev = ?.ops[]entry getsprevon update/delete (prior record CID), omitted on create.tooBigpinned tofalse. It was previouslycarBytes.length > 1_000_000, which is meaningless under the modern spec —tooBigis deprecated.rebase: falseretained for back-compat (unchanged).New event types
#sync—{did, rev, blocks: CAR with the commit block, time}. Emitted on activation so relays pick up the current state (e.g. after migration import) without diffing.#account—{did, active, status?}withstatus ∈ {takendown, suspended, deleted, deactivated}. Emitted onrpcActivateAccount(active=true) andrpcDeactivateAccount(active=false, status='deactivated'). Both previously just flipped a SQLite flag.#identityhandleis now optional (per spec — presence does not signal a change).rpcEmitIdentityEventnow routes throughsequencer.sequenceIdentityinstead of writing empty payloads to the DB.Cursor handling
#infoframe withname: "OutdatedCursor"is sent when a client connects with a cursor before the retention window; the stream continues from the oldest available event instead of disconnecting.Spec caps
applyWritesrejects >200 operations.Implementation notes
rpcCreateRecord,rpcDeleteRecord,rpcPutRecord,rpcApplyWrites) refactored fromrepo.applyWrites(ops, keypair)torepo.formatCommit(ops, keypair)+repo.applyCommit(commit)so theCommitData(withnewBlocks,relevantBlocks, etc.) is exposed to the sequencer.{op: 1, t: '#${event.type}'}— single dispatch path for all four event kinds.broadcastCommit→broadcastEvent(the function was already generic overSeqEvent).prevDatais captured asrepo.commit.databeforeapplyCommit. The signed commit object'sprevfield (prior commit CID) is unchanged — that's a different concept, handled by@atproto/repo.Test plan
firehose.test.tscovering:prevDatapresence + linkage to prior commit'sdataCID,ops[].prevfor update/delete and absence for create,tooBig=false, the 200-op rejection,#account+#identity+#syncemitted on activate,#accounton deactivate, idempotent activate not double-emitting, framet-tag dispatch for#sync/#account,OutdatedCursorinfo frame, optional handle on#identity.pnpm exec tsc --noEmitintroduces zero new TypeScript errors (16 pre-existing unrelated errors remain).pnpm buildsucceeds.wisp.mk.gg'smissing prevData fieldcount should drop to zero.