refactor(p2p): merge FastTxCollection into TxCollection with sequential pipeline#23245
Merged
Merged
Conversation
0d77c01 to
d6be609
Compare
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
d6be609 to
eb8d696
Compare
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
spalladino
approved these changes
May 13, 2026
| /** Clears all pending entries. */ | ||
| public clearPending(): void { | ||
| this.entries.clear(); | ||
| await Promise.allSettled(times(this.config.workerCount, () => this.workerLoop(entries, requestTracker, context))); |
Contributor
There was a problem hiding this comment.
Heads up this changes the semantics of workerCount. If I understood correctly, now we have workerCount per request, not global. I think this is fine though, since worker count is not too high (defaults to 5).
This was referenced May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
FastTxCollectionas a separate class and absorbs all its logic directly intoTxCollectionIRequestTracker— the same synchronization primitive used by node and reqresp paths. The tracker is the single source of truth for "is this tx still missing?" and "is this request still alive?"FileStoreTxCollectionsimplified: droppedstart()/stop()/persistent worker pool/wakeSignal.startCollecting(requestTracker, context)returnsPromise<void>, spins up its own per-call worker pool, and workers self-terminate when the tracker is cancelled (all-fetched / deadline / external)Collection flow inside
collectFasttxCollectionFastNodesTimeoutBeforeReqRespMs— interruptible by cancellation or by node exhaustion (so when no nodes are configured, reqresp starts immediately)txCollectionFileStoreFastDelayMs— interruptible by cancellation or reqresp completionPromise.allSettledon node + reqresp + file storetxCollectionFileStoreFastDelayMsdescription updated to reflect it is now anchored to reqresp start, not collection start.File store / tracker integration
FileStoreTxCollection.startCollectingno longer takes(txHashes, context, deadline); it takes(requestTracker, context)and reads the missing txs + deadline from the trackerrequestTracker.isMissing(hash)each scan — if the tx was found via another path (node/reqresp/gossipsub), the entry is dropped without an extra fetchrequestTracker.cancellationToken— cancelling a request (deadline,stopCollectingForBlocksUpTo/After, orstop()) propagates to file store workers immediatelyfoundTxs/clearPendingplumbing onFileStoreTxCollection— the tracker handles both implicitlystartCollectingyields once after building its entry set, so a synchronous follow-up call (e.g.markFetchedin tests, or the gossipsub-found path in production) lands before workers begin scanningTests
tx_collection.test.ts: collapsed theTestFastTxCollectionsubclass; all accesses go directly throughTxCollection. Added "starts reqresp immediately when no nodes are configured" covering the node-exhaustion shortcutfile_store_tx_collection.test.ts: rewritten for the new shape — nostart()/stop(), lifecycle driven by the tracker (cancel to terminate workers). New "workers exit when tracker is cancelled" covers the per-call worker-pool teardownCloses https://linear.app/aztec-labs/issue/A-933/tx-collection-dont-retrieve-transactions-that-have-already-been via new synchronization with the request tracker.