Add PubsubRpcLimits decoding count-cap to reduce memory footprint#477
Merged
lucassaldanha merged 20 commits intoMay 19, 2026
Conversation
Addresses the gossipsub OOM amplification reported in ethereum-bounty/teku#7 / libp2p#6 by enforcing repeated-field count limits in the pubsub decode pipeline before Rpc.RPC materialisation.
Add Result.Malformed sealed variant so RpcCountFrameDecoder can dispatch on type rather than inspecting the reason string prefix.
Fix Detekt SwallowedException warning in RpcMessageCountValidator by including the exception message in the Malformed result string.
… gap Empty idontwant entries (0x2a 0x00) are the same cheap amplification gadget as empty publish entries: each materialises an Rpc$ControlIDontWant allocation. Add rejectEmptyIDontWantEntries (default true) to PubsubRpcLimits, enforce it in the CTRL_IDONTWANT arm of RpcMessageCountValidator, enable it in GossipRouter.rpcLimits, and cover both the rejection and the NONE-is-a-no-op cases in tests.
…fields Hoist IHAVE/IWANT/GRAFT/PRUNE/IDONTWANT counters out of validateControl into a shared ControlCounters object allocated once per validateRpc call, so multiple top-level control fields (valid per proto2 merge semantics) accumulate into the same counters rather than resetting them each time. Prevents a peer from splitting crafted control messages across N fields each just under the configured cap to bypass the pre-decode OOM guard.
Adds two tests to lock in that RpcMessageCountValidator is off by default: - FloodRouter inherits PubsubRpcLimits.NONE from AbstractRouter, so non-Gossip routers see no change in wire behaviour. - RpcCountFrameDecoder forwards a frame that would otherwise be rejected (empty publish entry + extra publishes) when constructed with NONE, proving pass-through at the pipeline level — not just at the validator function.
Two changes that together force any future change to rpc.proto to be acknowledged in RpcMessageCountValidator: - Switch hardcoded field-number constants to the protobuf-generated *_FIELD_NUMBER constants, so renames or removals in rpc.proto break compilation. - Add ACKNOWLEDGED_REPEATED_FIELDS as the single source of truth for the repeated fields the validator inspects, plus a coverage test that recursively walks every message descriptor reachable from Rpc.RPC and asserts the acknowledged set equals the actual set. A new repeated field anywhere in the proto fails the test until the author adds it (and the corresponding decode-time guard).
lucassaldanha
changed the title
StefanBratanov
approved these changes
May 19, 2026
Collaborator
StefanBratanov
left a comment
StefanBratanov
left a comment
There was a problem hiding this comment.
LGTM, apart from one thing
|
|
||
| override fun decode(ctx: ChannelHandlerContext, msg: ByteBuf, out: MutableList<Any>) { | ||
| val result = try { | ||
| RpcMessageCountValidator.validate(msg, limits) |
Collaborator
There was a problem hiding this comment.
Isn't it better when PubsubRpcLimits is NONE we don't walk the buffer at all, it will fail when converting to the objecy anyways in this case
Collaborator
Author
There was a problem hiding this comment.
True. I'll add it.
When PubsubRpcLimits has no configured caps and no reject-empty flags (e.g. PubsubRpcLimits.NONE), the validator walks the buffer and always returns Accepted. Skip the walk entirely on that path — malformed bytes still surface from the downstream ProtobufDecoder, which already drives the same wire-exception handling. A new `PubsubRpcLimits.isNoop` predicate gates the fast-path. Defining it as a property of the limits (rather than checking equality with NONE) keeps the fast-path correct even if a caller hand-builds an all-null PubsubRpcLimits and forces future field additions to be considered. Adds PubsubRpcLimitsTest covering isNoop semantics and an RpcCountFrameDecoder test that pipes a truncated frame through with NONE and asserts it is forwarded — proof that the validator did not run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.