feat(rpc): add eth_capabilities method for routing capability discovery#11438
feat(rpc): add eth_capabilities method for routing capability discovery#11438divi2806 wants to merge 38 commits into
Conversation
This commit introduces the `eth_capabilities` method to the Eth RPC module, which returns the node's historical data availability and head block information. It includes the new `EthCapabilitiesResult` class and associated classes (`CapabilityHead`, `CapabilityResource`, and `CapabilityDeleteStrategy`) to structure the response. Additionally, tests for the new functionality have been added to ensure correct behavior across various configurations, including archive and pruned nodes.
|
@claude review |
|
Claude finished @LukaszRozmej's task in 5m 43s —— View job Code Review
SummaryThe overall structure is clean — the DI wiring is correct ( Three correctness issues with the boundary calculations warrant attention before merge:
|
…tate reporting This commit updates the `eth_capabilities` method in the Eth RPC module to correctly report the availability of blocks and receipts based on the configured pruning mode. It ensures that the `OldestBlock` field is set to null for full pruning scenarios, preventing misleading information. Additionally, tests have been added to verify the behavior of the method across different configurations, including archive nodes and various pruning settings, ensuring compliance with the expected JSON structure.
|
@claude re-review |
|
Claude finished @LukaszRozmej's task in 3m 35s —— View job Re-Review
SummaryAll three Medium and all three Low findings from the prior review have been addressed in commit
The test suite grew from 4 to 8 tests, covering: archive, memory-pruned, full-pruned, no-receipts, archive-receipts-genesis, null-configs, JSON key names, and all-resources-present. Two new observations (both Low, non-blocking):
Verdict: ready to merge. No Medium or higher findings remain unresolved. |
This commit introduces the `EthCapabilitiesProvider` class, which encapsulates the logic for retrieving the capabilities of the Ethereum node, including block and receipt availability based on the current sync and pruning configurations. The `eth_capabilities` method in the `EthRpcModule` is refactored to utilize this new provider, streamlining the code and improving maintainability. Additionally, the interface for the `eth_capabilities` method is updated to include an example response for better clarity in usage.
LukaszRozmej
left a comment
There was a problem hiding this comment.
one last thing - cleanup json type
| [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] | ||
| public string? OldestBlock { get; init; } |
There was a problem hiding this comment.
don't format in the method, keep it as long. Json serialization will format it without allocating strings.
There was a problem hiding this comment.
Done — OldestBlock is now long?. Removed all .ToHexString() calls from EthCapabilitiesProvider; NullableLongConverter (globally registered in EthereumJsonSerializer) handles hex encoding on the wire.
| /// Response for eth_capabilities — describes the head block and the historical data availability of each resource type. | ||
| /// Follows ethereum/execution-apis#755. | ||
| /// </summary> | ||
| public class EthCapabilitiesResult |
There was a problem hiding this comment.
readonly record struct? Or record class
There was a problem hiding this comment.
@claude from performance perspective, better to make it struct or keeping this one class?
|
@claude which ones to make structs for optimal performance? |
|
Claude finished @LukaszRozmej's task in 1m 25s —— View job Struct vs Class Analysis for
|
| Type | Recommendation | Reasoning |
|---|---|---|
EthCapabilitiesResult |
record class |
If the nested types become structs, each CapabilityResource is ~40 bytes × 6 = ~240 bytes inline. A 256-byte struct is too large — copying it is worse than a single class allocation. It's the top-level container; one heap allocation is fine. |
CapabilityHead |
readonly record struct |
Two string references = ~16 bytes. Stored inline in EthCapabilitiesResult, saves 1 heap allocation per call. Ideal struct size. |
CapabilityResource |
readonly record struct |
Used 6× per result — saves 6 heap allocations per call. Bool + string ref + Nullable<CapabilityDeleteStrategy> lands at ~32–40 bytes, acceptable when stored as fields in a class (not passed around by value). |
CapabilityDeleteStrategy |
readonly record struct |
String ref + long = ~16 bytes. Used as CapabilityDeleteStrategy? → becomes Nullable<CapabilityDeleteStrategy> inline in CapabilityResource — no heap allocation for the strategy object. |
Net effect on allocations per eth_capabilities call:
- Before: result + head + 6× resource + 0–1× deleteStrategy = 8–9 allocations
- After structs on all nested types: result only = 1 allocation
One caveat on CapabilityDeleteStrategy?: When CapabilityDeleteStrategy is a struct, DeleteStrategy becomes Nullable<CapabilityDeleteStrategy>. [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] on a nullable value type works correctly in System.Text.Json — it omits the property when the nullable has no value.
Separate from the struct question — LukaszRozmej's inline comments also ask to keep Number as long and Hash as Hash256 (let the serializer format them, avoiding intermediate string allocations). That applies on top of these struct changes.
…g and Hash256 types This commit updates the `EthCapabilitiesResult` and `EthCapabilitiesProvider` classes to replace string representations of block numbers and hashes with their appropriate types: `long` for block numbers and `Hash256` for block hashes. This change enhances type safety and consistency across the codebase. Additionally, the related tests have been updated to reflect these changes, ensuring accurate validation of capabilities reporting.
Changed to |
…avros - Move OldestStateBlock off IBlockFinder: now declared on IBlockTree (writer side) and IWorldStateManager (external consumers). BlockTree implements a narrow IOldestStateBlockStore so WorldStateManager can delegate without taking a Nethermind.Blockchain dependency. - Gate Stateproofs separately from State via IWorldStateManager.SupportsTrieProofs: trie-backed managers return true, FlatWorldStateManager returns false to avoid advertising O(state-size) or snapshot-limited proof reconstruction. - Encapsulate the epoch->blocks conversion behind IHistoryPruner.GetRetentionBlocks; SlotsPerEpoch becomes private const again. - Clarify XML doc: Tx/Logs aliasing Receipts (with LogIndex caveat), JsonPropertyOrder is example-payload parity, not a wire requirement. - Replace Keccak.Zero in ExampleResponse with a realistic 32-byte hash. - Add CapabilitiesScenario row covering AncientReceiptsBarrier > AncientBodiesBarrier. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@claude re-review |
|
Claude finished @LukaszRozmej's task in 5m 31s —— View job Code Review (Pass 5)
SummaryCommit
Architecture is clean. The One remaining Low observation:
Verdict: ready to merge. No Critical, High, or Medium findings remain. The single Low observation is a test-coverage gap for the stateproofs flat-storage fix — worth adding before merge but not a blocker. |
|
|
||
| IWorldStateManager wsm = Substitute.For<IWorldStateManager>(); | ||
| wsm.OldestStateBlock.Returns(oldestStateBlock); | ||
| wsm.SupportsTrieProofs.Returns(true); |
There was a problem hiding this comment.
Low — no test for SupportsTrieProofs = false (flat-storage stateproofs disabled)
The GetCaps helper hardcodes wsm.SupportsTrieProofs.Returns(true) for every scenario. The new Stateproofs: worldStateManager.SupportsTrieProofs ? state : Disabled path — the whole fix for @svlachakis's 🔴 finding — is never exercised by the test suite. A regression that swapped the conditional would go undetected.
Consider adding one scenario row where SupportsTrieProofs = false and asserting caps.Stateproofs == Disabled:
yield return new CapabilitiesScenario(
Name: "flat_storage_disables_stateproofs",
ExpectedState: Available, ExpectedStateproofs: Disabled,
ExpectedReceipts: Available, ExpectedBlocks: Available)
{ SyncConfig = fullSync, SupportsTrieProofs = false }; // or pass as a flag(You'd need to thread supportsTrieProofs through CapabilitiesScenario and GetCaps similarly to how HistoryPruner is passed.)
…StateManager - Move the absolute oldest-state-block floor out of BlockTree entirely. WorldStateManager now owns the metadata-DB persistence (same key) via a small OldestStateBlockStore helper; FlatWorldStateManager does the same. IBlockTree/IBlockFinder no longer expose OldestStateBlock. - Route FullPruner and StateSyncRunner writes through IWorldStateManager.OldestStateBlock (proper layering). - FullPrunerFactory.Create now takes IWorldStateManager (not IStateReader) so the pruner can write the floor without going through the block tree. - Make CopyTrie synchronous (it has no async work; was already returning Task.FromResult). - Reformat multi-arg EthCapabilitiesProvider constructor calls one-per-line. - Drop IOldestStateBlockStore glue interface no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Flat storage retains trie nodes in path-keyed columns (see BaseTriePersistence: StateNodesTop, StateNodes, StorageNodes, FallbackNodes) and serves eth_getProof in O(trie-depth) for any block where the flat snapshot is present — same complexity as trie storage. The previous SupportsTrieProofs gate was based on a wrong assumption. Drop the property; stateproofs availability matches state availability for both Nethermind backends. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tartup Adds a one-shot consistency check at world-state-manager activation: - If OldestStateBlock points at a block whose state root is not present in the state DB, clear it. - Same for IBlockTree.BestPersistedState. This handles the scenario where a user wipes the state directory without touching MetadataDb / BlockInfoDb — without the check, eth_capabilities would over-report state availability until sync rewrites the floor. The check uses GlobalStateReader.HasStateForBlock so it's meaningful (verifies the root, not just "is the DB empty"). Wired via OnActivate<IWorldStateManager> in both WorldStateModule and FlatWorldStateModule. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move StateMetadataValidator from Nethermind.Init to Nethermind.Blockchain (where IBlockTree lives) so it can be tested in Nethermind.Blockchain.Test. Use explicit BlockTreeLookupOptions overload so NSubstitute can intercept the FindHeader call without going through the default interface method. Tests cover: - OldestStateBlock cleared when state root missing - OldestStateBlock kept when state root present - OldestStateBlock kept when header unknown (defensive) - No-op when OldestStateBlock is null - BestPersistedState clear/keep symmetric to OldestStateBlock - Markers independent (one stale, other ok) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@claude review |
|
Claude finished @LukaszRozmej's task in 5m 33s —— View job Code Review (Pass 6)
SummaryThis is pass 6. Since pass 5 (four commits were merged):
All outstanding findings from prior rounds are resolved:
Architecture is clean. The One new Low observation:
Verdict: ready to merge. No Critical, High, or Medium findings remain. |
| _value = value; | ||
| if (value.HasValue) | ||
| { | ||
| metadataDb.Set(MetadataDbKeys.OldestStateBlock, Rlp.Encode(value.Value).Bytes); |
There was a problem hiding this comment.
Low — DB key is not deleted when set to null
IKeyValueStore.Remove(key) is defined as Set(key, null) — setting null deletes the entry in RocksDB. But the setter only writes when value.HasValue; when StateMetadataValidator calls worldStateManager.OldestStateBlock = null to clear a stale floor, the in-memory _value becomes null but the DB key persists. On the next startup the stale value is loaded again and the validator re-clears it each time.
BlockTree.BestPersistedState has the same pattern (no-delete-on-null), so this follows existing convention. Functionally correct: the validator catches the stale floor on every startup. Still worth fixing for consistency:
| metadataDb.Set(MetadataDbKeys.OldestStateBlock, Rlp.Encode(value.Value).Bytes); | |
| if (value.HasValue) | |
| metadataDb.Set(MetadataDbKeys.OldestStateBlock, Rlp.Encode(value.Value).Bytes); | |
| else | |
| metadataDb.Remove(MetadataDbKeys.OldestStateBlock); |
…leared - Address claude-bot review (pass 6): OldestStateBlockStore setter now deletes the metadata DB entry on null, so a stale floor isn't reloaded on every restart after StateMetadataValidator clears it. - Remove unused `using Nethermind.Blockchain.Find;` from StateMetadataValidator and its tests — BlockTreeLookupOptions lives in Nethermind.Blockchain. Fixes IDE0005 lint failures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…stabilize retentionBlocks - Block/Receipt resources are now Disabled while fast sync is configured to download but the descending pointer is still null. Reporting the barrier (eventual floor) as oldestBlock before any insertion over-claimed availability. - Replace IWorldStateManager.GetOldestStateBlock(head) with RetentionWindowBlocks. Provider derives windowOldest itself and emits the configured retention so retentionBlocks stays correct when head < retention (fresh chain) instead of rounding down to head. - Lock OldestStateBlockStore reads/writes: long? is two fields, the RPC path reads concurrently with sync runners and the full pruner. - Add a regression-scenario test for the pre-batch fast-sync window and update existing scenarios to set descending pointers consistent with their post-sync framing.
|
@claude re-review |
|
Claude finished @LukaszRozmej's task in 5m 13s —— View job Code Review (Pass 7)
SummaryThis is pass 7. Two commits landed since pass 6 ( Pass 6 Low finding:
New changes in
Verification of key logic paths:
Verdict: ready to merge. No Critical, High, or Medium findings remain. No new Low observations. |
# Conflicts: # src/Nethermind/Nethermind.Runner/packages.lock.json
…tests - Promote OldestStateBlockStore to a DI singleton (registered in BlockTreeModule via IDbProvider.MetadataDb). FullPruner, StateSyncRunner, EthCapabilitiesProvider, StateMetadataValidator now depend on the narrow store instead of IWorldStateManager. Removes OldestStateBlock from IWorldStateManager. - Drop IDbProvider from FlatWorldStateManager (the store is injected directly). - Extract MetadataLongStore reusable base class taking IDb + key; OldestStateBlockStore becomes a thin sealed wrapper hardcoding MetadataDbKeys.OldestStateBlock. - IFullPrunerFactory.Create signature: IWorldStateManager → IStateReader (mirrors what FullPruner actually needs); factory pulls the store via DI. - IsDescendingResourceDownloaded as single expression. - Drop the stateproofs-aliases-state comment. - Deduplicate StateMetadataValidatorTests with [TestCase] parameterization across (Marker × initial × StateAt × shouldClear); test count: 9 (8 cases + 1 independent-markers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@claude review |
|
Claude finished @LukaszRozmej's task in 6m 50s —— View job Code Review (Pass 8)
SummaryThis is pass 8. One commit landed since pass 7 ( What changed in
Architecture is clean. The ownership model is now unambiguous: one singleton Verdict: ready to merge. No Critical, High, or Medium findings remain. |
| IBlockTree blockTree = ctx.Resolve<IBlockTree>(); | ||
| ILogManager logManager = ctx.Resolve<ILogManager>(); | ||
| new TrieStoreBoundaryWatcher(worldStateManager, blockTree, logManager); | ||
| StateMetadataValidator.DiscardStaleFloors( |
There was a problem hiding this comment.
So this is kind of deliberate.
Now we have in metadata db info that is related to state.
Now if someone does drop state db (like a directory) - which IMO is useful feature to force resync - this data becomes stale and misleading. So I added a quick check for it.
Alternatively I can put it into state db (metadata column?)
There was a problem hiding this comment.
Put it in trie world state factory I guess.
|
|
||
| .Map<IWorldStateManager, PruningTrieStateFactoryOutput>((o) => o.WorldStateManager) | ||
| .OnActivate<IWorldStateManager>((wsm, ctx) => | ||
| StateMetadataValidator.DiscardStaleFloors( |
| public sealed class EthCapabilitiesProvider( | ||
| IReadOnlyBlockTree blockTree, | ||
| IWorldStateManager worldStateManager, | ||
| OldestStateBlockStore oldestStateBlockStore, |
There was a problem hiding this comment.
Hide behind IWorldStateManager please. Implement separately for flatdb.
There was a problem hiding this comment.
Yeah that was my previous iteration, but:
- It living into metadata db caused issues and IMO wrong dependencies
- WordStateManager needed to be injected in places where it kind of shouldn't
There was a problem hiding this comment.
maybe if I put it under a separate interface that IWorldStatemanager would inherit + living in a separate metadata column in StateDb would be optimal...?
There was a problem hiding this comment.
For flat it does not even use metadata db, it has its own metadata db, which it can consult to report this. What kind of place the WorldStateManager should not be put? THen put it in IStateReader?
…-backend state-DB-resident storage Addresses @asdacap review on `fabd507183`: - Re-add `OldestStateBlock` (+ `RetentionWindowBlocks`) on a new narrow `IStateBoundary` interface; `IWorldStateManager : IStateBoundary`. - Trie `WorldStateManager`: floor persists in `dbProvider.StateDb` under `Keccak("OldestStateBlock")` (collision-free vs 32-byte hash keys and HalfPath section bytes 0/1/2). - `FlatWorldStateManager`: floor persists in `FlatDbColumns.Metadata` under the same keccak key, alongside the existing `CurrentState`/`Layout` keys. - Co-locating the floor with state nodes means wiping the state directory drops the floor automatically — no startup validator needed. - Drop `StateMetadataValidator` (+ tests), `OldestStateBlockStore`, `MetadataLongStore`, `MetadataDbKeys.OldestStateBlock` (the metadata-DB storage was never deployed), and the duplicated `OnActivate` validator wiring in both world-state modules. - `FullPruner` / `StateSyncRunner` / `EthCapabilitiesProvider` depend on the narrow `IStateBoundary` (auto-resolved via `IWorldStateManager`). - `IFullPrunerFactory.Create` signature returns to `IWorldStateManager`. - Capabilities test mock simplifies to a single `IStateBoundary` substitute. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes #11260
Changes
eth_capabilitiesRPC method toIEthRpcModuleandEthRpcModuleper ethereum/execution-apis#755EthCapabilitiesResultresponse model withCapabilityHead,CapabilityResource, andCapabilityDeleteStrategytypesISyncConfigandIPruningConfigas optional constructor parameters intoEthRpcModuleandEthModuleFactoryto derive historical data availabilitynumber+hash) and six resource descriptors:state,tx,logs,receipts,blocks,stateproofsoldestBlockanddeleteStrategyare omitted (null) for disabled resources and archive nodes;deleteStrategyonly appears for in-memory pruned nodes (type: "window")stateproofsis disabled on pruned nodes sinceeth_getProofrequires full trie historyTypes of changes
What types of changes does your code introduce?
Testing
Requires testing
If yes, did you write tests?
Notes on testing
4 tests added in
EthRpcModuleTests.Capabilities.cs:eth_capabilities_returns_all_resources_and_head— verifies all 6 resource fields and head block info are presenteth_capabilities_archive_node_has_stateproofs_enabled_and_no_delete_strategy— archive node (PruningMode.None) serves state proofs, has no window delete strategyeth_capabilities_pruned_node_disables_stateproofs_and_reports_window— memory-pruned node disablesstateproofs, reportsdeleteStrategy.type = "window"with correctretentionBlockseth_capabilities_no_receipts_disables_tx_logs_receipts— whenDownloadReceiptsInFastSync = false,tx,logs, andreceiptsare disabled with nooldestBlockAll 5 tests pass (
total: 5 | failed: 0 | succeeded: 5).Documentation
Requires documentation update
Requires explanation in Release Notes
Add
eth_capabilitiesJSON-RPC method. Returns the node's current head block and historical data availability forstate,tx,logs,receipts,blocks, andstateproofsresources. Enables RPC routers and load balancers to make routing decisions without active probing. Implements ethereum/execution-apis#755.