Skip to content

feat(rpc): add eth_capabilities method for routing capability discovery#11438

Open
divi2806 wants to merge 38 commits into
NethermindEth:masterfrom
divi2806:feature/11260-eth-capabilities-support
Open

feat(rpc): add eth_capabilities method for routing capability discovery#11438
divi2806 wants to merge 38 commits into
NethermindEth:masterfrom
divi2806:feature/11260-eth-capabilities-support

Conversation

@divi2806
Copy link
Copy Markdown
Contributor

Closes #11260

Changes

  • Add eth_capabilities RPC method to IEthRpcModule and EthRpcModule per ethereum/execution-apis#755
  • Add EthCapabilitiesResult response model with CapabilityHead, CapabilityResource, and CapabilityDeleteStrategy types
  • Inject ISyncConfig and IPruningConfig as optional constructor parameters into EthRpcModule and EthModuleFactory to derive historical data availability
  • Response includes current head block (number + hash) and six resource descriptors: state, tx, logs, receipts, blocks, stateproofs
  • oldestBlock and deleteStrategy are omitted (null) for disabled resources and archive nodes; deleteStrategy only appears for in-memory pruned nodes (type: "window")
  • stateproofs is disabled on pruned nodes since eth_getProof requires full trie history

Types of changes

What types of changes does your code introduce?

  • Bugfix (a non-breaking change that fixes an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a change that causes existing functionality not to work as expected)
  • Optimization
  • Refactoring
  • Documentation update
  • Build-related changes
  • Other: Description

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

4 tests added in EthRpcModuleTests.Capabilities.cs:

  • eth_capabilities_returns_all_resources_and_head — verifies all 6 resource fields and head block info are present
  • eth_capabilities_archive_node_has_stateproofs_enabled_and_no_delete_strategy — archive node (PruningMode.None) serves state proofs, has no window delete strategy
  • eth_capabilities_pruned_node_disables_stateproofs_and_reports_window — memory-pruned node disables stateproofs, reports deleteStrategy.type = "window" with correct retentionBlocks
  • eth_capabilities_no_receipts_disables_tx_logs_receipts — when DownloadReceiptsInFastSync = false, tx, logs, and receipts are disabled with no oldestBlock

All 5 tests pass (total: 5 | failed: 0 | succeeded: 5).

Documentation

Requires documentation update

  • Yes
  • No

Requires explanation in Release Notes

  • Yes
  • No

Add eth_capabilities JSON-RPC method. Returns the node's current head block and historical data availability for state, tx, logs, receipts, blocks, and stateproofs resources. Enables RPC routers and load balancers to make routing decisions without active probing. Implements ethereum/execution-apis#755.

This commit introduces the `eth_capabilities` method to the Eth RPC module, which returns the node's historical data availability and head block information. It includes the new `EthCapabilitiesResult` class and associated classes (`CapabilityHead`, `CapabilityResource`, and `CapabilityDeleteStrategy`) to structure the response. Additionally, tests for the new functionality have been added to ensure correct behavior across various configurations, including archive and pruned nodes.
@LukaszRozmej
Copy link
Copy Markdown
Member

@claude review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented Apr 30, 2026

Claude finished @LukaszRozmej's task in 5m 43s —— View job


Code Review

  • Gather context and read changed files
  • Read relevant rules files (coding-style, robustness, di-patterns)
  • Review EthCapabilitiesResult.cs
  • Review EthRpcModule.cs changes
  • Review EthModuleFactory.cs changes
  • Review IEthRpcModule.cs changes
  • Review test file
  • Post inline comments and final summary

Summary

The overall structure is clean — the DI wiring is correct (ConfigRegistrationSource will automatically resolve ISyncConfig/IPruningConfig in production), the model types are well-structured, and the camelCase serialization from EthereumJsonSerializer maps property names to the spec correctly.

Three correctness issues with the boundary calculations warrant attention before merge:

Severity Finding
Medium AncientReceiptsBarrierCalc always returns ≥ 1 via Math.Max(1, ...). For an archive node (PivotNumber = 0), tx/logs/receipts report oldestBlock = "0x1" instead of "0x0". Genesis receipts exist, the API says they don't. → inline
Medium Blocks.Disabled = false and OldestBlock = "0x0" are reported even when the node hasn't downloaded any headers (e.g., freshly started). Callers will assume full block history is available when it isn't. → inline
Medium PruningMode.Full-only (no memory component) computes state.OldestBlock = head.Number, implying only the tip's state is available. This is overly conservative — full pruning sweeps run periodically, not block-by-block. → inline
Low _blockFinder as IBlockTree silently returns null if the finder isn't a block tree; lowestBlock falls back to 0. Works today but is a fragile assumption. → inline
Low Missing blank line before eth_capabilities. → inline
Low No test validates the serialized JSON key names — only in-memory properties are asserted. A regression on naming policy or a misplaced [JsonPropertyName] would go undetected. → inline

Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
divi2806 and others added 2 commits April 30, 2026 21:49
…tate reporting

This commit updates the `eth_capabilities` method in the Eth RPC module to correctly report the availability of blocks and receipts based on the configured pruning mode. It ensures that the `OldestBlock` field is set to null for full pruning scenarios, preventing misleading information. Additionally, tests have been added to verify the behavior of the method across different configurations, including archive nodes and various pruning settings, ensuring compliance with the expected JSON structure.
@LukaszRozmej
Copy link
Copy Markdown
Member

@claude re-review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

Claude finished @LukaszRozmej's task in 3m 35s —— View job


Re-Review

  • Read relevant rules files
  • Read changed files
  • Analyze correctness, security, DI patterns, and tests
  • Post inline comments
  • Post final summary

Summary

All three Medium and all three Low findings from the prior review have been addressed in commit ed261ee. Here is the status of each:

Prior Finding Status
Medium AncientReceiptsBarrierCalc returns ≥ 1, wrong for archive nodes ✅ Fixed: (syncConfig?.PivotNumber ?? 0) == 0 ? 0 : .... Regression test added.
Medium Blocks.OldestBlock = "0x0" when node hasn't finished syncing ✅ Fixed: headersAvailable = blockTree?.BestSuggestedHeader is not null guards the field.
Medium PruningMode.Full reports misleading state.OldestBlock = head.Number ✅ Fixed: Full-only pruning now returns null for OldestBlock. Test added.
Low _blockFinder as IBlockTree silent null fallback ✅ Addressed: if blockTree is null, headersAvailable = falseDisabled = true, so the ?? 0 fallback is never exposed. Comment added.
Low Missing blank line before eth_capabilities ✅ Fixed.
Low No test for wire-format JSON keys ✅ Fixed: eth_capabilities_serializes_to_spec_compliant_json_key_names added.

The test suite grew from 4 to 8 tests, covering: archive, memory-pruned, full-pruned, no-receipts, archive-receipts-genesis, null-configs, JSON key names, and all-resources-present.

Two new observations (both Low, non-blocking):

  • State.Disabled = false is hardcoded. On a freshly-started memory-pruned node with no chain head, this yields { disabled: false, oldestBlock: null }. Callers should tolerate null oldestBlock on a non-disabled resource — worth documenting the intent. → inline
  • eth_capabilities in IEthRpcModule has no ExampleResponse unlike every other implemented method. Minor, affects auto-generated docs only. → inline

Verdict: ready to merge. No Medium or higher findings remain unresolved.

Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthRpcModule.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/IEthRpcModule.cs Outdated
This commit introduces the `EthCapabilitiesProvider` class, which encapsulates the logic for retrieving the capabilities of the Ethereum node, including block and receipt availability based on the current sync and pruning configurations. The `eth_capabilities` method in the `EthRpcModule` is refactored to utilize this new provider, streamlining the code and improving maintainability. Additionally, the interface for the `eth_capabilities` method is updated to include an example response for better clarity in usage.
@divi2806 divi2806 requested a review from LukaszRozmej May 1, 2026 10:07
Copy link
Copy Markdown
Member

@LukaszRozmej LukaszRozmej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one last thing - cleanup json type

Comment on lines +56 to +57
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? OldestBlock { get; init; }
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't format in the method, keep it as long. Json serialization will format it without allocating strings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — OldestBlock is now long?. Removed all .ToHexString() calls from EthCapabilitiesProvider; NullableLongConverter (globally registered in EthereumJsonSerializer) handles hex encoding on the wire.

Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthCapabilitiesResult.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthCapabilitiesResult.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthCapabilitiesResult.cs Outdated
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Eth/EthCapabilitiesResult.cs Outdated
/// Response for eth_capabilities — describes the head block and the historical data availability of each resource type.
/// Follows ethereum/execution-apis#755.
/// </summary>
public class EthCapabilitiesResult
Copy link
Copy Markdown
Member

@LukaszRozmej LukaszRozmej May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

readonly record struct? Or record class

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@claude from performance perspective, better to make it struct or keeping this one class?

@LukaszRozmej
Copy link
Copy Markdown
Member

@claude which ones to make structs for optimal performance?

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

Claude finished @LukaszRozmej's task in 1m 25s —— View job


Struct vs Class Analysis for EthCapabilitiesResult types

  • Read EthCapabilitiesResult.cs
  • Read EthCapabilitiesProvider.cs
  • Analyze performance tradeoffs
  • Post answer

Answer

Rule of thumb: a struct saves a heap allocation only when it's stored as a field in a class (it lives inline) or in a Nullable<T> slot. A struct that itself contains 6+ struct fields becomes too large to pass by value safely (guideline: keep structs ≤ 16–24 bytes).

Type Recommendation Reasoning
EthCapabilitiesResult record class If the nested types become structs, each CapabilityResource is ~40 bytes × 6 = ~240 bytes inline. A 256-byte struct is too large — copying it is worse than a single class allocation. It's the top-level container; one heap allocation is fine.
CapabilityHead readonly record struct Two string references = ~16 bytes. Stored inline in EthCapabilitiesResult, saves 1 heap allocation per call. Ideal struct size.
CapabilityResource readonly record struct Used 6× per result — saves 6 heap allocations per call. Bool + string ref + Nullable<CapabilityDeleteStrategy> lands at ~32–40 bytes, acceptable when stored as fields in a class (not passed around by value).
CapabilityDeleteStrategy readonly record struct String ref + long = ~16 bytes. Used as CapabilityDeleteStrategy? → becomes Nullable<CapabilityDeleteStrategy> inline in CapabilityResource — no heap allocation for the strategy object.

Net effect on allocations per eth_capabilities call:

  • Before: result + head + 6× resource + 0–1× deleteStrategy = 8–9 allocations
  • After structs on all nested types: result only = 1 allocation

One caveat on CapabilityDeleteStrategy?: When CapabilityDeleteStrategy is a struct, DeleteStrategy becomes Nullable<CapabilityDeleteStrategy>. [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] on a nullable value type works correctly in System.Text.Json — it omits the property when the nullable has no value.

Separate from the struct question — LukaszRozmej's inline comments also ask to keep Number as long and Hash as Hash256 (let the serializer format them, avoiding intermediate string allocations). That applies on top of these struct changes.

…g and Hash256 types

This commit updates the `EthCapabilitiesResult` and `EthCapabilitiesProvider` classes to replace string representations of block numbers and hashes with their appropriate types: `long` for block numbers and `Hash256` for block hashes. This change enhances type safety and consistency across the codebase. Additionally, the related tests have been updated to reflect these changes, ensuring accurate validation of capabilities reporting.
@divi2806
Copy link
Copy Markdown
Contributor Author

divi2806 commented May 1, 2026

Claude finished @LukaszRozmej's task in 1m 25s —— View job

Struct vs Class Analysis for EthCapabilitiesResult types

  • Read EthCapabilitiesResult.cs
  • Read EthCapabilitiesProvider.cs
  • Analyze performance tradeoffs
  • Post answer

Answer

Rule of thumb: a struct saves a heap allocation only when it's stored as a field in a class (it lives inline) or in a Nullable<T> slot. A struct that itself contains 6+ struct fields becomes too large to pass by value safely (guideline: keep structs ≤ 16–24 bytes).

Type Recommendation Reasoning
EthCapabilitiesResult record class If the nested types become structs, each CapabilityResource is ~40 bytes × 6 = ~240 bytes inline. A 256-byte struct is too large — copying it is worse than a single class allocation. It's the top-level container; one heap allocation is fine.
CapabilityHead readonly record struct Two string references = ~16 bytes. Stored inline in EthCapabilitiesResult, saves 1 heap allocation per call. Ideal struct size.
CapabilityResource readonly record struct Used 6× per result — saves 6 heap allocations per call. Bool + string ref + Nullable<CapabilityDeleteStrategy> lands at ~32–40 bytes, acceptable when stored as fields in a class (not passed around by value).
CapabilityDeleteStrategy readonly record struct String ref + long = ~16 bytes. Used as CapabilityDeleteStrategy? → becomes Nullable<CapabilityDeleteStrategy> inline in CapabilityResource — no heap allocation for the strategy object.
Net effect on allocations per eth_capabilities call:

  • Before: result + head + 6× resource + 0–1× deleteStrategy = 8–9 allocations
  • After structs on all nested types: result only = 1 allocation

One caveat on CapabilityDeleteStrategy?: When CapabilityDeleteStrategy is a struct, DeleteStrategy becomes Nullable<CapabilityDeleteStrategy>. [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] on a nullable value type works correctly in System.Text.Json — it omits the property when the nullable has no value.

Separate from the struct question — LukaszRozmej's inline comments also ask to keep Number as long and Hash as Hash256 (let the serializer format them, avoiding intermediate string allocations). That applies on top of these struct changes.

Changed to record class based on Claude Bot's analysis — the nested types (CapabilityHead, CapabilityResource, CapabilityDeleteStrategy) are all readonly record struct, so EthCapabilitiesResult contains ~240 bytes of inline struct data. Making the top-level container a struct would cause large copies on every pass-by-value. Keeping it as a record class gives 1 heap allocation instead of 8–9.

@divi2806 divi2806 requested a review from LukaszRozmej May 1, 2026 10:58
…avros

- Move OldestStateBlock off IBlockFinder: now declared on IBlockTree
  (writer side) and IWorldStateManager (external consumers). BlockTree
  implements a narrow IOldestStateBlockStore so WorldStateManager can
  delegate without taking a Nethermind.Blockchain dependency.
- Gate Stateproofs separately from State via IWorldStateManager.SupportsTrieProofs:
  trie-backed managers return true, FlatWorldStateManager returns false to
  avoid advertising O(state-size) or snapshot-limited proof reconstruction.
- Encapsulate the epoch->blocks conversion behind IHistoryPruner.GetRetentionBlocks;
  SlotsPerEpoch becomes private const again.
- Clarify XML doc: Tx/Logs aliasing Receipts (with LogIndex caveat),
  JsonPropertyOrder is example-payload parity, not a wire requirement.
- Replace Keccak.Zero in ExampleResponse with a realistic 32-byte hash.
- Add CapabilitiesScenario row covering AncientReceiptsBarrier > AncientBodiesBarrier.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@LukaszRozmej
Copy link
Copy Markdown
Member

@claude re-review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 12, 2026

Claude finished @LukaszRozmej's task in 5m 31s —— View job


Code Review (Pass 5)

  • Read relevant rules files (coding-style, robustness, di-patterns, performance)
  • Diff and read changed files since pass 4
  • Assess @asdacap's CHANGES_REQUESTED: move OldestStateBlock off IBlockFinder
  • Assess @svlachakis's 🔴 findings (Stateproofs + LogIndex)
  • Check all prior Low observations and 🟡 minor points
  • Post inline comment and final summary

Summary

Commit ac03884 addresses all outstanding review feedback from @asdacap and @svlachakis. Here is the complete status:

Finding Source Status
OldestStateBlock on IBlockFinder (wrong interface) @asdacap CHANGES_REQUESTED ✅ Fixed — moved to IBlockTree (writer side) + IWorldStateManager (consumer). IOldestStateBlockStore narrow interface carries the write path without a Nethermind.Blockchain dep into Nethermind.State.
Stateproofs aliased to state for flat-storage nodes @svlachakis 🔴 ✅ Fixed — IWorldStateManager.SupportsTrieProofs added. WorldStateManager returns true, FlatWorldStateManager returns false. Provider gates: stateproofs = worldStateManager.SupportsTrieProofs ? state : Disabled.
Tx/Logs alias Receipts, ignores LogIndex retention @svlachakis 🔴 ✅ Acknowledged — XML doc on EthCapabilities explains the invariant: log index is built from receipts so coverage equals receipt coverage; if index started late, eth_getLogs falls back to the receipt scanner. Comment captures the assumption explicitly. Full implementation deferred.
SlotsPerEpoch public solely for EthCapabilitiesProvider @svlachakis 🟡 ✅ Fixed — GetRetentionBlocks(long) added to IHistoryPruner; SlotsPerEpoch is private const again.
JsonPropertyOrder comment claimed wire contract @svlachakis 🟡 ✅ Fixed — comment now reads "matches the spec's example-payload key order for visual parity — not a wire requirement".
ExampleResponse uses Keccak.Zero @svlachakis 🟡 / pass 4 Low ✅ Fixed — now uses 0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3 (genesis hash).
Test gap: AncientReceiptsBarrier > AncientBodiesBarrier @svlachakis 🟡 ✅ Fixed — fast_sync_receipts_barrier_above_bodies_barrier scenario added.

Architecture is clean. The IOldestStateBlockStore narrow interface correctly prevents Nethermind.State from depending on Nethermind.Blockchain. BlockTreeModule.cs casts IBlockTree → IOldestStateBlockStore in DI; both WorldStateManager and FlatWorldStateManager take it as a constructor param. BlockTreeOverlay correctly falls back to _baseTree.OldestStateBlock on reads. BlockTree persists the value to the metadata DB and loads it on startup.

One remaining Low observation:

Severity Finding
Low No test exercises SupportsTrieProofs = false. Every scenario in CapabilitiesScenarios() and GetCaps hardcodes wsm.SupportsTrieProofs.Returns(true). The stateproofs = worldStateManager.SupportsTrieProofs ? state : Disabled branch added as the fix for the 🔴 flat-storage finding is never covered. A one-row addition to CapabilitiesScenarios with SupportsTrieProofs = false would close this. → inline

Verdict: ready to merge. No Critical, High, or Medium findings remain. The single Low observation is a test-coverage gap for the stateproofs flat-storage fix — worth adding before merge but not a blocker.


IWorldStateManager wsm = Substitute.For<IWorldStateManager>();
wsm.OldestStateBlock.Returns(oldestStateBlock);
wsm.SupportsTrieProofs.Returns(true);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — no test for SupportsTrieProofs = false (flat-storage stateproofs disabled)

The GetCaps helper hardcodes wsm.SupportsTrieProofs.Returns(true) for every scenario. The new Stateproofs: worldStateManager.SupportsTrieProofs ? state : Disabled path — the whole fix for @svlachakis's 🔴 finding — is never exercised by the test suite. A regression that swapped the conditional would go undetected.

Consider adding one scenario row where SupportsTrieProofs = false and asserting caps.Stateproofs == Disabled:

yield return new CapabilitiesScenario(
    Name: "flat_storage_disables_stateproofs",
    ExpectedState: Available, ExpectedStateproofs: Disabled,
    ExpectedReceipts: Available, ExpectedBlocks: Available)
{ SyncConfig = fullSync, SupportsTrieProofs = false };  // or pass as a flag

(You'd need to thread supportsTrieProofs through CapabilitiesScenario and GetCaps similarly to how HistoryPruner is passed.)

LukaszRozmej and others added 4 commits May 12, 2026 11:04
…StateManager

- Move the absolute oldest-state-block floor out of BlockTree entirely.
  WorldStateManager now owns the metadata-DB persistence (same key) via a
  small OldestStateBlockStore helper; FlatWorldStateManager does the same.
  IBlockTree/IBlockFinder no longer expose OldestStateBlock.
- Route FullPruner and StateSyncRunner writes through
  IWorldStateManager.OldestStateBlock (proper layering).
- FullPrunerFactory.Create now takes IWorldStateManager (not IStateReader)
  so the pruner can write the floor without going through the block tree.
- Make CopyTrie synchronous (it has no async work; was already returning
  Task.FromResult).
- Reformat multi-arg EthCapabilitiesProvider constructor calls one-per-line.
- Drop IOldestStateBlockStore glue interface no longer needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Flat storage retains trie nodes in path-keyed columns (see
BaseTriePersistence: StateNodesTop, StateNodes, StorageNodes,
FallbackNodes) and serves eth_getProof in O(trie-depth) for any block
where the flat snapshot is present — same complexity as trie storage.
The previous SupportsTrieProofs gate was based on a wrong assumption.
Drop the property; stateproofs availability matches state availability
for both Nethermind backends.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tartup

Adds a one-shot consistency check at world-state-manager activation:
- If OldestStateBlock points at a block whose state root is not present
  in the state DB, clear it.
- Same for IBlockTree.BestPersistedState.

This handles the scenario where a user wipes the state directory without
touching MetadataDb / BlockInfoDb — without the check, eth_capabilities
would over-report state availability until sync rewrites the floor.

The check uses GlobalStateReader.HasStateForBlock so it's meaningful
(verifies the root, not just "is the DB empty"). Wired via
OnActivate<IWorldStateManager> in both WorldStateModule and
FlatWorldStateModule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move StateMetadataValidator from Nethermind.Init to Nethermind.Blockchain
(where IBlockTree lives) so it can be tested in Nethermind.Blockchain.Test.
Use explicit BlockTreeLookupOptions overload so NSubstitute can intercept
the FindHeader call without going through the default interface method.

Tests cover:
- OldestStateBlock cleared when state root missing
- OldestStateBlock kept when state root present
- OldestStateBlock kept when header unknown (defensive)
- No-op when OldestStateBlock is null
- BestPersistedState clear/keep symmetric to OldestStateBlock
- Markers independent (one stale, other ok)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@LukaszRozmej
Copy link
Copy Markdown
Member

@claude review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 12, 2026

Claude finished @LukaszRozmej's task in 5m 33s —— View job


Code Review (Pass 6)

  • Read relevant rules files (coding-style, robustness, di-patterns)
  • Read changed files and git log since pass 5 (1bb0bcb, 4106a1e, b499b2d, cfd6015)
  • Assess @asdacap's latest CHANGES_REQUESTED (OldestStateBlock on IBlockFinder)
  • Review StateMetadataValidator and tests
  • Inspect OldestStateBlockStore null-delete behavior
  • Post inline comments and final summary

Summary

This is pass 6. Since pass 5 (four commits were merged):

Commit What changed
1bb0bcb OldestStateBlock removed from IBlockFinder/IBlockTree; WorldStateManager + FlatWorldStateManager own persistence via OldestStateBlockStore. FullPruner/StateSyncRunner write to worldStateManager.OldestStateBlock. IOldestStateBlockStore glue interface dropped.
4106a1e SupportsTrieProofs removed; Stateproofs: state alias restored with a comment explaining flat storage retains trie nodes in path-keyed columns and serves eth_getProof in O(trie-depth).
b499b2d StateMetadataValidator.DiscardStaleFloors added — startup check that clears OldestStateBlock and BestPersistedState when the recorded state root is no longer present on disk. Wired via OnActivate<IWorldStateManager> in both WorldStateModule and FlatWorldStateModule.
cfd6015 StateMetadataValidator moved from Nethermind.Init to Nethermind.Blockchain for testability; 7 unit tests added covering all code paths.

All outstanding findings from prior rounds are resolved:

Prior Finding Status
@asdacap CHANGES_REQUESTED: OldestStateBlock on IBlockFinder ✅ Fixed — moved to IWorldStateManager exclusively
Pass 5 Low: no test for SupportsTrieProofs = false ✅ N/A — SupportsTrieProofs removed entirely (4106a1e)
@svlachakis 🔴 stateproofs-vs-flat correctness ✅ Resolved — both backends serve trie proofs, comment documents reasoning

Architecture is clean. The OldestStateBlockStore ownership is well-placed in Nethermind.State with proper layering — IWorldStateManager exposes the property, FullPruner and StateSyncRunner write through it, and neither touches IBlockTree for state availability. The StateMetadataValidator startup guard is a sound defensive measure for the state-wipe scenario.

One new Low observation:

Severity Finding
Low OldestStateBlockStore setter doesn't delete the DB key when set to null. StateMetadataValidator correctly clears the in-memory value on each startup (functionally correct), but the DB key persists indefinitely — meaning after a state wipe the stale floor is reloaded and re-detected on every restart. BlockTree.BestPersistedState has the same pattern. Fix: add else metadataDb.Remove(MetadataDbKeys.OldestStateBlock). → inline

Verdict: ready to merge. No Critical, High, or Medium findings remain.

_value = value;
if (value.HasValue)
{
metadataDb.Set(MetadataDbKeys.OldestStateBlock, Rlp.Encode(value.Value).Bytes);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low — DB key is not deleted when set to null

IKeyValueStore.Remove(key) is defined as Set(key, null) — setting null deletes the entry in RocksDB. But the setter only writes when value.HasValue; when StateMetadataValidator calls worldStateManager.OldestStateBlock = null to clear a stale floor, the in-memory _value becomes null but the DB key persists. On the next startup the stale value is loaded again and the validator re-clears it each time.

BlockTree.BestPersistedState has the same pattern (no-delete-on-null), so this follows existing convention. Functionally correct: the validator catches the stale floor on every startup. Still worth fixing for consistency:

Suggested change
metadataDb.Set(MetadataDbKeys.OldestStateBlock, Rlp.Encode(value.Value).Bytes);
if (value.HasValue)
metadataDb.Set(MetadataDbKeys.OldestStateBlock, Rlp.Encode(value.Value).Bytes);
else
metadataDb.Remove(MetadataDbKeys.OldestStateBlock);

LukaszRozmej and others added 2 commits May 12, 2026 13:23
…leared

- Address claude-bot review (pass 6): OldestStateBlockStore setter now
  deletes the metadata DB entry on null, so a stale floor isn't reloaded
  on every restart after StateMetadataValidator clears it.
- Remove unused `using Nethermind.Blockchain.Find;` from
  StateMetadataValidator and its tests — BlockTreeLookupOptions lives in
  Nethermind.Blockchain. Fixes IDE0005 lint failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…stabilize retentionBlocks

- Block/Receipt resources are now Disabled while fast sync is configured to download but the
  descending pointer is still null. Reporting the barrier (eventual floor) as oldestBlock
  before any insertion over-claimed availability.
- Replace IWorldStateManager.GetOldestStateBlock(head) with RetentionWindowBlocks. Provider
  derives windowOldest itself and emits the configured retention so retentionBlocks stays
  correct when head < retention (fresh chain) instead of rounding down to head.
- Lock OldestStateBlockStore reads/writes: long? is two fields, the RPC path reads concurrently
  with sync runners and the full pruner.
- Add a regression-scenario test for the pre-batch fast-sync window and update existing
  scenarios to set descending pointers consistent with their post-sync framing.
@LukaszRozmej
Copy link
Copy Markdown
Member

@claude re-review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 12, 2026

Claude finished @LukaszRozmej's task in 5m 13s —— View job


Code Review (Pass 7)

  • Read relevant rules files (coding-style, robustness, di-patterns)
  • Diff and read changed files since pass 6 (b9f1947, 675346c)
  • Check pass 6 Low finding: OldestStateBlockStore null-delete behavior
  • Verify all prior findings resolved
  • Post inline comments and final summary

Summary

This is pass 7. Two commits landed since pass 6 (b9f1947 and 675346c). All findings from prior rounds are resolved, and the new commits introduce two additional improvements.

Pass 6 Low finding:

Prior Low Status
OldestStateBlockStore setter doesn't delete DB key when set to null ✅ Fixed — b9f1947 adds metadataDb.Delete(MetadataDbKeys.OldestStateBlock) in the null branch. Uses the C# 14 extension Delete(long key) => db.Remove(...) which correctly removes the entry from RocksDB.

New changes in 675346c:

Change Assessment
Block/Receipt resources now Disabled when fast-sync descending pointers are null (pre-first-batch) ✅ Correct fix — IsDescendingResourceDownloaded returns false when pointer is null with a non-zero pivot. Previously the config barrier would be reported as oldestBlock before any data was actually downloaded.
IWorldStateManager.GetOldestStateBlock(long) replaced by RetentionWindowBlocks { get; } property ✅ Cleaner API — separates the configured window size from the head - retention computation. BuildState now derives windowOldest itself; this also fixes retentionBlocks being rounded down to head when head < retention on a fresh chain.
OldestStateBlockStore reads/writes locked with System.Threading.Lock ✅ Correct fix — long? is not atomically read/written on 32-bit-wide slots; the RPC path reads concurrently with FullPruner and StateSyncRunner. Targets net10.0, so System.Threading.Lock is available. The early if (_value == value) return; guard is inside the lock, so it doesn't introduce a TOCTOU.
New test scenario fast_sync_before_first_batch_disables_blocks_and_receipts ✅ Covers the case where LowestInsertedBodyNumber = null with a non-zero pivot — all four resources correctly reported Disabled.

Verification of key logic paths:

  • Pre-batch with state not finalised: fastSyncing && OldestStateBlock is nullDisabled; pointers null → blocks/receipts Disabled. ✅
  • Post-sync with memory pruning dominating: windowOldest >= stateFloorDeleteStrategy emitted with configured retention, not clamped to current head. ✅
  • No-head path: ChainHead(0, Keccak.Zero) with all-Disabled returned. Schema-valid (the JSON schema test validates this case indirectly via the main scenario tests). ✅

Verdict: ready to merge. No Critical, High, or Medium findings remain. No new Low observations.

@svlachakis svlachakis requested a review from asdacap May 12, 2026 18:33
LukaszRozmej and others added 2 commits May 13, 2026 08:03
# Conflicts:
#	src/Nethermind/Nethermind.Runner/packages.lock.json
…tests

- Promote OldestStateBlockStore to a DI singleton (registered in BlockTreeModule
  via IDbProvider.MetadataDb). FullPruner, StateSyncRunner, EthCapabilitiesProvider,
  StateMetadataValidator now depend on the narrow store instead of IWorldStateManager.
  Removes OldestStateBlock from IWorldStateManager.
- Drop IDbProvider from FlatWorldStateManager (the store is injected directly).
- Extract MetadataLongStore reusable base class taking IDb + key; OldestStateBlockStore
  becomes a thin sealed wrapper hardcoding MetadataDbKeys.OldestStateBlock.
- IFullPrunerFactory.Create signature: IWorldStateManager → IStateReader (mirrors
  what FullPruner actually needs); factory pulls the store via DI.
- IsDescendingResourceDownloaded as single expression.
- Drop the stateproofs-aliases-state comment.
- Deduplicate StateMetadataValidatorTests with [TestCase] parameterization
  across (Marker × initial × StateAt × shouldClear); test count: 9 (8 cases + 1
  independent-markers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@LukaszRozmej
Copy link
Copy Markdown
Member

@claude review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 13, 2026

Claude finished @LukaszRozmej's task in 6m 50s —— View job


Code Review (Pass 8)

  • Read relevant rules files (coding-style, robustness, di-patterns)
  • Diff and read changed files since pass 7 (fabd507)
  • Verify all prior findings resolved
  • Check new changes
  • Post inline comments and final summary

Summary

This is pass 8. One commit landed since pass 7 (fabd507: refactor(eth_capabilities): narrow OldestStateBlockStore deps, dedup tests). All prior findings remain resolved.

What changed in fabd507:

Change Assessment
MetadataLongStore base class extracted — generic nullable-long persister over an IDb with a fixed int key ✅ Sound design — thread-safe, null → Delete(key), real implementation re-used in future
OldestStateBlockStore becomes a thin sealed wrapper: : MetadataLongStore(metadataDb, MetadataDbKeys.OldestStateBlock) ✅ Correct — one line, no logic duplication
Promoted to DI singleton in BlockTreeModule via IDbProvider.MetadataDb ✅ Single source of truth; eliminates the prior dual-instance risk (each of WorldStateManager/FlatWorldStateManager constructing its own store). FullPruner, StateSyncRunner, EthCapabilitiesProvider, StateMetadataValidator all receive the same instance
IWorldStateManager.OldestStateBlock { get; set; } removed ✅ Cleaner interface — rolling-window retention is all IWorldStateManager needs to expose
IFullPrunerFactory.Create(IStateReader, IPruningTrieStore) — narrowed from IWorldStateManager ✅ Matches actual use; FullPruner gets the store via DI, only needs IStateReader for CopyTrie
FlatWorldStateManager drops IDbProvider constructor param ✅ The store is injected; FlatWorldStateManager no longer manages persistence
StateMetadataValidator.DiscardStaleFloors takes OldestStateBlockStore + IStateReader separately ✅ Both callers (WorldStateModule, FlatWorldStateModule) use ctx.Resolve<OldestStateBlockStore>() and wsm.GlobalStateReader
StateMetadataValidatorTests deduplicated with [TestCase] (8 parameterized cases + 1 independent test) ✅ Uses a real OldestStateBlockStore(new MemDb()) — no mocking of DI types, follows anti-pattern guidance
FullPrunerTests / FullPruningDiskTest updated to pass new OldestStateBlockStore(new MemDb()) / Container.Resolve<OldestStateBlockStore>() ✅ Correct — Container is the full production-module Autofac container; BlockTreeModule registers the store
IsDescendingResourceDownloaded inlined as a single expression ✅ Logic-equivalent; easier to read

Architecture is clean. The ownership model is now unambiguous: one singleton OldestStateBlockStore written by FullPruner and StateSyncRunner, read by EthCapabilitiesProvider and validated at startup by StateMetadataValidator. IWorldStateManager retains only RetentionWindowBlocks for the rolling-window concept.

Verdict: ready to merge. No Critical, High, or Medium findings remain.

IBlockTree blockTree = ctx.Resolve<IBlockTree>();
ILogManager logManager = ctx.Resolve<ILogManager>();
new TrieStoreBoundaryWatcher(worldStateManager, blockTree, logManager);
StateMetadataValidator.DiscardStaleFloors(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is kind of deliberate.
Now we have in metadata db info that is related to state.
Now if someone does drop state db (like a directory) - which IMO is useful feature to force resync - this data becomes stale and misleading. So I added a quick check for it.
Alternatively I can put it into state db (metadata column?)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put it in trie world state factory I guess.


.Map<IWorldStateManager, PruningTrieStateFactoryOutput>((o) => o.WorldStateManager)
.OnActivate<IWorldStateManager>((wsm, ctx) =>
StateMetadataValidator.DiscardStaleFloors(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two time?

public sealed class EthCapabilitiesProvider(
IReadOnlyBlockTree blockTree,
IWorldStateManager worldStateManager,
OldestStateBlockStore oldestStateBlockStore,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide behind IWorldStateManager please. Implement separately for flatdb.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that was my previous iteration, but:

  • It living into metadata db caused issues and IMO wrong dependencies
  • WordStateManager needed to be injected in places where it kind of shouldn't

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe if I put it under a separate interface that IWorldStatemanager would inherit + living in a separate metadata column in StateDb would be optimal...?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For flat it does not even use metadata db, it has its own metadata db, which it can consult to report this. What kind of place the WorldStateManager should not be put? THen put it in IStateReader?

…-backend state-DB-resident storage

Addresses @asdacap review on `fabd507183`:
- Re-add `OldestStateBlock` (+ `RetentionWindowBlocks`) on a new narrow
  `IStateBoundary` interface; `IWorldStateManager : IStateBoundary`.
- Trie `WorldStateManager`: floor persists in `dbProvider.StateDb` under
  `Keccak("OldestStateBlock")` (collision-free vs 32-byte hash keys and
  HalfPath section bytes 0/1/2).
- `FlatWorldStateManager`: floor persists in `FlatDbColumns.Metadata` under
  the same keccak key, alongside the existing `CurrentState`/`Layout` keys.
- Co-locating the floor with state nodes means wiping the state directory
  drops the floor automatically — no startup validator needed.
- Drop `StateMetadataValidator` (+ tests), `OldestStateBlockStore`,
  `MetadataLongStore`, `MetadataDbKeys.OldestStateBlock` (the metadata-DB
  storage was never deployed), and the duplicated `OnActivate` validator
  wiring in both world-state modules.
- `FullPruner` / `StateSyncRunner` / `EthCapabilitiesProvider` depend on
  the narrow `IStateBoundary` (auto-resolved via `IWorldStateManager`).
- `IFullPrunerFactory.Create` signature returns to `IWorldStateManager`.
- Capabilities test mock simplifies to a single `IStateBoundary` substitute.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot removed the database label May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add eth_capabilities support

4 participants