feat(StateDiffsWriter): replace StateComposition plugin (#11788)

* feat(StateDiffsWriter): replace StateComposition plugin

Replaces the in-process Nethermind.StateComposition plugin with a leaner
Nethermind.StateDiffsWriter plugin that writes per-block trie diffs to a
dedicated BlockDiffs column family. The full state-composition tracker
moves out of the engine into an external Go sidecar that consumes the CF
via RocksDB secondary mode.

Why
- StateComposition kept the codehash/slot-count tracker in process. At
  5x mainnet state it OOM-ed mid-scan and pinned the engine's heap with
  hundreds of millions of dict entries.
- The bootstrap-scan path was the hot spot. Moving it out of the engine
  lets the engine run lean while a separate sidecar does the scanning
  with whatever budget it needs (and can be restarted independently).
- StateDiffsWriter only emits a deterministic, append-only stream of
  trie diffs per block. No in-engine tracker state, no bootstrap scan.

Scope
- Add Nethermind.StateDiff.Core (shared trie-diff walker + records)
- Add Nethermind.StateDiffsWriter (plugin: per-block diff -> RocksDB CF)
- Add Nethermind.StateDiffsWriter.Test
- Remove Nethermind.StateComposition / Nethermind.StateComposition.Test
- Swap the registration in Runner/NethermindPlugins.cs
- Update Runner csproj project ref + plugin glob lists
- Update Nethermind.slnx folder layout

The matching sidecar consumer lives in
NethermindEth/eth-perf-research#77.

* chore(CODEOWNERS): track plugin rename

Mirror the directory swap in this PR. The CODEOWNERS entries pointed at the
removed plugin tree; repoint them at the new one.

* feat(StateDiffsWriter): per-block memtable flush for sidecar visibility

The bloating sidecar opens the BlockDiffs DB in RocksDB secondary mode
and calls TryCatchUpWithPrimary to pick up new writes. Secondary-mode
iterators read SSTs and the manifest — they cannot materialise the
primary's in-memory memtable. Without an explicit flush after each
WriteBlockDiff, new records sit in the memtable indefinitely and the
sidecar's tracker.lastBlock stalls, causing the orchestrator's strict
waitSensorForBlock to time out at 2 s per poll.

Flush the Default column family inside the per-block WriteLock so the
sequence write→flush is atomic w.r.t. the pruner. RocksDB flush of a
sub-KB memtable is one small SST write — cheap enough at our cadence
(~3 blocks/s).

A WAL-only sync (Flush(onlyWal: true)) was tried and reverted on
2026-05-27: secondary iterators still couldn't read memtable contents,
so the 3 s sensor-lag ceiling persisted. Full memtable flush remains
the minimum viable visibility primitive.

* chore(Runner): regenerate packages.lock.json for StateDiffsWriter projects

* style(StateDiffsWriter): remove unused usings (IDE0005)

* docs(StateDiffsWriter): drop useless ISortedKeyValueStore using-comment

The 3-line note explaining why there is no separate using for
ISortedKeyValueStore (it lives in Nethermind.Core, already imported) is pure
trivia between the usings and the namespace. Remove it.

* docs(StateDiffsWriter): drop stale reference to deleted StateComposition.Service

This PR deletes Nethermind.StateComposition, so 'Matches the legacy
StateComposition.Service approach' points at code that no longer exists and
will mislead the next reader. Keep the scope-per-side explanation, drop the
dangling reference.

* chore(StateDiffsWriter): port to master RlpReader/RlpWriter API and ulong block numbers

Rebasing onto master surfaced two breaking API changes that the plugin
must follow to compile:

- RLP serialization moved from RlpStream/Rlp.ValueDecoderContext to the
  generic RlpReader/RlpWriter (IRlpWriteBackend) surface. Port the
  BlockDiffRecord encoder/decoder, the store's write/read paths, and the
  back-compat decoder tests. Wire bytes are unchanged (the byte-identical
  and legacy round-trip tests still pass), so the Go sidecar is unaffected.
- Block.Number is now ulong; cast to long at the NM-API boundary where it
  feeds the long-typed BlockDiffRecord/metrics/pruner domain (block numbers
  are < 2^63, so the narrowing is safe).

* fix(StateDiffsWriter): address PR review feedback

Resolves the seven review findings on #11788:

- High  — reorg/SlotCounts: detect a new head that does not build on the
  last-written block (DetectReorg) and surface it via a warning plus the new
  StateDiffsWriterReorgsTotal counter. Document that OldCount is advisory across
  a reorg; a per-height rollback is intentionally not implemented (the sidecar
  re-derives running totals from the diff chain, the bloatnet is reorg-free,
  PoS mainnet reorgs are single-slot).
- Medium — code-missing: extract ResolveNewCodeSize; when an account reports a
  code hash but the code DB has no bytes, record 0 and emit a warning + the
  code_missing labeled counter instead of undercounting silently.
- Medium — negative KeepLastNBlocks: guard PruneOnce so a negative window
  disables pruning (and warns) instead of wrapping into a delete-everything
  cutoff; document the >= 0 contract on the config item.
- Low — pruner shutdown: implement IAsyncDisposable (await the loop) and make
  Dispose a non-blocking best-effort cancel, dropping the up-to-5s Task.Wait.
- Low — Dispose idempotency: guard DiffsWriterService.Dispose with a _disposed
  flag so double-dispose unsubscribes once.
- Low — GetSlotCount allocation: read into a stack buffer via the span Get
  overload, removing the per-lookup byte[] in the block-write path.
- Low — walker isolation: allocate a fresh TrieDiffWalker per ComputeRecord
  instead of sharing a mutable instance field.

Adds tests for reorg detection, code-size resolution (present/missing/none),
negative-keep pruning, idempotent dispose, and async dispose.

* style(StateDiffsWriter): minimise comments to why-only

Per repo rule "comments explain why, not what": delete comments that restate
the code, drop project-internal jargon (an external consumer's version label,
project phases) that means nothing upstream, and hold every remaining comment
to 1-2 lines. Kept: SPDX headers, [Description]/[ConfigItem] metadata, the
BlockDiffRecord RLP wire-format layout (a cross-process contract), one-line XML
summaries on public APIs, and genuine non-obvious why-notes.

* docs(StateDiffsWriter): note why FlushDefault is a full flush, not WAL-only

A secondary-mode reader's iterator surfaces only flushed SSTs, not WAL/memtable
rows, so Flush(onlyWal: true) would leave the newest block invisible to the
consumer until it reached an L0 SST. The full flush is deliberate.

Author: AnkushinDaniil

.github/CODEOWNERS

@@ -38,8 +38,9 @@
 /src/Nethermind/Nethermind.Sockets @LukaszRozmej @benaadams
 /src/Nethermind/Nethermind.Specs @smartprogrammer93
 /src/Nethermind/Nethermind.State @LukaszRozmej @benaadams @asdacap @flcl42 @damian-orzechowski
-/src/Nethermind/Nethermind.StateComposition @daniilankusin
-/src/Nethermind/Nethermind.StateComposition.Test @daniilankusin
+/src/Nethermind/Nethermind.StateDiff.Core @daniilankusin
+/src/Nethermind/Nethermind.StateDiffsWriter @daniilankusin
+/src/Nethermind/Nethermind.StateDiffsWriter.Test @daniilankusin
 /src/Nethermind/Nethermind.Stateless* @rubo @LukaszRozmej
 /src/Nethermind/Nethermind.Synchronization @LukaszRozmej @benaadams @asdacap @flcl42 @marcindsobczak
 /src/Nethermind/Nethermind.Taiko @smartprogrammer93 @dipkakwani

src/Nethermind/Nethermind.Runner/Nethermind.Runner.csproj

@@ -80,7 +80,8 @@
     <ProjectReference Include="..\Nethermind.Synchronization\Nethermind.Synchronization.csproj" />
     <ProjectReference Include="..\Nethermind.UPnP.Plugin\Nethermind.UPnP.Plugin.csproj" />
     <ProjectReference Include="..\Nethermind.Xdc\Nethermind.Xdc.csproj" />
-    <ProjectReference Include="..\Nethermind.StateComposition\Nethermind.StateComposition.csproj" />
+    <ProjectReference Include="..\Nethermind.StateDiff.Core\Nethermind.StateDiff.Core.csproj" />
+    <ProjectReference Include="..\Nethermind.StateDiffsWriter\Nethermind.StateDiffsWriter.csproj" />
   </ItemGroup>
 
   <ItemGroup>
@@ -113,8 +114,8 @@
 
   <Target Name="CollectPlugins" AfterTargets="AfterBuild;AfterPublish">
     <ItemGroup>
-      <PluginsForBuild Include="$(OutputPath)\Nethermind.Merge.AuRa.*;$(OutputPath)\Nethermind.Shutter.*;$(OutputPath)\Nethermind.Merge.Plugin.*;$(OutputPath)\Nethermind.Consensus.AuRa.*;$(OutputPath)\Nethermind.Init.*;$(OutputPath)\Nethermind.HealthChecks.*;$(OutputPath)\Nethermind.Api.*;$(OutputPath)\Nethermind.EthStats.*;$(OutputPath)\Nethermind.JsonRpc.TraceStore.*;$(OutputPath)\Nethermind.Init.Snapshot.*;$(OutputPath)\Nethermind.Optimism.*;$(OutputPath)\Nethermind.ExternalSigner.Plugin.*;$(OutputPath)\Nethermind.Taiko.*;$(OutputPath)\Nethermind.Flashbots.*;$(OutputPath)\Nethermind.StateComposition.*;$(OutputPath)\Nethermind.OpcodeTracing.Plugin.*" />
-      <PluginsForPublish Include="$(OutputPath)\Nethermind.Merge.AuRa.dll;$(OutputPath)\Nethermind.Shutter.dll;$(OutputPath)\Nethermind.Merge.Plugin.dll;$(OutputPath)\Nethermind.Consensus.AuRa.dll;$(OutputPath)\Nethermind.Init.dll;$(OutputPath)\Nethermind.HealthChecks.dll;$(OutputPath)\Nethermind.Api.dll;$(OutputPath)\Nethermind.EthStats.dll;$(OutputPath)\Nethermind.JsonRpc.TraceStore.dll;$(OutputPath)\Nethermind.Init.Snapshot.dll;$(OutputPath)\Nethermind.Optimism.dll;$(OutputPath)\Nethermind.ExternalSigner.Plugin.dll;$(OutputPath)\Nethermind.Taiko.dll;$(OutputPath)\Nethermind.Flashbots.dll;$(OutputPath)\Nethermind.StateComposition.dll;$(OutputPath)\Nethermind.OpcodeTracing.Plugin.dll" />
+      <PluginsForBuild Include="$(OutputPath)\Nethermind.Merge.AuRa.*;$(OutputPath)\Nethermind.Shutter.*;$(OutputPath)\Nethermind.Merge.Plugin.*;$(OutputPath)\Nethermind.Consensus.AuRa.*;$(OutputPath)\Nethermind.Init.*;$(OutputPath)\Nethermind.HealthChecks.*;$(OutputPath)\Nethermind.Api.*;$(OutputPath)\Nethermind.EthStats.*;$(OutputPath)\Nethermind.JsonRpc.TraceStore.*;$(OutputPath)\Nethermind.Init.Snapshot.*;$(OutputPath)\Nethermind.Optimism.*;$(OutputPath)\Nethermind.ExternalSigner.Plugin.*;$(OutputPath)\Nethermind.Taiko.*;$(OutputPath)\Nethermind.Flashbots.*;$(OutputPath)\Nethermind.StateDiff.Core.*;$(OutputPath)\Nethermind.StateDiffsWriter.*;$(OutputPath)\Nethermind.OpcodeTracing.Plugin.*" />
+      <PluginsForPublish Include="$(OutputPath)\Nethermind.Merge.AuRa.dll;$(OutputPath)\Nethermind.Shutter.dll;$(OutputPath)\Nethermind.Merge.Plugin.dll;$(OutputPath)\Nethermind.Consensus.AuRa.dll;$(OutputPath)\Nethermind.Init.dll;$(OutputPath)\Nethermind.HealthChecks.dll;$(OutputPath)\Nethermind.Api.dll;$(OutputPath)\Nethermind.EthStats.dll;$(OutputPath)\Nethermind.JsonRpc.TraceStore.dll;$(OutputPath)\Nethermind.Init.Snapshot.dll;$(OutputPath)\Nethermind.Optimism.dll;$(OutputPath)\Nethermind.ExternalSigner.Plugin.dll;$(OutputPath)\Nethermind.Taiko.dll;$(OutputPath)\Nethermind.Flashbots.dll;$(OutputPath)\Nethermind.StateDiff.Core.dll;$(OutputPath)\Nethermind.StateDiffsWriter.dll;$(OutputPath)\Nethermind.OpcodeTracing.Plugin.dll" />
     </ItemGroup>
   </Target>
 

src/Nethermind/Nethermind.Runner/NethermindPlugins.cs

@@ -28,6 +28,6 @@ public static class NethermindPlugins
         typeof(Nethermind.UPnP.Plugin.UPnPPlugin),
         typeof(Nethermind.Xdc.XdcPlugin),
         typeof(Nethermind.Xdc.XdcSubnetPlugin),
-        typeof(Nethermind.StateComposition.StateCompositionPlugin),
+        typeof(Nethermind.StateDiffsWriter.StateDiffsWriterPlugin),
     ];
 }

src/Nethermind/Nethermind.Runner/packages.lock.json

@@ -1140,12 +1140,23 @@
           "System.IO.Hashing": "[10.0.9, )"
         }
       },
-      "nethermind.statecomposition": {
+      "nethermind.statediff.core": {
+        "type": "Project",
+        "dependencies": {
+          "Nethermind.Core": "[1.39.0-unstable, )",
+          "Nethermind.Serialization.Rlp": "[1.39.0-unstable, )",
+          "Nethermind.Trie": "[1.39.0-unstable, )"
+        }
+      },
+      "nethermind.statediffswriter": {
         "type": "Project",
         "dependencies": {
           "Nethermind.Api": "[1.39.0-unstable, )",
           "Nethermind.Db": "[1.39.0-unstable, )",
+          "Nethermind.Db.Rocks": "[1.39.0-unstable, )",
           "Nethermind.Init": "[1.39.0-unstable, )",
+          "Nethermind.State.Flat": "[1.39.0-unstable, )",
+          "Nethermind.StateDiff.Core": "[1.39.0-unstable, )",
           "Nethermind.Trie": "[1.39.0-unstable, )"
         }
       },