go/storage/mkvs/db/pathbadger: Fix pruning performance

[martintomazic]

Closes #6334.

Benchmarks

On the baremetal instance:

• 20x speed-up with 100k versions locally.
• With more versions performance gain increases:
  • E.g. 3.2M versions (see Sapphire snapshot) were pruned in around 7min. Using the old code it takes 20s to prune just one version!

Sanity: Runtime state pruning is now as efficient as consensus state pruning even though runtime state is 1000x bigger than the consensus state (looking at the checkpoint size). This still makes sense given that IO state is minimal compared to the EVM state, and that EVM state pruning is only about dropping writelogs and increasing the discard timestamp. The work was already partly done during commit operation by marking the nodes as ready to be removed and we still have a pending compaction (disk not reclaimed automatically).

Considerations:

• For NoChildRoots: true (IO roots), at version N we should eagerly delete all nodes in N-1 so that we have better locality and thus less compaction overhead (we are unknowingly using this trick for the state root already).
  • Given that IO state is minimal probably not critical, but we would have to bench compaction after the pruning to see the difference.

Implications/follow-up

• All pruning issues we had are fixed.
  • This includes combination of syncing and pruning being incredibly slow with prune operation dominating the syncing time.
• For the follow-up I will include runtime state pruning and compaction to storage prune/compact commands.
  • The workaround for the snapshot creation with exact start version using storage checkpoint create/import will most likely be no longer needed

[netlify]

✅ Deploy Preview for oasisprotocol-oasis-core canceled.

Name	Link
🔨 Latest commit	31648e6
🔍 Latest deploy log	https://app.netlify.com/projects/oasisprotocol-oasis-core/deploys/69f2804e589ec400081637c6

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.36%. Comparing base (7db1ccb) to head (31648e6).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6518      +/-   ##
==========================================
- Coverage   63.86%   63.36%   -0.51%     
==========================================
  Files         698      698              
  Lines       68289    68297       +8     
==========================================
- Hits        43615    43275     -340     
- Misses      19713    20052     +339     
- Partials     4961     4970       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[peternose]

Nice 👍