Skip to content

indexer/beacon: fix finalize-epoch panic when canonicalBlocks is empty#710

Merged
barnabasbusa merged 2 commits into
masterfrom
bbusa/fix-finalize-epoch-empty-canonicals
May 21, 2026
Merged

indexer/beacon: fix finalize-epoch panic when canonicalBlocks is empty#710
barnabasbusa merged 2 commits into
masterfrom
bbusa/fix-finalize-epoch-empty-canonicals

Conversation

@barnabasbusa
Copy link
Copy Markdown
Collaborator

The crash

Observed in a local devnet run:

time="2026-05-21T11:26:21+02:00" level=error msg="uncaught panic in indexer.beacon.Indexer.runIndexerLoop subroutine: runtime error: index out of range [0] with length 0, stack: goroutine 70086 [running]:
runtime/debug.Stack()
	/opt/homebrew/Cellar/go/1.26.3/libexec/src/runtime/debug/stack.go:26 +0x64
github.com/ethpandaops/dora/indexer/beacon.(*Indexer).runIndexerLoop.func1()
	/Users/bbusa/Ethereum/dora/indexer/beacon/indexer.go:493 +0xb8
panic({0x105f82300?, 0x7e2c1f2499e0?})
	/opt/homebrew/Cellar/go/1.26.3/libexec/src/runtime/panic.go:860 +0x12c
github.com/ethpandaops/dora/indexer/beacon.(*Indexer).finalizeEpoch(0x7e2c19c40820, 0x8, {0xad, 0x6f, 0x89, 0xc0, 0xb0, 0xaf, 0x3e, 0x9a, ...}, ...)
	/Users/bbusa/Ethereum/dora/indexer/beacon/finalization.go:284 +0x3860
github.com/ethpandaops/dora/indexer/beacon.(*Indexer).processFinalityEvent(0x7e2c19c40820, 0x7e2c1f2c52d8)
	/Users/bbusa/Ethereum/dora/indexer/beacon/finalization.go:68 +0x3e0
github.com/ethpandaops/dora/indexer/beacon.(*Indexer).runIndexerLoop(0x7e2c19c40820)
	/Users/bbusa/Ethereum/dora/indexer/beacon/indexer.go:505 +0x298
created by github.com/ethpandaops/dora/indexer/beacon.(*Indexer).runIndexerLoop.func1 in goroutine 65681
	/Users/bbusa/Ethereum/dora/indexer/beacon/indexer.go:496 +0x164
" error="runtime error: index out of range [0] with length 0" service=cl-indexer

How I tracked it down

  1. The stack pointed at finalization.go:284, inside finalizeEpoch. The offending line was:

    indexer.epochCache.ensureEpochDependentState(epochStats, canonicalBlocks[0].Root)

    canonicalBlocks[0] on an empty slice is exactly the [0] with length 0 panic.

  2. Earlier in finalizeEpoch (lines 200-271), there's explicit handling for the case where the epoch has no canonical blocks: it walks backward through parent roots to find a dependentRoot and breaks out of the loop. Execution then continues into the block at line 279+ with canonicalBlocks still empty - which is the bug.

  3. Before adding a len(canonicalBlocks) > 0 guard or substituting dependentRoot, I checked what ensureEpochDependentState actually does with that root:

    func (cache *epochCache) ensureEpochDependentState(epochStats *EpochStats, firstBlockRoot phase0.Root) {
    	cache.cacheMutex.Lock()
    	defer cache.cacheMutex.Unlock()
    
    	if epochStats.dependentState != nil {
    		return
    	}
    
    	stateKey := getEpochStatsKey(epochStats.epoch, epochStats.dependentRoot)
    	epochState := cache.stateMap[stateKey]
    	if epochState == nil && !epochStats.ready {
    		epochState = newEpochState(epochStats.dependentRoot, epochStats.epoch)
    		cache.stateMap[stateKey] = epochState
    		cache.indexer.logger.Infof(... epochStats.dependentRoot.String())
    	}
    	// ...
    }

    firstBlockRoot is never referenced - the function uses epochStats.dependentRoot exclusively. The parameter is dead.

  4. grep shows four call sites, all just computing a block root inline that the callee ignores:

    • finalization.go:284 - canonicalBlocks[0].Root (the one that panicked)
    • pruning.go:120 - blocks[0].Root
    • indexer.go:467 - genesisBlock[0].Root
    • client.go:307 - currentBlock.Root

The fix

Drop the unused parameter from ensureEpochDependentState and remove the second argument from all four callers. This:

  • Fixes the panic - the offending canonicalBlocks[0].Root expression is gone.
  • Removes the same latent panic shape from pruning.go:120 (where blocks comes from a map iteration and could theoretically be empty under future refactors).
  • Is the cleanest patch I considered. The alternative - adding a len(canonicalBlocks) > 0 ? ... : dependentRoot guard - papers over the issue while still passing a value the callee discards.

5 files changed, 5 insertions, 5 deletions.

Test plan

  • go build ./... compiles.
  • Run dora against the devnet that triggered the original panic; confirm finalization no longer crashes on an epoch with no canonical blocks.
  • Confirm indexer continues normally past the previously-panicking epoch (epoch 8 in the reported stack).

The second argument was passed by all four callers but never read
inside the function (it uses epochStats.dependentRoot exclusively).
finalization.go called it with canonicalBlocks[0].Root, which panics
when the epoch has no canonical blocks - the empty-canonical-blocks
branch above sets dependentRoot via parent walk-back and falls through
to the call site without populating canonicalBlocks. Removing the dead
parameter fixes the panic and incidentally removes latent similar
indexing at the other call sites.
@barnabasbusa barnabasbusa enabled auto-merge May 21, 2026 09:40
@barnabasbusa barnabasbusa merged commit 9907be4 into master May 21, 2026
5 checks passed
@barnabasbusa barnabasbusa deleted the bbusa/fix-finalize-epoch-empty-canonicals branch May 21, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants