fix: close head tracker/broadcaster before balance monitor to prevent race

[Fletch153]

Summary

Fixes the root cause of a long-standing data race in go_core_race_tests by reordering chain.Close() to stop event sources before event consumers.

Root Cause

chain.Close() stopped the balance monitor (event consumer) while the headTracker and headBroadcaster (event sources) were still running. This allowed a last-second OnNewLongestChain callback to wake the balance monitor's SleeperTask during shutdown:

Before (broken order):

1. logPoller.Close()
2. balanceMonitor.Close()    ← consumer stopped
3. logBroadcaster.Close()
4. headTracker.Close()       ← source still running at step 2!
5. headBroadcaster.Close()   ← source still running at step 2!
6. txm.Close()

The race sequence:

1. chain.Close() calls balanceMonitor.Close() → sleeperTask.Stop() → closes chStop
2. Meanwhile, headBroadcaster (still alive) delivers OnNewLongestChain → sleeperTask.WakeUp() → queues work in chQueue
3. In workerLoop's select, both chStop and chQueue are ready — Go picks randomly
4. 50% chance: picks chQueue, runs one more Work(ctx) → calls BalanceAt on the mock client
5. Concurrently, CtxCancel goroutine fires cancel() on the context
6. testify's callString() reads context via fmt.Sprintf("%#v", ctx) while cancel() writes → DATA RACE

Fix

Move headTracker and headBroadcaster shutdown before balanceMonitor:

After (correct order):

1. logPoller.Close()
2. logBroadcaster.Close()
3. headTracker.Close()       ← sources stopped FIRST
4. headBroadcaster.Close()   ← sources stopped FIRST
5. balanceMonitor.Close()    ← consumer stopped safely
6. txm.Close()

No new heads can arrive → no OnNewLongestChain → no last-second WakeUp() → no race.

Also fixed merr = c.balanceMonitor.Close() → merr = multierr.Combine(merr, ...) which was silently dropping prior errors.

Evidence

• Failing CI: https://github.com/smartcontractkit/chainlink/actions/runs/24509952973/job/71638172320
• Job: Core Tests (go_core_race_tests), test: TestJobsController_Index_HappyPath (leaked goroutine from earlier test)
• Race between monitor.(*worker).checkAccountBalance and services.StopRChan.CtxCancel.func1
• Previous fix attempt (core/internal/testutils/evmtest: run service to ensure closure chainlink#21397) addressed test mock setup but not the shutdown ordering

[github-actions]

👋 Fletch153, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

[github-actions]

📊 API Diff Results

No changes detected for module github.com/smartcontractkit/chainlink-evm

View full report