Skip to content

fix: add BalanceAt mock to prevent balance monitor race#22041

Closed
Fletch153 wants to merge 1 commit intodevelopfrom
fix/evm-balance-monitor-race
Closed

fix: add BalanceAt mock to prevent balance monitor race#22041
Fletch153 wants to merge 1 commit intodevelopfrom
fix/evm-balance-monitor-race

Conversation

@Fletch153
Copy link
Copy Markdown
Collaborator

Summary

Adds a default BalanceAt mock expectation to NewEthMocksWithStartupAssertions, preventing a data race when the EVM balance monitor is enabled.

Root Cause

NewEthMocksWithStartupAssertions sets up common mock expectations but omits BalanceAt. When a test enables the balance monitor (default behavior), its background SleeperTask worker calls BalanceAt on the mock client. With no matching expectation, testify hits its "unexpected method call" error path, which calls callString()fmt.Sprintf("%#v", ctx) → reflect-reads the context. This races with concurrent context cancellation during test teardown.

Race stack trace (from CI)

Read by goroutine 3144:        // BalanceAt → mock.callString → fmt.Sprintf → reflect
  github.com/stretchr/testify/mock.callString()        mock.go:463
  github.com/stretchr/testify/mock.(*Mock).MethodCalled()  mock.go:507  ← ERROR PATH
  github.com/smartcontractkit/chainlink-evm/pkg/monitor.(*worker).checkAccountBalance()

Previous write by goroutine 2837:   // context cancel() during teardown
  sync/atomic.AddInt32()
  github.com/smartcontractkit/chainlink-common/pkg/services.StopRChan.CtxCancel.func1()

Note: mock.go:507 is the "method called over N times" / unexpected call error path — this only fires because there's no BalanceAt expectation at all.

Fix

One line: add BalanceAt with .Maybe() to the mock helper. This gives the balance monitor a matching expectation with unlimited repeatability, so callString() is never invoked.

Related

The EVM balance monitor's background worker calls BalanceAt on the eth
client. When this mock helper is used without a BalanceAt expectation
and the balance monitor is enabled, the call hits testify's unexpected
method error path which formats arguments via fmt.Sprintf("%#v", ctx).
This reflect-based read races with concurrent context cancellation
during test teardown.

Adding a default BalanceAt expectation with .Maybe() prevents the
unexpected call error path from firing.
@github-actions
Copy link
Copy Markdown
Contributor

👋 Fletch153, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

@github-actions
Copy link
Copy Markdown
Contributor

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@github-actions
Copy link
Copy Markdown
Contributor

✅ No conflicts with other open PRs targeting develop

@Fletch153
Copy link
Copy Markdown
Collaborator Author

Closing in favor of the upstream root-cause fix in chainlink-common: smartcontractkit/chainlink-common#1984

The catch-all BalanceAt mock in the shared helper would shadow test-specific BalanceAt expectations (testify matches in registration order). The chainlink-common fix eliminates the race at the source by replacing the CtxCancel goroutine with a read-only context — no test changes needed in this repo.

@Fletch153 Fletch153 closed this Apr 16, 2026
@cl-sonarqube-production
Copy link
Copy Markdown

@trunk-io
Copy link
Copy Markdown

trunk-io Bot commented Apr 16, 2026

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant