fix(traces): retry Cloudflare blocks instead of aborting identification#14981
Open
0xMars42 wants to merge 1 commit into
Open
fix(traces): retry Cloudflare blocks instead of aborting identification#149810xMars42 wants to merge 1 commit into
0xMars42 wants to merge 1 commit into
Conversation
The trace ExternalIdentifier fetches contract metadata through a rate-limit aware stream. RateLimitExceeded is handled with backoff and re-queue, but BlockedByCloudflare was treated like an invalid API key: it set the invalid-key flag and returned Poll::Ready(None), ending the stream and abandoning every address still queued. Those contracts never get sources or source maps, so cast run --decode-internal decodes selectors only partially and needs several runs to fill in from the on-disk cache. A Cloudflare block is transient rate limiting, not a permanent failure. Treat it like RateLimitExceeded (back off and retry), bounded by MAX_CLOUDFLARE_RETRIES so a persistent block can't loop forever; after the bound the address is given up individually instead of aborting the whole stream. InvalidApiKey still aborts. Adds a regression test that is red without the fix. Fixes foundry-rs#9880.
0xMars42
requested review from
0xrusowsky,
DaniPopes,
figtracer,
grandizzy,
mablr,
mattsse and
stevencartavia
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Addresses #9880. Under load,
cast run --decode-internal(and other tracing thatresolves verified sources) identifies contracts only partially and needs several
runs to produce a complete trace, filling in from the on-disk cache between runs.
Solution
The trace
ExternalIdentifierfetches contract metadata from Etherscan / Sourcifythrough a rate-limit-aware stream (
ExternalFetcher);cast run --decode-internalthen compiles those sources into source maps (
get_compiled_contracts), so acontract whose metadata never arrives is never decoded.
RateLimitExceededis already handled with backoff and re-queue, but aBlockedByCloudflarewas routed into theInvalidApiKeyhandling (it set thefetcher's invalid-key flag and returned
Poll::Ready(None), carrying the same// mark key as invalidcomment). That classifies a transient block as apermanent auth failure: it ends the stream and abandons every address still queued
in that pass, and disables the fetcher for the rest of the process. Etherscan sits
behind Cloudflare, which returns this when a request burst trips its protection, so
a single transient block stops identification for every remaining contract. The
on-disk cache is why re-running fills the gaps progressively.
This treats a Cloudflare block as the transient rate limiting it is: back off and
retry the address, bounded by
MAX_CLOUDFLARE_RETRIES(5) so a persistent blockcan't hang the command in an infinite loop. After the bound is reached the address
is given up individually (yielded as
None) instead of aborting the whole stream,so one blocked contract no longer prevents the rest from being identified.
InvalidApiKeyis left unchanged and still aborts, since retrying a bad key ispointless.
Additional context
Adds a regression test (
cloudflare_block_retries_instead_of_abandoning_queue)with a fetcher that returns a transient Cloudflare block on first contact with each
address and then succeeds. Before the change the stream aborts on the first block
and yields nothing; after it, every address is yielded in a single run. The test is
deterministic and does not hit the network.