fix(ci): fix flaky_testdata vyper regression and p256 snapshot#14278
Merged
Conversation
- Add Vyper installation to test-isolate.yml flaky job (was missing, causing 'Found Vyper sources, but no compiler versions are available' since the flaky_testdata test was added in #14052) - Wildcard ambiguous selector names and gas values in flaky_osaka_can_run_p256_precompile snapshot (the OpenChain 4byte API returns non-deterministic ordering for colliding selectors, and gas values shifted after the generic cast run refactor in #14121) Co-Authored-By: zerosnacks <95942363+zerosnacks@users.noreply.github.com>
decofe
requested review from
0xrusowsky,
DaniPopes,
figtracer,
grandizzy,
mablr,
mattsse,
onbjerg,
stevencartavia and
zerosnacks
as code owners
Same missing step as test-isolate.yml — the flaky_testdata test needs vyper to compile .vy files in the testdata directory. Co-Authored-By: zerosnacks <95942363+zerosnacks@users.noreply.github.com>
zerosnacks
approved these changes
Apr 13, 2026
mablr
approved these changes
Apr 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix two persistent nightly CI failures that have been red since April 2.
Motivation
Both
test-isolateandtest-flakynightly workflows have been failing for 12 days straight. Two tests fail deterministically every run.Changes
flaky_testdata(failing since April 2)The
flakyjobs in bothtest-isolate.ymlandtest-flaky.ymlwere missing the Vyper installation step thattest.ymlhas. Whenflaky_testdatarunsforge test --mc=Issue4640Testagainst the testdata directory (which contains.vyfiles), forge fails withFound Vyper sources, but no compiler versions are available for it. No other workflows are affected —ci.ymldelegates totest.ymlwhich already installs Vyper.Fix: add
setup-python+pip install vyper==0.4.3to both flaky jobs, matchingtest.yml.flaky_osaka_can_run_p256_precompile(failing since April 8)Two issues caused the snapshot mismatch:
Function name non-determinism: The OpenChain 4byte API returns non-deterministic ordering for colliding selectors (
selfCallPayVerifyCall537021665vsfulfillBasicOrder_efficient_6GL6yc). Fix: wildcard the ambiguous selector names with[..].P256VERIFY gas correctness fix: The generic
cast runrefactor in refactor(cast): genericcast run#14121 changedset_spec()→set_spec_and_mainnet_gas_params(), which correctly applies Osaka gas pricing for P256VERIFY (3450gas instead of the previous incorrect6900). The old code only set the spec but leftgas_paramsat their default, causing the precompile to charge double its correct cost. This is NOT a regression — it's a bug fix. The 3450 delta cascades through all parent calls. Fix: wildcard the affected gas values since the test is inherently flaky (external RPC + selector API).The P256VERIFY precompile trace lines (sha256, P256VERIFY call, return value) remain exact — this is the core assertion the test verifies.
Testing
Traced the failure history across 12 nightly runs and bisected both regressions to their introducing commits. Verified all other workflows (
ci.yml,crate-checks.yml, etc.) are unaffected.Prompted by: zerosnacks