Skip to content

chore(ci): speed up json-loader and fill jobs#3096

Open
spencer-tb wants to merge 2 commits into
ethereum:forks/amsterdamfrom
spencer-tb:ci/json-loader-speedup
Open

chore(ci): speed up json-loader and fill jobs#3096
spencer-tb wants to merge 2 commits into
ethereum:forks/amsterdamfrom
spencer-tb:ci/json-loader-speedup

Conversation

@spencer-tb

@spencer-tb spencer-tb commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

🗒️ Description

Speed up the per-PR CI by:

  • Moving the new-fork CLI and linter tool tests out of the json-loader job (into tests/evm_tools, run by test-tests)
  • Switch coverage to the sys.monitoring core
  • Fill "one" fixture format per test in the fill jobs.

json-loader drops from ~7m to ~4m30s (was the current bottleneck) and fill (amsterdam) from ~5m30s to ~4m (the next bottleneck).

🔗 Related Issues or PRs

Additionally fixes #3022.

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Verifying Changes:
    just static
  • All: PR title have the form <type>(<area>):, where <type> and <area> come from an approrpriate C-<type>, respectively A-<area>, label. The title should match the a target squash commit message.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.30%. Comparing base (64ed676) to head (d30dcfa).
⚠️ Report is 10 commits behind head on forks/amsterdam.

Additional details and impacted files
@@                 Coverage Diff                 @@
##           forks/amsterdam    #3096      +/-   ##
===================================================
+ Coverage            93.23%   93.30%   +0.06%     
===================================================
  Files                  624      624              
  Lines                36924    36986      +62     
  Branches              3373     3383      +10     
===================================================
+ Hits                 34427    34510      +83     
+ Misses                1704     1694      -10     
+ Partials               793      782      -11     
Flag Coverage Δ
unittests 93.30% <ø> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@spencer-tb spencer-tb force-pushed the ci/json-loader-speedup branch from 46ad799 to a4efbc7 Compare July 3, 2026 15:47
@spencer-tb spencer-tb changed the title chore(ci): move new-fork tool tests off json-loader, enable sysmon coverage chore(ci): speed up json-loader and fill jobs Jul 3, 2026
@spencer-tb spencer-tb added C-chore Category: chore A-ci Area: Continuous Integration labels Jul 3, 2026
@spencer-tb spencer-tb requested a review from danceratopz July 3, 2026 15:53
@spencer-tb spencer-tb force-pushed the ci/json-loader-speedup branch from cfde400 to d30dcfa Compare July 3, 2026 16:15
@spencer-tb spencer-tb marked this pull request as ready for review July 3, 2026 16:22
Comment thread Justfile
[group('unit tests')]
test-tests *args:
@mkdir -p "{{ output_dir }}/test-tests/tmp"
@mkdir -p "{{ output_dir }}/test-tests/tmp" "{{ output_dir }}/test-tests/tmp-evm-tools"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth defining a Just variable for the temp dir, or better yet, use something like recipe_name to factor out making temp dirs from all the recipes.

-m "not slow and (blockchain_test or state_test_only or blockchain_test_engine_only)"
env:
PYTEST_XDIST_AUTO_NUM_WORKERS: auto
COVERAGE_CORE: sysmon

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you shouldn't define this in the Justfile? Seems like it would be just as useful for local builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci Area: Continuous Integration C-chore Category: chore

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move new-fork CLI tests out of the json-loader job

2 participants