chore(ci,tooling): speed up json-loader job#3010
Conversation
Coverage was enabled for the `json-loader` job in ethereum#2975, but it still runs on the default Python 3.13, so `coverage` uses the slower C tracer. Pin the job to 3.14 like `fill`, where coverage.py auto-selects `sys.monitoring`.
The fixture-consuming `pytest` phase was pinned to `-n auto
--maxprocesses 6`, capping it at 6 workers, while the fill phase uses
`-n {{ xdist_workers }}` (`auto` in CI) and scales with the runner. The
cap carried over from the `tox` config in ethereum#2555. Use the same worker
setting so phase 2 is not throttled below the available cores.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3010 +/- ##
===================================================
- Coverage 93.20% 92.82% -0.39%
===================================================
Files 620 620
Lines 38777 36714 -2063
Branches 3342 3341 -1
===================================================
- Hits 36144 34081 -2063
Misses 1773 1773
Partials 860 860
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add `--durations=50` to both phases of the `json-loader` recipe (the `fill` step and the `pytest` step) so CI surfaces the slowest fills and fixture executions. The standalone `fill` recipe already reports these; this brings `json-loader` to parity and gives data to guide further trimming of the `eels_base_coverage` set.
`--dist=loadfile` pinned every test in a file to one worker, so the two `test_tools_new_fork.py::test_end_to_end` cases (~256s and ~203s) ran back-to-back on a single worker while the rest sat idle. Switch phase 2 to `--dist=loadgroup` and tag each fixtures file's cases with an `xdist_group` keyed by the file. Fixture cases keep file-locality (the `data` JSON cache parses once per file), while the ungrouped tooling tests distribute individually onto separate workers.
The previous commit added the `xdist_group` in a default-ordered `pytest_collection_modifyitems`, but xdist reads each item's group before that hook runs, so the marker was ignored and phase 2 still ran as plain `load` (fixtures scattered, re-parsing their JSON per case). Mark the hook `tryfirst=True` so the group is assigned first: fixture cases regroup by file (parsed once) and the slow tooling tests still spread across workers.
Reverts the `loadgroup` rebalance and its `tryfirst` follow-up. In CI the regrouping did not reliably split the two slow `new_fork` CLI tests: phase 2 stayed ~420-470s either way (monsters serialized under one layout, fixtures imbalanced by file-grouping under the other), within runner variance. Phase 2 returns to `--dist=loadfile`. The slow tooling tests are better moved out of json-loader entirely, tracked as a follow-up.
|
Hey @SamWilsn, thanks for the review here. I reverted the attempts to speed-up the json-loader job by splitting up the The |
|
@SamWilsn added an issue here: |
🗒️ Description
The
json-loaderjob takes ~23 min. This PR applies two small quick wins to reduce this:fill, socoverageusessys.monitoringinstead of the slower C tracer. Coverage was enabled forjson-loaderin fix(ci,tooling): include json-loader in coverage reports #2975, but the job still runs on the default 3.13, reintroducing the C-tracer overhead that chore(ci): add setup-uv action, bump default ci python to 3.14 #2310 removed elsewhere (see the regression tracked in py3 CI job timing regression (~18min → ~38min) after PR #2252 #2314).pytestphase with the same worker count as the fill phase (-n {{ xdist_workers }}, which isautoin CI). It was pinned to--maxprocesses 6, capping it at 6 workers instead of scaling with the runner. The cap carried over from thetoxconfiguration in feat(ci,tooling): replacetoxwithjust#2555.🔗 Related Issues or PRs
✅ Checklist
just statictype(scope):.Cute Animal Picture
🐱