Skip to content

perf: lazy-load heavy deps to speed up CLI startup#142

Open
lwalew wants to merge 1 commit into
developfrom
perf/lazy-cli-imports
Open

perf: lazy-load heavy deps to speed up CLI startup#142
lwalew wants to merge 1 commit into
developfrom
perf/lazy-cli-imports

Conversation

@lwalew

@lwalew lwalew commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Why

mlipaudit -h takes ~6s. The cost is entirely import time, not argument parsing: main.py imports mlipaudit.benchmarks (needed to build the --benchmarks choices for the parser), and that transitively pulls in the full ML/plotting stack just to print a help string:

Chain ~cost pulled in via
mlip.models (ESEN network, flax) ~2.2s Benchmark base / ForceField type hints
mlip.simulation.asejax_md.rigid_body ~1.5s utils.simulation engine
sklearn.metrics ~0.7s conformer / dihedral / water benchmarks
bare import jax ~0.5s utils.stability

None of this is needed to list benchmarks or print help — only to actually run a benchmark.

What

Defer the heavy imports to call time:

  • benchmark.py, utils/{inference,simulation,stability,unallowed_elements}.py: move mlip.models / mlip.simulation.ase (JAX-MD) / jax / sklearn imports into the functions that use them; type-only references go under TYPE_CHECKING with from __future__ import annotations.
  • utils/_ase_engine.py (new): ASESimulationEngineWithCalculator subclasses the heavy ASESimulationEngine, so a class definition can't be lazy. Moved it to its own module imported lazily when a simulation runs; the historical mlipaudit.utils[.simulation].ASESimulationEngineWithCalculator paths still work via module-level __getattr__.
  • nudged_elastic_band, conformer_selection, dihedral_scan, water_radial_distribution: defer their mlip / sklearn imports into the run/analyze methods.
  • main.py: import launch_app / run_benchmarks inside their command branches.

Tests

The benchmark tests patched the now-lazy symbols at the use-site module (e.g. mlipaudit.utils.simulation.JaxMDSimulationEngine), which no longer exists there. Updated them to patch at the definition site (e.g. mlip.simulation.jax_md.JaxMDSimulationEngine) — the lazy from ... import picks the patch up at call time, and create_autospec in conftest (which binds the real class at import) is unaffected.

Result

  • mlipaudit -h: ~5.6s → ~0.9s
  • import mlipaudit.benchmarks: ~5.0s → ~1.1s
  • All 128 tests pass; ruff / ruff-format / mypy clean.

Notes

  • mlipaudit benchmark -h has a pre-existing argparse AssertionError (unrelated to this change; reproduces on main) and is left as-is.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jun 24, 2026

Copy link
Copy Markdown

Coverage

Coverage Report
FileStmtsMissCoverMissing
src/mlipaudit
   __init__.py20100% 
   benchmark.py78791%154, 173, 239, 245–246, 256, 270
   exceptions.py20100% 
   io.py1002575%117, 121, 146–150, 152, 154, 171–172, 174–175, 192–193, 195, 213–221
   io_helpers.py520100% 
   run_mode.py50100% 
   scoring.py29775%101–102, 104, 107, 111–112, 114
src/mlipaudit/benchmarks
   __init__.py290100% 
src/mlipaudit/benchmarks/bond_length_distribution
   bond_length_distribution.py95396%208, 211, 276
src/mlipaudit/benchmarks/conformer_selection
   conformer_selection.py132596%253, 256, 304, 309, 349
src/mlipaudit/benchmarks/dihedral_scan
   dihedral_scan.py116595%221, 224, 258, 261, 303
src/mlipaudit/benchmarks/folding_stability
   folding_stability.py106496%212, 270, 275, 339
   helpers.py350100% 
src/mlipaudit/benchmarks/noncovalent_interactions
   noncovalent_interactions.py1681094%245, 248, 272, 283, 407, 424, 441, 525–527
src/mlipaudit/benchmarks/nudged_elastic_band
   engine.py1269425%62, 81–82, 84–89, 91, 93, 95–96, 98–99, 101–103, 112–113, 115–116, 120–121, 123, 130, 132, 134–136, 138–139, 141, 143–145, 147–151, 153, 159–163, 165–167, 169–171, 173–175, 177–180, 182, 185, 189–190, 192, 194–195, 197, 201–202, 205–210, 212–213, 229, 234, 236–238, 240, 242–243, 246–247, 253, 256–257, 261–262, 267
   nudged_elastic_band.py124695%302–303, 310, 322, 328, 335
src/mlipaudit/benchmarks/reactivity
   reactivity.py117397%239–240, 272
src/mlipaudit/benchmarks/reference_geometry_stability
   reference_geometry_stability.py122397%233, 305, 328
src/mlipaudit/benchmarks/ring_planarity
   ring_planarity.py99693%229, 232, 264, 267–268, 287
src/mlipaudit/benchmarks/sampling
   helpers.py64493%46, 67, 97, 129
   sampling.py2251095%303, 344, 385, 388–390, 440, 707, 825–826
src/mlipaudit/benchmarks/scaling
   scaling.py1033466%128–130, 138–139, 141–144, 154–156, 193–198, 201–203, 209–210, 212–213, 215–216, 219–220, 222, 243, 252, 271, 284
src/mlipaudit/benchmarks/solvent_radial_distribution
   solvent_radial_distribution.py101892%189–190, 192, 231, 236–237, 302, 325
src/mlipaudit/benchmarks/stability
   stability.py1501888%166, 169, 201–202, 205–206, 208, 210, 213, 215, 431–432, 442, 478, 487, 522, 537, 548
src/mlipaudit/benchmarks/tautomers
   tautomers.py92594%182, 208, 211–212, 231
src/mlipaudit/benchmarks/water_radial_distribution
   water_radial_distribution.py94297%190, 272
src/mlipaudit/ui
   __init__.py160100% 
   bond_length_distribution.py581181%89–90, 95–96, 141–142, 147, 150, 176, 188, 192
   conformer_selection.py100793%133–134, 139–140, 195, 320, 324
   dihedral_scan.py1151289%141–142, 147–148, 259–260, 302, 313, 353, 355, 365, 369
   folding_stability.py96792%44, 226–227, 232–233, 406, 410
   leaderboard.py801087%67, 74, 78, 85, 287–289, 291, 293, 303
   noncovalent_interactions.py152994%205–206, 211–212, 392, 397, 432, 454, 458
   nudged_elastic_band.py553929%42–46, 50–51, 53, 66, 68, 75, 82, 89–90, 93, 97–99, 101, 103–105, 107–109, 111–113, 115, 117, 119, 121–122, 124, 135, 137, 149, 159, 163
   page_wrapper.py14285%36, 46
   reactivity.py59689%96–97, 102–103, 173, 177
   reference_geometry_stability.py66986%62, 131–132, 137–138, 199, 226, 236, 240
   ring_planarity.py56983%71–72, 77–78, 123–124, 160, 170, 174
   sampling.py87989%85, 129–130, 135–136, 166, 234, 265, 269
   scaling.py52688%135–136, 147–148, 162, 166
   solvent_radial_distribution.py832075%117–118, 123–124, 150, 156, 162–166, 168–169, 175, 178, 190, 197, 199, 211, 215
   stability.py601083%106–107, 112–113, 134–135, 138, 148, 177, 181
   tautomers.py62690%82–83, 88–89, 194, 198
   utils.py1061982%69, 73–74, 130, 192–195, 201, 251, 269–270, 273–274, 277, 287–289, 291
   water_radial_distribution.py98693%132–133, 138–139, 279, 283
src/mlipaudit/utils
   __init__.py9455%28–29, 33–34
   _ase_engine.py270100% 
   inference.py29775%66–68, 77–78, 81, 84
   simulation.py42685%45, 49, 122, 160–162
   stability.py31583%60, 67, 91, 102, 106
   trajectory_helpers.py340100% 
   unallowed_elements.py110100% 
TOTAL396447887% 

Tests Skipped Failures Errors Time
128 0 💤 0 ❌ 0 🔥 13.632s ⏱️

Importing mlipaudit.benchmarks (which the CLI does to build the argument
parser) eagerly pulled in mlip.models, JAX/JAX-MD and scikit-learn, making
even `mlipaudit -h` take ~6s. These libraries are only needed when a
benchmark is actually run.

Defer those imports to call time:
- benchmark.py / utils.{inference,simulation,stability,unallowed_elements}:
  move mlip.models / mlip.simulation.ase (JAX-MD) / jax / sklearn imports
  into the functions that use them; type-only refs go under TYPE_CHECKING
  with `from __future__ import annotations`.
- Move ASESimulationEngineWithCalculator into utils/_ase_engine.py so the
  heavy ASE-engine base class is only imported when a simulation runs; keep
  the historical import paths working via module-level __getattr__.
- nudged_elastic_band / conformer_selection / dihedral_scan /
  water_radial_distribution: defer their mlip / sklearn imports.
- main.py: import launch_app / run_benchmarks inside their command branches.

Tests patched the now-lazy symbols at the use-site module; update them to
patch at the definition site (e.g. mlip.simulation.jax_md.JaxMDSimulationEngine),
which the lazy `from ... import` picks up at call time.

`mlipaudit -h` drops from ~5.6s to ~0.9s; importing mlipaudit.benchmarks
drops from ~5.0s to ~1.1s. All 128 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lwalew lwalew force-pushed the perf/lazy-cli-imports branch from d317e03 to d8a4bb2 Compare June 24, 2026 09:14
@lwalew lwalew changed the base branch from main to develop June 24, 2026 09:14
@lwalew lwalew added the experimentation A prototype or experiment label Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

experimentation A prototype or experiment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant