test(ckpt): isolate conversion launcher utility imports#5003
Conversation
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
LGTM Test-only change hardening the _load_run_conversion_module loader and its coverage. The loader now tracks utils in previous_modules and pops any preexisting utils before loading run_conversion.py (which does a top-level import of utils), so the real scripts/conversion/utils.py loads fresh and any global utils module is restored afterward. The new test_loader_restores_preexisting_utils_module correctly exercises the restore path: it installs a sentinel utils, runs the loader, and asserts restoration. This would have failed before the fix since utils was not previously tracked. No bugs, typos, or doc gaps found. Suggested test cases:
Run the full test_run_conversion.py module to confirm the utils pop/restore does not regress the CPU/GPU dispatch tests. No perf tests impacted. |
What does this PR do ?
Fix the main-branch core unit-test failure caused by the conversion launcher test leaking its top-level
utilsmodule into the shared pytest process.Changelog
utilsmodule before dynamically loadingscripts/conversion/run_conversion.py.Root Cause
PR #4909 added
from utils import resolve_hf_commit_revision, resolve_hf_model_revisiontoscripts/conversion/run_conversion.py. The unit-test loader prependsscripts/conversiontosys.path, so that import cachesscripts/conversion/utils.pyas the top-level moduleutils. The loader restored its other temporary modules but notutils. Later,test_override_precedence.pyimportedutils.utilsandutils.overridesfromscripts/performance; Python reused the leaked plain module and raisedModuleNotFoundError: 'utils' is not a packagefor 15 tests.Failing main run: https://github.com/NVIDIA-NeMo/Megatron-Bridge/actions/runs/29876906382/job/88790258314
GitHub Actions CI
Focused validation:
test_run_conversion.py: 9 passed in a lightweight isolated environment.uvx pre-commit run --all-files: all hooks passed.git diff --check: passed.uv run pre-commit run --all-filescannot resolve the complete project environment on macOS ARM becausenvidia-resiliency-ext==0.6.0only publishes Linux wheels; the same repository hooks were therefore run withuvx pre-commit.Before your PR is "Ready for review"
Pre checks:
Additional Information