Fix: Refactor LoRA checkpoint restoration and simplify NNX weight extraction

[RexBearIU]

… updating base and LoRA checkpoints

Description

This PR introduces robust post-training LoRA checkpoint conversion and restoration, specifically optimized for scanned layer designs (like Gemma 3).

1. Scanned Weight Support:
   • main only supported 2D/3D unscanned matrices.
   • This branch adds a generalized contraction helper ( _get_lora_delta ) in to_huggingface.py to handle 3D/4D scanned layer weight matrices
   (handling layer dimensions at axis 0/1) for seamless HuggingFace adapter exports.
2. Recursive Checkpoint Merging:
   • main only loaded a single checkpoint directory.
   • This branch updates load_orbax_checkpoint to dynamically load and recursively merge a base model checkpoint and a separate LoRA-only adapter
   checkpoint without overwriting keys.
3. O(1) LoRA Restoration:
   • Replaced slow whole-tree to_pure_dict calls in lora_utils.py with detect_and_extract_checkpoint for fast, flat parameter lookups.
4. Simplified NNX Extraction:
   • Removed list-traversal path normalization in utils.py to rely on standard JAX flat-attribute naming.

Tests

The changes have been thoroughly verified with the unit test suite:
```bash
JAX_PLATFORMS=cpu PYTHONPATH=src python -m pytest tests/unit/hf_checkpoint_conversion_test.py tests/post_training/unit/lora_utils_test.py
tests/unit/lora_utils_nnx_test.py

All 29 tests passed successfully.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[codecov]

Codecov Report

❌ Patch coverage is 2.27273% with 43 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...rc/maxtext/checkpoint_conversion/to_huggingface.py	0.00%	32 Missing ⚠️
src/maxtext/checkpoint_conversion/utils/utils.py	8.33%	11 Missing ⚠️

📢 Thoughts on this report? Let us know!