Skip to content

Commit 9f5c0b8

Browse files
committed
Add HF Molmo2 loader and logit parity tests
- molmo2_loader.py: HF Molmo2 → MultimodalTransformer weight converter (embedding split, fused QKV/MLP split, patch-embedding permute) - molmo2_parity_test.py: vision-encoder parity vs HF reference - molmo2_logits_parity_test.py: embedding + LM + full-pipeline logit parity (4B/8B pass; O-7B skipped pending per-layer YaRN support) - Fix transformers 5.x RoPE compat: re-register ROPE_INIT_FUNCTIONS['default'] and reinit non-persistent inv_freq buffers after from_pretrained fast-init
1 parent 34ba65d commit 9f5c0b8

6 files changed

Lines changed: 1646 additions & 0 deletions

File tree

src/olmo_core/nn/vision/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
VisionConnectorConfig,
1111
)
1212
from .image_vit import SiglipVisionTransformer, VisionTransformer
13+
from .molmo2_loader import molmo2_hf_state_dict_to_multimodal_transformer
1314
from .multimodal import MultimodalTransformer, MultimodalTransformerConfig
1415

1516
__all__ = [
@@ -23,4 +24,5 @@
2324
"VisionConnector",
2425
"MultimodalTransformerConfig",
2526
"MultimodalTransformer",
27+
"molmo2_hf_state_dict_to_multimodal_transformer",
2628
]

0 commit comments

Comments
 (0)