Commit 1965d5f
committed
refactor(mtmd): move multimodal handlers to separate module
- Move MTMDChatHandler, GenericMTMDChatHandler, and model-specific multimodal
chat handlers out of llama_chat_format.py into llama_multimodal.py.
- llama_chat_format.py has grown too large and difficult to maintain, especially
as MTMD support expands beyond image-only use cases. Splitting multimodal
handling into its own module makes the chat formatting layer smaller and keeps
media loading, MTMD tokenization, multimodal KV-cache bookkeeping, and handler
implementations in a dedicated place.
- This also prepares the codebase for broader multimodal support and future video
frame / image batch evaluation, where the media-processing path will need to
evolve independently from text-only chat formatting.
- Keep backward-compatible re-exports from llama_chat_format.py so existing
imports continue to work.
- Also keep `clip_model_path` as a deprecated initialization alias for
`mmproj_path` in the base MTMD handler.
Signed-off-by: JamePeng <jame_peng@sina.com>llama_multimodal
1 parent 29b3f3a commit 1965d5f
3 files changed
Lines changed: 3690 additions & 3602 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
48 | 49 | | |
49 | 50 | | |
50 | 51 | | |
| |||
711 | 712 | | |
712 | 713 | | |
713 | 714 | | |
714 | | - | |
715 | | - | |
716 | | - | |
717 | 715 | | |
718 | 716 | | |
719 | 717 | | |
720 | 718 | | |
721 | 719 | | |
722 | | - | |
| 720 | + | |
723 | 721 | | |
724 | 722 | | |
725 | 723 | | |
726 | 724 | | |
727 | 725 | | |
| 726 | + | |
| 727 | + | |
728 | 728 | | |
729 | 729 | | |
730 | 730 | | |
| |||
0 commit comments