Skip to content

Commit dbd5add

Browse files
committed
docs: add addt vision model support
Signed-off-by: Anh Uong <anh.uong@ibm.com>
1 parent f2ba2c3 commit dbd5add

1 file changed

Lines changed: 10 additions & 6 deletions

File tree

README.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -909,6 +909,8 @@ For information on supported dataset formats and how to tune a vision-language m
909909

910910
### Supported vision model
911911

912+
Note that vision models are supported starting with `fms-hf-tuning` v2.8.0 or later.
913+
912914
- Legend:
913915

914916
✅ Ready and available
@@ -921,12 +923,14 @@ For information on supported dataset formats and how to tune a vision-language m
921923

922924
Model Name & Size | Model Architecture | LoRA Tuning | Full Finetuning |
923925
-------------------- | ---------------- | --------------- | --------------- |
924-
Llama 3.2-11B Vision | MllamaForConditionalGeneration | ✅* | ✅* |
925-
Llava 1.5-7B | LlavaForConditionalGeneration | ✅* | ✅* |
926-
Granite 3.1-2B Vision | LlavaNextForConditionalGeneration | ✅* | ✅* |
927-
Llava Mistral 1.6-7B | LlavaNextForConditionalGeneration | ✅* | ✅* |
928-
929-
(*) - Supported with `fms-hf-tuning` v2.8.0 or later.
926+
Llama 3.2-11B Vision | MllamaForConditionalGeneration | ✅ | ✅ |
927+
Llama 3.2-90B Vision | MllamaForConditionalGeneration | ✔️ | ✔️ |
928+
Granite 3.2-2B Vision | LlavaNextForConditionalGeneration | ✅ | ✅ |
929+
Llava Mistral 1.6-7B | LlavaNextForConditionalGeneration | ✅ | ✅ |
930+
Llava 1.6-34B | LlavaNextForConditionalGeneration | ✔️ | ✔️ |
931+
Llava next-72B | LlavaNextForConditionalGeneration | ✔️ | ✔️ |
932+
Llava 1.5-7B | LlavaForConditionalGeneration | ✅ | ✅ |
933+
Llava 1.5-13B | LlavaForConditionalGeneration | ✅ | ✅ |
930934

931935
**Note**: vLLM currently does not support inference with LoRA-tuned vision models. To use a tuned LoRA adapter of vision model, please merge it with the base model before running vLLM inference.
932936

0 commit comments

Comments
 (0)