Skip to content

Commit 2ce0cd2

Browse files
committed
Add note about changing tokenizer_class to support transformers<5.0.0
1 parent 9c461b5 commit 2ce0cd2

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

  • content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark

content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark/4-testing.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,10 @@ Notice how the original model gets stuck in a repetitive loop, continuing to out
130130

131131
Now test your fine-tuned model to see how the Alpaca instruction dataset improved its response quality. Stop the current vLLM server (press Ctrl+C in the container terminal) before launching the fine-tuned model.
132132

133+
{{% notice Dependency Conflict %}}
134+
As of this writing, vLLM does not support version 5 of the `transformers` library that was used when fine-tuning the model, so you need to make a slight change to it's `tokenizer_config.json`. Open that file and set the `tokenizer_class` to `PreTrainedTokenizerFast`, which is available in the older version of the `transformers` library used by vLLM.
135+
{{% /notice %}}
136+
133137
### Launch vLLM
134138

135139
Start the vLLM server with your fine-tuned model:

0 commit comments

Comments
 (0)