Skip to content

Batch tokenization in the transformers backend's scoring loop#1286

Open
ErenAta16 wants to merge 3 commits into
huggingface:mainfrom
ErenAta16:optimize-tokenization
Open

Batch tokenization in the transformers backend's scoring loop#1286
ErenAta16 wants to merge 3 commits into
huggingface:mainfrom
ErenAta16:optimize-tokenization

Commits

Commits on Jul 6, 2026