Skip to content

Batch tokenization in the transformers backend's scoring loop#1286

Open
ErenAta16 wants to merge 3 commits into
huggingface:mainfrom
ErenAta16:optimize-tokenization
Open

Batch tokenization in the transformers backend's scoring loop#1286
ErenAta16 wants to merge 3 commits into
huggingface:mainfrom
ErenAta16:optimize-tokenization

Add regression test for the batched tokenization call count

25bb0e9
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs