Skip to content

Commit 4a56bd5

Browse files
fix: use save_pretrained from PreTrainedTokenizerFast object
1 parent 593ce32 commit 4a56bd5

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

torchTextClassifiers/tokenizers/WordPiece.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ def train(
8282
self._post_training()
8383

8484
if save_path:
85-
self.tokenizer.save(save_path)
85+
self.tokenizer.save_pretrained(save_path)
8686
logger.info(f"💾 Tokenizer saved at {save_path}")
8787
if filesystem and s3_save_path:
8888
parent_dir = os.path.dirname(save_path)

0 commit comments

Comments
 (0)