Update testn_tokenize.py

bact · bact · commit 9998c3b95aee · 2026-02-01T23:09:30.000Z
diff --git a/tests/noauto/testn_tokenize.py b/tests/noauto/testn_tokenize.py
@@ -16,14 +16,11 @@
     paragraph_tokenize,
     sefr_cut,
     sent_tokenize,
-    subword_tokenize,
     word_dict_trie,
     word_tokenize,
 )
 
 from ..core.test_tokenize import (
-    SENT_1,
-    SENT_2,
     SENT_3,
     TEXT_1,
 )

Original file line number	Diff line number	Diff line change
`@@ -16,14 +16,11 @@`
`16`	`16`	`paragraph_tokenize,`
`17`	`17`	`sefr_cut,`
`18`	`18`	`sent_tokenize,`
`19`		`- subword_tokenize,`
`20`	`19`	`word_dict_trie,`
`21`	`20`	`word_tokenize,`
`22`	`21`	`)`
`23`	`22`
`24`	`23`	`from ..core.test_tokenize import (`
`25`		`- SENT_1,`
`26`		`- SENT_2,`
`27`	`24`	`SENT_3,`
`28`	`25`	`TEXT_1,`
`29`	`26`	`)`