useTokenizer(
tokenizerProps):TokenizerType
Defined in: hooks/natural_language_processing/useTokenizer.ts:13
React hook for managing a Tokenizer instance.
Configuration object containing tokenizer source and optional preventLoad flag.
Ready to use Tokenizer model.