feat: native tokenization engine for BaseModel#195

ptomecek wants to merge 2 commits intomainPoint72/ccflow:mainfrom

pit/tokenizationPoint72/ccflow:pit/tokenizationCopy head branch name to clipboard