Remove MiniCPM-2B SentencePiece BPE support

TinyBPE now exclusively supports ByteLevel BPE tokenizers.
- Remove models/minicpm.tbm (SentencePiece BPE, replaced by minicpm5)
- Remove scripts/convert_minicpm.py (SentencePiece converter)
- Remove tests/test_minicpm.py
- Update registry, README, and model docs

Author: neluca

README.md

@@ -8,7 +8,7 @@
 
 **An ultra-fast, lightweight BPE tokenizer and trainer with a pure-C core.**
 
-Ever wished you could load a GPT-4 compatible tokenizer in **one line** without network calls? TinyBPE ships 8 pre-built tokenizer models directly in the package. The CPython C core runs BPE encoding/decoding at native speed — typically **10-50× faster** than pure-Python implementations while depending only on `regex`.
+Ever wished you could load a GPT-4 compatible tokenizer in **one line** without network calls? TinyBPE ships 8 pre-built ByteLevel BPE models directly in the package. The CPython C core runs BPE encoding/decoding at native speed — typically **10-50× faster** than pure-Python implementations while depending only on `regex`.
 
 ## Why TinyBPE?
 
@@ -75,7 +75,7 @@ tinybpe.list_models()
 | `qwen25` | Qwen 2.5 (0.5B–72B) | 151,643 |
 | `phi2` | Microsoft Phi-2 | 50,257 |
 | `deepseek-llm` | DeepSeek V2 (7B-Chat) | 100,013 |
-| `minicpm` | MiniCPM-2B (SentencePiece BPE) | 129,850 |
+| `minicpm5` | MiniCPM5-1B (ByteLevel BPE) | 130,050 |
 
 ### Training
 
@@ -206,8 +206,6 @@ python scripts/convert_tiktoken.py cl100k_base -o models/cl100k_base.tbm
 python scripts/convert_hf_tokenizer.py tokenizer.json -o output.tbm
 python scripts/convert_hf_tokenizer.py Qwen/Qwen2.5-0.5B -o models/qwen25.tbm
 
-# MiniCPM (SentencePiece BPE)
-python scripts/convert_minicpm.py -o models/minicpm.tbm
 ```
 
 See [`scripts/README.md`](scripts/README.md) for details.

README_zh.md

@@ -8,7 +8,7 @@
 
 **纯 C 内核的超轻量高性能 BPE 分词器与训练器。**
 
-只需**一行代码**即可加载 GPT-4 兼容分词器，无需联网。TinyBPE 内置 8 个预训练模型，开箱即用。CPython C 扩展以原生速度执行 BPE 编码/解码——通常比纯 Python 实现**快 10–50 倍**，且仅依赖 `regex` 一个第三方库。
+只需**一行代码**即可加载 GPT-4 兼容分词器，无需联网。TinyBPE 内置 8 个 ByteLevel BPE 预训练模型，开箱即用。CPython C 扩展以原生速度执行 BPE 编码/解码——通常比纯 Python 实现**快 10–50 倍**，且仅依赖 `regex` 一个第三方库。
 
 ## 为什么选择 TinyBPE？
 
@@ -75,7 +75,7 @@ tinybpe.list_models()
 | `qwen25` | Qwen 2.5（0.5B–72B） | 151,643 |
 | `phi2` | Microsoft Phi-2 | 50,257 |
 | `deepseek-llm` | DeepSeek V2（7B-Chat） | 100,013 |
-| `minicpm` | MiniCPM-2B（SentencePiece BPE） | 129,850 |
+| `minicpm5` | MiniCPM5-1B（ByteLevel BPE） | 130,050 |
 
 ### 训练分词器
 
@@ -206,8 +206,6 @@ python scripts/convert_tiktoken.py cl100k_base -o models/cl100k_base.tbm
 python scripts/convert_hf_tokenizer.py tokenizer.json -o output.tbm
 python scripts/convert_hf_tokenizer.py Qwen/Qwen2.5-0.5B -o models/qwen25.tbm
 
-# MiniCPM（SentencePiece BPE）
-python scripts/convert_minicpm.py -o models/minicpm.tbm
 ```
 
 详见 [`scripts/README.md`](scripts/README.md)。

models/README.md

@@ -34,12 +34,6 @@ ids = tok.encode("hello world")
 | `phi2` | Microsoft Phi-2 | 50,257 | GPT-2 regex | `microsoft/phi-2` |
 | `deepseek-llm` | DeepSeek V2 (7B-Chat) | 100,013 | None (raw) | `deepseek-ai/deepseek-llm-7b-chat` |
 
-### SentencePiece BPE (converted)
-
-| Model | LLM Compatibility | Vocab | Pre-tokenization | Notes |
-|---|---|---|---|---|
-| `minicpm` | MiniCPM-2B | 129,850 | None (SentencePiece) | Byte-level converted; needs manual SP normalization |
-
 ## Model Format
 
 All files use the `.tbm` (TinyBPE Model v1) text format:
@@ -63,9 +57,6 @@ python scripts/convert_tiktoken.py o200k_base -o models/o200k_base.tbm
 
 # HuggingFace tokenizer.json (local or Hub ID)
 python scripts/convert_hf_tokenizer.py Qwen/Qwen2.5-0.5B -o models/qwen25.tbm
-
-# MiniCPM / SentencePiece BPE
-python scripts/convert_minicpm.py -o models/minicpm.tbm
 ```
 
 After conversion, add the model to the registry in `tinybpe/_registry.py` so it becomes available via `Tokenizer.from_pretrained()`.
@@ -82,7 +73,7 @@ After conversion, add the model to the registry in `tinybpe/_registry.py` so it
 | Phi-2 | ByteLevel BPE | Full support |
 | Llama 3 / 4 | ByteLevel BPE | Via `convert_hf_tokenizer.py` |
 | DeepSeek V2 / V3 | ByteLevel BPE | Full support |
-| MiniCPM | SentencePiece BPE | Converted (byte-level) |
+| MiniCPM5-1B | ByteLevel BPE | Full support |
 | Mistral | SentencePiece BPE / Tekken | Not supported |
 | Gemma 2 / 3 | SentencePiece Unigram | Not supported |
 | Claude 3 / 4 | Proprietary BPE | No public tokenizer |