Skip to content

Commit d86fa8f

Browse files
committed
Fix CI: ruff format + enhance sdist verification
- Format files flagged by CI lint workflow - Add model loading check to sdist build verification - Verify from_pretrained works in installed package
1 parent 6662adb commit d86fa8f

3 files changed

Lines changed: 3 additions & 4 deletions

File tree

.github/workflows/python-package.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ jobs:
8181
- name: Install from sdist
8282
run: |
8383
pip install dist/*.tar.gz
84-
- name: Verify import
84+
- name: Verify import and model loading
8585
run: |
8686
cd /tmp
87-
python -c "import tinybpe; print(tinybpe.__version__)"
87+
python -c "import tinybpe; print(tinybpe.__version__); print(tinybpe.list_models()); from tinybpe import Tokenizer; tok = Tokenizer.from_pretrained('cl100k_base'); print(len(tok.encode('hello world')))"

tests/test_hf_models.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -186,8 +186,6 @@ def test_code_text(self):
186186
assert decoded == code
187187

188188

189-
190-
191189
@pytest.mark.skipif(not HAS_HF, reason="huggingface_hub not installed")
192190
class TestDeepSeekSpecific:
193191
"""DeepSeek-V4 tokenizer specific tests.

tinybpe/_registry.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ class ModelInfo(TypedDict):
3838
# Load registry from JSON
3939
# ---------------------------------------------------------------------------
4040

41+
4142
def _load_registry() -> tuple[dict[str, ModelInfo], dict[str, str | None]]:
4243
"""Load model registry and pattern definitions from ``models/models.json``.
4344

0 commit comments

Comments
 (0)