Commit 01bebe8
committed
feat: safetensors header parser + streaming indexer
Parses the safetensors JSON header (no serde dependency) and produces
GgufFile-compatible types so stream_index_gguf_bf16_with_header works
unchanged on safetensors files.
Safetensors stores full BF16 weights — no quantization noise.
For the reasoning diff pipeline, BF16→Base17 gives cleaner
fingerprints than Q8_0→f32→Base17.
Includes test_stream_index_qwen35_safetensors for 11-shard
Qwen3.5-27B indexing at full BF16 precision.1 parent 073fb0b commit 01bebe8
1 file changed
Lines changed: 414 additions & 0 deletions
0 commit comments