Skip to content

v0.9: Q4 weight quantization — 38.2 tok/s (2.5x from Q8) #43

v0.9: Q4 weight quantization — 38.2 tok/s (2.5x from Q8)

v0.9: Q4 weight quantization — 38.2 tok/s (2.5x from Q8) #43