Feature Request: Is 1bit compatible with TurboQuant+?

[cusclo]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

https://github.com/TheTom/turboquant_plus
https://github.com/TheTom/llama-cpp-turboquant.git

Motivation

Does this make edge devices perform better?

Possible Implementation

No response

[cusclo]

llama.cpp CPU ：ggml-org#21273
llama.cpp Metal ：ggml-org#21528
Thank you very much, I have seen the merge. I have merged turboquant + 1bit locally. Bonsai-8B.gguf runs very fast on my Mac mini m4 16g device. My next step is to package it into a mobile app.

[khosravipasha]

@cusclo that's interesting curios to know how good it is with turbo-quant

overall we added commmunity benchmakrs to gather more numbers on differetn devices, would love a PR there, see template and readme: https://github.com/PrismML-Eng/Bonsai-demo/tree/main/community-benchmarks