Skip to content

Commit 25ecb9f

Browse files
committed
fix(gallery): use Q8_0 for lfm2.5-8b-a1b to fix poor tool-call quality
The Q4_K_M quant degraded tool-call reliability for LFM2.5-8B-A1B. Switch the gallery entry to the Q8_0 GGUF (sha256 verified via HF x-linked-etag) while keeping the native jinja tool-parsing config. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
1 parent 2be495f commit 25ecb9f

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

gallery/index.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1716,17 +1716,17 @@
17161716
- use_jinja:true
17171717
parameters:
17181718
min_p: 0.15
1719-
model: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q4_K_M.gguf
1719+
model: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
17201720
repeat_penalty: 1.05
17211721
temperature: 0.1
17221722
top_k: 50
17231723
top_p: 0.1
17241724
template:
17251725
use_tokenizer_template: true
17261726
files:
1727-
- filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q4_K_M.gguf
1728-
uri: https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q4_K_M.gguf
1729-
sha256: 4923ec14f06b968b74d663e5949867d2d9c3bf13a20b8be1a9f9af39989b2bb0
1727+
- filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
1728+
uri: https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
1729+
sha256: 33ab3b8ce6a964fb8ebac89360c9b3cf72c4fa418d5e4c0a94d46883124d5c02
17301730
- name: "qwopus3.5-9b-coder-mtp"
17311731
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
17321732
urls:

0 commit comments

Comments
 (0)