Skip to content

Commit 06a1524

Browse files
localai-botmudler
andauthored
chore(model gallery): 🤖 add 1 new models via gallery agent (#9681)
chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
1 parent 70cf8ac commit 06a1524

1 file changed

Lines changed: 74 additions & 0 deletions

File tree

gallery/index.yaml

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,78 @@
11
---
2+
- name: "qwen3.6-27b-heretic-uncensored-finetune-neo-code-di-imatrix-max"
3+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
4+
urls:
5+
- https://huggingface.co/DavidAU/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF
6+
description: |
7+
Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking
8+
9+
Yes... fully uncensored AND fine tuned lightly.
10+
11+
Freedom and brainpower.
12+
13+
Trained on different Heretic base, with different KLD/Refusals.
14+
15+
Model fine tune was used to finalize and "firm up" Heretic / uncensored changes.
16+
17+
The goal here was light, minor fixes rather than full / heavy fine tune.
18+
19+
That being said, the tuning still raised critical metrics.
20+
21+
This is Version 2, using "trohrbaugh" Heretic, which has a lower refusal rate, and tuning bumped up the metrics a bit more too.
22+
23+
This has also positively impacted "NEO-Coder Di-Matrix" (dual imatrix) GGUF quants as well (vs heretic/non heretic too).
24+
25+
https://huggingface.co/DavidAU/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF
26+
27+
```
28+
IN HOUSE BENCHMARKS [by Nightmedia]:
29+
30+
arc-c arc/e boolq hswag obkqa piqa wino
31+
32+
Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking
33+
mxfp8 0.673,0.846,0.905... [instruct mode]
34+
35+
Qwen3.6-27B-Heretic-Uncensored-Finetune-Thinking
36+
mxfp8 0.669,0.835,0.906,... [instruct mode]
37+
38+
BASE UNTUNED MODEL:
39+
40+
Qwen3.6-27B HERETIC (by llmfan46) [instruct mode]
41+
mxfp8 0.644,0.788,0.902,...
42+
43+
...
44+
license: "apache-2.0"
45+
tags:
46+
- llm
47+
- gguf
48+
overrides:
49+
backend: llama-cpp
50+
function:
51+
automatic_tool_parsing_fallback: true
52+
grammar:
53+
disable: true
54+
known_usecases:
55+
- chat
56+
mmproj: llama-cpp/mmproj/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF/mmproj-F32.gguf
57+
options:
58+
- use_jinja:true
59+
parameters:
60+
min_p: 0
61+
model: llama-cpp/models/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF/Qwen3.6-27B-NEO-CODE-HERE-2T-OT-Q4_K_M.gguf
62+
presence_penalty: 1.5
63+
repeat_penalty: 1
64+
temperature: 0.7
65+
top_k: 20
66+
top_p: 0.8
67+
template:
68+
use_tokenizer_template: true
69+
files:
70+
- filename: llama-cpp/models/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF/Qwen3.6-27B-NEO-CODE-HERE-2T-OT-Q4_K_M.gguf
71+
sha256: 4b271d8bb53345513fcfc52eb2c38f91ecfd3c7d978e43481d335fca47a595a3
72+
uri: https://huggingface.co/DavidAU/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF/resolve/main/Qwen3.6-27B-NEO-CODE-HERE-2T-OT-Q4_K_M.gguf
73+
- filename: llama-cpp/mmproj/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF/mmproj-F32.gguf
74+
sha256: fdc443e974cad1f61c45af1cfd5580855855ddce0d6c14cc500a5714c486ac1d
75+
uri: https://huggingface.co/DavidAU/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF/resolve/main/mmproj-F32.gguf
276
- name: "qwen3.5-9b-deepseek-v4-flash"
377
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
478
urls:

0 commit comments

Comments
 (0)