Skip to content

Commit 6070402

Browse files
localai-botmudler
andauthored
chore(model gallery): 🤖 add 1 new models via gallery agent (#10209)
chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
1 parent 67f80a1 commit 6070402

1 file changed

Lines changed: 54 additions & 0 deletions

File tree

gallery/index.yaml

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,58 @@
11
---
2+
- name: "gemma-4-26b-a4b-it-qat"
3+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
4+
urls:
5+
- https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF
6+
description: |
7+
Hugging Face |
8+
GitHub |
9+
Launch Blog |
10+
Documentation
11+
12+
License: Apache 2.0 | Authors: Google DeepMind
13+
14+
> [!Note]
15+
> This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.
16+
> Four versions of the QAT checkpoints are available:
17+
> * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models.
18+
> * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
19+
> * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B.
20+
> * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B
21+
22+
...
23+
license: "apache-2.0"
24+
tags:
25+
- llm
26+
- gguf
27+
- gemma
28+
icon: https://ai.google.dev/gemma/images/gemma4_banner.png
29+
overrides:
30+
backend: llama-cpp
31+
function:
32+
automatic_tool_parsing_fallback: true
33+
grammar:
34+
disable: true
35+
known_usecases:
36+
- chat
37+
mmproj: llama-cpp/mmproj/gemma-4-26B-A4B-it-qat-GGUF/mmproj-F32.gguf
38+
options:
39+
- use_jinja:true
40+
parameters:
41+
min_p: 0
42+
model: llama-cpp/models/gemma-4-26B-A4B-it-qat-GGUF/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf
43+
repeat_penalty: 1
44+
temperature: 1
45+
top_k: 64
46+
top_p: 0.95
47+
template:
48+
use_tokenizer_template: true
49+
files:
50+
- filename: llama-cpp/models/gemma-4-26B-A4B-it-qat-GGUF/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf
51+
sha256: dcf179a91153e3a7ece792e48ef872180d9d6ef9b7677f0a0bd3e83cfe624d5e
52+
uri: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF/resolve/main/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf
53+
- filename: llama-cpp/mmproj/gemma-4-26B-A4B-it-qat-GGUF/mmproj-F32.gguf
54+
sha256: ef269e294502d6ee3722cbf129681b2586c2e6ceb79d0507963c92146e058cd4
55+
uri: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF/resolve/main/mmproj-F32.gguf
256
- name: "gemma-4-12b-it-qat-q4_0"
357
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
458
urls:

0 commit comments

Comments
 (0)