Skip to content

Commit b361d2d

Browse files
mudlermudler
committed
chore(gallery): add new llama.cpp supported models (qwen-asr, ocr)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 1e4c457 commit b361d2d

1 file changed

Lines changed: 205 additions & 0 deletions

File tree

‎gallery/index.yaml‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2200,6 +2200,211 @@
22002200
- filename: mmproj/mmproj-Qwen3-VL-8B-Thinking-F16.gguf
22012201
sha256: 64d5be3f16fb91cfb451155fe4745266e2169ccbe1f29f57bfab27fb7fec389e
22022202
uri: huggingface://unsloth/Qwen3-VL-8B-Thinking-GGUF/mmproj-F16.gguf
2203+
- &ggmlorg-llamacpp
2204+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
2205+
icon: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/-s1gyJfvbE1RgO5iBeNOi.png
2206+
license: apache-2.0
2207+
name: "qwen3-omni-30b-a3b-instruct"
2208+
urls:
2209+
- https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
2210+
- https://huggingface.co/ggml-org/Qwen3-Omni-30B-A3B-Instruct-GGUF
2211+
tags:
2212+
- llm
2213+
- gguf
2214+
- gpu
2215+
- image-to-text
2216+
- audio-to-text
2217+
- multimodal
2218+
- cpu
2219+
- qwen
2220+
- qwen3
2221+
- omni
2222+
description: |
2223+
Qwen3-Omni is the natively end-to-end multilingual omni-modal foundation model. It processes text, images, audio, and video, and delivers real-time streaming responses in both text and natural speech. This GGUF build runs on llama.cpp with the bundled mmproj for multimodal inputs.
2224+
overrides:
2225+
backend: llama-cpp
2226+
mmproj: mmproj-Qwen3-Omni-30B-A3B-Instruct-Q8_0.gguf
2227+
parameters:
2228+
model: Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf
2229+
template:
2230+
use_tokenizer_template: true
2231+
options:
2232+
- use_jinja:true
2233+
files:
2234+
- filename: Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf
2235+
sha256: d9e2876556e7873e02c0359f832432ee2d67ab7dd0cee3efe0f77fd7a1f4dd85
2236+
uri: huggingface://ggml-org/Qwen3-Omni-30B-A3B-Instruct-GGUF/Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf
2237+
- filename: mmproj-Qwen3-Omni-30B-A3B-Instruct-Q8_0.gguf
2238+
sha256: 1104376db833f1e89c84834144ac3863340c2cd1ddaeddb39cb0247fb5c20c8d
2239+
uri: huggingface://ggml-org/Qwen3-Omni-30B-A3B-Instruct-GGUF/mmproj-Qwen3-Omni-30B-A3B-Instruct-Q8_0.gguf
2240+
- !!merge <<: *ggmlorg-llamacpp
2241+
name: "qwen3-omni-30b-a3b-thinking"
2242+
urls:
2243+
- https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Thinking
2244+
- https://huggingface.co/ggml-org/Qwen3-Omni-30B-A3B-Thinking-GGUF
2245+
tags:
2246+
- llm
2247+
- gguf
2248+
- gpu
2249+
- image-to-text
2250+
- audio-to-text
2251+
- multimodal
2252+
- cpu
2253+
- qwen
2254+
- qwen3
2255+
- omni
2256+
- thinking
2257+
- reasoning
2258+
description: |
2259+
Qwen3-Omni-30B-A3B-Thinking is the reasoning-enhanced variant of Qwen3-Omni, a natively end-to-end multilingual omni-modal foundation model. It processes text, images, and audio and produces chain-of-thought reasoning before the final answer. This GGUF build runs on llama.cpp with the bundled mmproj.
2260+
overrides:
2261+
backend: llama-cpp
2262+
mmproj: mmproj-Qwen3-Omni-30B-A3B-Thinking-Q8_0.gguf
2263+
parameters:
2264+
model: Qwen3-Omni-30B-A3B-Thinking-Q4_K_M.gguf
2265+
template:
2266+
use_tokenizer_template: true
2267+
options:
2268+
- use_jinja:true
2269+
files:
2270+
- filename: Qwen3-Omni-30B-A3B-Thinking-Q4_K_M.gguf
2271+
sha256: afdaeff6f23c740429aadb3fa180f9d53b78278fe0d331b594b0b71bd9bf4835
2272+
uri: huggingface://ggml-org/Qwen3-Omni-30B-A3B-Thinking-GGUF/Qwen3-Omni-30B-A3B-Thinking-Q4_K_M.gguf
2273+
- filename: mmproj-Qwen3-Omni-30B-A3B-Thinking-Q8_0.gguf
2274+
sha256: 2bd5459571f8230a0c251d3d0dd36267753f0800ed145449a34f220a31f93898
2275+
uri: huggingface://ggml-org/Qwen3-Omni-30B-A3B-Thinking-GGUF/mmproj-Qwen3-Omni-30B-A3B-Thinking-Q8_0.gguf
2276+
- !!merge <<: *ggmlorg-llamacpp
2277+
name: "qwen3-asr-0.6b"
2278+
urls:
2279+
- https://huggingface.co/Qwen/Qwen3-ASR
2280+
- https://huggingface.co/ggml-org/Qwen3-ASR-0.6B-GGUF
2281+
tags:
2282+
- llm
2283+
- gguf
2284+
- gpu
2285+
- audio-to-text
2286+
- asr
2287+
- cpu
2288+
- qwen
2289+
- qwen3
2290+
description: |
2291+
Qwen3-ASR 0.6B is a compact automatic speech recognition model from the Qwen3 family, distributed as a GGUF for llama.cpp. It accepts audio input through the paired mmproj and transcribes it to text, supporting multilingual speech.
2292+
overrides:
2293+
backend: llama-cpp
2294+
mmproj: mmproj-Qwen3-ASR-0.6B-Q8_0.gguf
2295+
parameters:
2296+
model: Qwen3-ASR-0.6B-Q8_0.gguf
2297+
template:
2298+
use_tokenizer_template: true
2299+
options:
2300+
- use_jinja:true
2301+
files:
2302+
- filename: Qwen3-ASR-0.6B-Q8_0.gguf
2303+
sha256: bca259818b50ca7c4c05e9bdb35a5dc04fa039653a6d6f3f0f331f96f6aa1971
2304+
uri: huggingface://ggml-org/Qwen3-ASR-0.6B-GGUF/Qwen3-ASR-0.6B-Q8_0.gguf
2305+
- filename: mmproj-Qwen3-ASR-0.6B-Q8_0.gguf
2306+
sha256: 41a342b5e4c514e968cb756de6cd1b7be39eff43c44c57a2ef5fc6522e36603d
2307+
uri: huggingface://ggml-org/Qwen3-ASR-0.6B-GGUF/mmproj-Qwen3-ASR-0.6B-Q8_0.gguf
2308+
- !!merge <<: *ggmlorg-llamacpp
2309+
name: "qwen3-asr-1.7b"
2310+
urls:
2311+
- https://huggingface.co/Qwen/Qwen3-ASR
2312+
- https://huggingface.co/ggml-org/Qwen3-ASR-1.7B-GGUF
2313+
tags:
2314+
- llm
2315+
- gguf
2316+
- gpu
2317+
- audio-to-text
2318+
- asr
2319+
- cpu
2320+
- qwen
2321+
- qwen3
2322+
description: |
2323+
Qwen3-ASR 1.7B is the larger automatic speech recognition model from the Qwen3 family, distributed as a GGUF for llama.cpp. It accepts audio input through the paired mmproj and produces higher-quality multilingual transcriptions than the 0.6B variant.
2324+
overrides:
2325+
backend: llama-cpp
2326+
mmproj: mmproj-Qwen3-ASR-1.7B-Q8_0.gguf
2327+
parameters:
2328+
model: Qwen3-ASR-1.7B-Q8_0.gguf
2329+
template:
2330+
use_tokenizer_template: true
2331+
options:
2332+
- use_jinja:true
2333+
files:
2334+
- filename: Qwen3-ASR-1.7B-Q8_0.gguf
2335+
sha256: 58e22d0532d4eacaf034cfac17a6fed159f37c41390c710186783be439d1fc57
2336+
uri: huggingface://ggml-org/Qwen3-ASR-1.7B-GGUF/Qwen3-ASR-1.7B-Q8_0.gguf
2337+
- filename: mmproj-Qwen3-ASR-1.7B-Q8_0.gguf
2338+
sha256: 46c1d533af3f354ceb37ce855dbceff7da7fa7cf1e6a523df3b13440bd164c0d
2339+
uri: huggingface://ggml-org/Qwen3-ASR-1.7B-GGUF/mmproj-Qwen3-ASR-1.7B-Q8_0.gguf
2340+
- !!merge <<: *ggmlorg-llamacpp
2341+
name: "glm-ocr"
2342+
icon: https://huggingface.co/zai-org.png
2343+
license: mit
2344+
urls:
2345+
- https://huggingface.co/zai-org/GLM-4.1V-9B-Thinking
2346+
- https://huggingface.co/ggml-org/GLM-OCR-GGUF
2347+
tags:
2348+
- llm
2349+
- gguf
2350+
- gpu
2351+
- image-to-text
2352+
- ocr
2353+
- multimodal
2354+
- cpu
2355+
- glm
2356+
description: |
2357+
GLM-OCR is a vision-language model specialized for optical character recognition and document understanding, built on the GLM architecture. This GGUF build runs on llama.cpp with the bundled mmproj.
2358+
overrides:
2359+
backend: llama-cpp
2360+
mmproj: mmproj-GLM-OCR-Q8_0.gguf
2361+
parameters:
2362+
model: GLM-OCR-Q8_0.gguf
2363+
template:
2364+
use_tokenizer_template: true
2365+
options:
2366+
- use_jinja:true
2367+
files:
2368+
- filename: GLM-OCR-Q8_0.gguf
2369+
sha256: 45bc244a6446aff850521dc41f18bc8d7105ad5f0c2c8c28af04e7cc4f4d50b1
2370+
uri: huggingface://ggml-org/GLM-OCR-GGUF/GLM-OCR-Q8_0.gguf
2371+
- filename: mmproj-GLM-OCR-Q8_0.gguf
2372+
sha256: 9c4b58e33e316ed142eb5dcb41abec3844d3e6e5dc361ffb782c3fa9d175141f
2373+
uri: huggingface://ggml-org/GLM-OCR-GGUF/mmproj-GLM-OCR-Q8_0.gguf
2374+
- !!merge <<: *ggmlorg-llamacpp
2375+
name: "deepseek-ocr"
2376+
icon: https://huggingface.co/deepseek-ai.png
2377+
license: mit
2378+
urls:
2379+
- https://huggingface.co/deepseek-ai/DeepSeek-OCR
2380+
- https://huggingface.co/ggml-org/DeepSeek-OCR-GGUF
2381+
tags:
2382+
- llm
2383+
- gguf
2384+
- gpu
2385+
- image-to-text
2386+
- ocr
2387+
- multimodal
2388+
- cpu
2389+
- deepseek
2390+
description: |
2391+
DeepSeek-OCR is a vision-language model from DeepSeek AI specialized for optical character recognition and document understanding. This GGUF build runs on llama.cpp with the bundled mmproj.
2392+
overrides:
2393+
backend: llama-cpp
2394+
mmproj: mmproj-DeepSeek-OCR-Q8_0.gguf
2395+
parameters:
2396+
model: DeepSeek-OCR-Q8_0.gguf
2397+
template:
2398+
use_tokenizer_template: true
2399+
options:
2400+
- use_jinja:true
2401+
files:
2402+
- filename: DeepSeek-OCR-Q8_0.gguf
2403+
sha256: 81ede3e256230707dccf7fa052570c3a939d57db99de655f43cbb1a830d14d92
2404+
uri: huggingface://ggml-org/DeepSeek-OCR-GGUF/DeepSeek-OCR-Q8_0.gguf
2405+
- filename: mmproj-DeepSeek-OCR-Q8_0.gguf
2406+
sha256: 786c9b5159898de3d1d94a102836df559fed0bcf09f41a32f62c3219b0e278e0
2407+
uri: huggingface://ggml-org/DeepSeek-OCR-GGUF/mmproj-DeepSeek-OCR-Q8_0.gguf
22032408
- &jamba
22042409
icon: https://cdn-avatars.huggingface.co/v1/production/uploads/65e60c0ed5313c06372446ff/QwehUHgP2HtVAMW5MzJ2j.png
22052410
name: "ai21labs_ai21-jamba-reasoning-3b"

0 commit comments

Comments
 (0)