Skip to content

Commit 5bef3d3

Browse files
committed
feat(gallery): add Arch-Router-1.5B (Q4_K_M, Q8_0)
Adds katanemo/Arch-Router-1.5B (mradermacher GGUFs) as a compact routing LLM that pairs with the router classifier as a preference-aligned alternative to embedding/ColBERT-based routing. Assisted-by: claude-code:claude-opus-4-7 [Read] [Edit] [Bash] [WebFetch]
1 parent d82ad5c commit 5bef3d3

1 file changed

Lines changed: 41 additions & 0 deletions

File tree

gallery/index.yaml

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23540,6 +23540,47 @@
2354023540
- python
2354123541
parameters:
2354223542
model: bge-m3-colbert
23543+
- &arch-router-1_5b
23544+
url: github:mudler/LocalAI/gallery/chatml.yaml@master
23545+
name: arch-router-1.5b-q4
23546+
icon: https://cdn-avatars.huggingface.co/v1/production/uploads/66b681906c8d3b36786b764c/uyP7mxDVv0HbV9Hv_KfHk.jpeg
23547+
license: other
23548+
urls:
23549+
- https://huggingface.co/katanemo/Arch-Router-1.5B
23550+
- https://huggingface.co/mradermacher/Arch-Router-1.5B-GGUF
23551+
description: |
23552+
Arch-Router-1.5B is a compact router LLM from Katanemo, fine-tuned from
23553+
Qwen2.5-1.5B-Instruct. Given a prompt and a set of user-defined route
23554+
policies (domain + action), it picks the best-matching policy name so
23555+
requests can be dispatched to the appropriate downstream model. Designed
23556+
for low-latency, high-throughput use inside the Arch proxy, it pairs
23557+
with LocalAI's router classifier as a preference-aligned alternative to
23558+
embedding/ColBERT-based routing on concrete, well-described policies.
23559+
tags:
23560+
- llm
23561+
- gguf
23562+
- qwen
23563+
- qwen2.5
23564+
- 1.5b
23565+
- router
23566+
- cpu
23567+
- gpu
23568+
overrides:
23569+
parameters:
23570+
model: Arch-Router-1.5B.Q4_K_M.gguf
23571+
files:
23572+
- filename: Arch-Router-1.5B.Q4_K_M.gguf
23573+
sha256: 9abe34414ebfe3921a1d157ed3ce8718e21e59a1f80693a33969a82ea40df636
23574+
uri: huggingface://mradermacher/Arch-Router-1.5B-GGUF/Arch-Router-1.5B.Q4_K_M.gguf
23575+
- !!merge <<: *arch-router-1_5b
23576+
name: arch-router-1.5b-q8
23577+
overrides:
23578+
parameters:
23579+
model: Arch-Router-1.5B.Q8_0.gguf
23580+
files:
23581+
- filename: Arch-Router-1.5B.Q8_0.gguf
23582+
sha256: 236fcf372bb25f314dafa1605d84566db60ddad98b889aaa072a3108ec48ef22
23583+
uri: huggingface://mradermacher/Arch-Router-1.5B-GGUF/Arch-Router-1.5B.Q8_0.gguf
2354323584
- name: dolphin-2.9-llama3-8b
2354423585
url: github:mudler/LocalAI/gallery/hermes-2-pro-mistral.yaml@master
2354523586
urls:

0 commit comments

Comments
 (0)