|
23540 | 23540 | - python |
23541 | 23541 | parameters: |
23542 | 23542 | model: bge-m3-colbert |
| 23543 | +- &arch-router-1_5b |
| 23544 | + url: github:mudler/LocalAI/gallery/chatml.yaml@master |
| 23545 | + name: arch-router-1.5b-q4 |
| 23546 | + icon: https://cdn-avatars.huggingface.co/v1/production/uploads/66b681906c8d3b36786b764c/uyP7mxDVv0HbV9Hv_KfHk.jpeg |
| 23547 | + license: other |
| 23548 | + urls: |
| 23549 | + - https://huggingface.co/katanemo/Arch-Router-1.5B |
| 23550 | + - https://huggingface.co/mradermacher/Arch-Router-1.5B-GGUF |
| 23551 | + description: | |
| 23552 | + Arch-Router-1.5B is a compact router LLM from Katanemo, fine-tuned from |
| 23553 | + Qwen2.5-1.5B-Instruct. Given a prompt and a set of user-defined route |
| 23554 | + policies (domain + action), it picks the best-matching policy name so |
| 23555 | + requests can be dispatched to the appropriate downstream model. Designed |
| 23556 | + for low-latency, high-throughput use inside the Arch proxy, it pairs |
| 23557 | + with LocalAI's router classifier as a preference-aligned alternative to |
| 23558 | + embedding/ColBERT-based routing on concrete, well-described policies. |
| 23559 | + tags: |
| 23560 | + - llm |
| 23561 | + - gguf |
| 23562 | + - qwen |
| 23563 | + - qwen2.5 |
| 23564 | + - 1.5b |
| 23565 | + - router |
| 23566 | + - cpu |
| 23567 | + - gpu |
| 23568 | + overrides: |
| 23569 | + parameters: |
| 23570 | + model: Arch-Router-1.5B.Q4_K_M.gguf |
| 23571 | + files: |
| 23572 | + - filename: Arch-Router-1.5B.Q4_K_M.gguf |
| 23573 | + sha256: 9abe34414ebfe3921a1d157ed3ce8718e21e59a1f80693a33969a82ea40df636 |
| 23574 | + uri: huggingface://mradermacher/Arch-Router-1.5B-GGUF/Arch-Router-1.5B.Q4_K_M.gguf |
| 23575 | +- !!merge <<: *arch-router-1_5b |
| 23576 | + name: arch-router-1.5b-q8 |
| 23577 | + overrides: |
| 23578 | + parameters: |
| 23579 | + model: Arch-Router-1.5B.Q8_0.gguf |
| 23580 | + files: |
| 23581 | + - filename: Arch-Router-1.5B.Q8_0.gguf |
| 23582 | + sha256: 236fcf372bb25f314dafa1605d84566db60ddad98b889aaa072a3108ec48ef22 |
| 23583 | + uri: huggingface://mradermacher/Arch-Router-1.5B-GGUF/Arch-Router-1.5B.Q8_0.gguf |
23543 | 23584 | - name: dolphin-2.9-llama3-8b |
23544 | 23585 | url: github:mudler/LocalAI/gallery/hermes-2-pro-mistral.yaml@master |
23545 | 23586 | urls: |
|
0 commit comments