File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -92,11 +92,11 @@ mlx-lm vs vllm-mlx at temperature=0: 50/50 identical (reasoning) and 30/30 ident
9292
9393## Supported backends
9494
95- | Backend | Type | Use case |
96- | --- | --- | --- |
97- | ** mlx-lm** | In-process | Local Apple Silicon inference with logprobs |
98- | ** llama. cpp** | HTTP | ` llama-server ` via ` /completion ` endpoint |
99- | ** vllm-mlx** | HTTP | Continuous batching on Apple Silicon |
95+ | Backend | Type | Use case |
96+ | ------------------- | --- | --- |
97+ | ** mlx-lm** | In-process | Local Apple Silicon inference with logprobs |
98+ | ** llama- cpp** | HTTP | ` llama-server ` via ` /completion ` endpoint |
99+ | ** vllm-mlx** | HTTP | Continuous batching on Apple Silicon |
100100| ** openai-compat** | HTTP | Any OpenAI-compatible server (vLLM, SGLang, Ollama) |
101101
102102See the [ backends documentation] ( https://nullpointerdepressivedisorder.github.io/infer-check/backends/ ) for setup and configuration details.
Original file line number Diff line number Diff line change 44
55## Overview
66
7- | Backend | Type | Default URL | Use case |
8- | ---------| ------| -------------| ----------|
9- | ** mlx-lm** | In-process | (local) | Local Apple Silicon inference with logprobs |
10- | ** llama. cpp** | HTTP | ` http://127.0.0.1:8080 ` | llama-server via ` /completion ` endpoint |
11- | ** vllm-mlx** | HTTP | ` http://127.0.0.1:8000 ` | Continuous batching on Apple Silicon |
7+ | Backend | Type | Default URL | Use case |
8+ | ------------------- | ------| -------------| ----------|
9+ | ** mlx-lm** | In-process | (local) | Local Apple Silicon inference with logprobs |
10+ | ** llama- cpp** | HTTP | ` http://127.0.0.1:8080 ` | llama-server via ` /completion ` endpoint |
11+ | ** vllm-mlx** | HTTP | ` http://127.0.0.1:8000 ` | Continuous batching on Apple Silicon |
1212| ** openai-compat** | HTTP | ` http://127.0.0.1:11434/v1 ` | Any OpenAI-compatible server (vLLM, SGLang, Ollama) |
1313
1414## mlx-lm
You can’t perform that action at this time.
0 commit comments