Skip to content

Commit 9db8623

Browse files
authored
docs(custom-inference): document local model tunneling via ngrok (#139)
1 parent db3d079 commit 9db8623

1 file changed

Lines changed: 8 additions & 0 deletions

File tree

src/content/docs/agent-platform/inference/custom-inference-endpoint.mdx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,14 @@ When you explicitly select an endpoint-routed model from the model picker, Warp
5454

5555
The configuration flow mirrors the [Bring Your Own API Key](/agent-platform/inference/bring-your-own-api-key/) setup, so the steps will feel familiar if you've already configured BYOK.
5656

57+
## Using local models
58+
59+
Warp routes inference requests through its servers, so endpoint URLs must be publicly accessible. `localhost`, `127.0.0.1`, and other private or local network URLs are rejected when configuring a custom inference endpoint.
60+
61+
To route through a model running on your own machine (for example, Ollama, LM Studio, vLLM, or llama.cpp), expose it through a tunneling service like [ngrok](https://ngrok.com/) and use the public tunnel URL as the base URL in your endpoint configuration.
62+
63+
For example, with a default Ollama install listening on port `11434`, run `ngrok http 11434` and use the resulting `https://*.ngrok-free.app/v1` URL as your endpoint. Other tunneling services that produce a publicly reachable HTTPS URL (Cloudflare Tunnel, Tailscale Funnel, and similar) work the same way.
64+
5765
## Billing behavior
5866

5967
### Warp AI credits

0 commit comments

Comments
 (0)