Local LLM

[Genudza]

Hi everyone! I’m a bit confused about something. Can I use only local models without connecting to OpenAI?

[Hackerbone]

Hey @Genudza currently we don't support this but sure we have this in the pipeline! We'll add an integration to this.

[Pseudosance]

I went thru the hurdle of getting a local model to run with copilot. Documenting this here in case others have as much trouble as me:

Note: Host machine is Windows and I am running on Kali on WSL2 and using it as my attack box.

Connecting Pentest-Copilot to Ollama: A Setup Guide

Pentest-copilot runs as a Docker Compose stack, meaning its backend container lives in an isolated network namespace. This creates a networking puzzle when connecting it to Ollama — whether Ollama is running locally or on a remote machine.

---

Prerequisites

• Pentest-copilot running via docker compose up -d
• Ollama installed and running with at least one model pulled
• Access to edit docker-compose.yml

---

Scenario 1: Ollama running locally on the same machine

The Problem

Docker containers can't reach localhost or 127.0.0.1 on the host — those resolve to the container itself, not your machine.

Fix: Add host-gateway to docker-compose.yml

Edit docker-compose.yml and add extra_hosts to the backend service:

services:
  backend:
    extra_hosts:
      - "host-gateway:host-gateway"

Restart the stack:

docker compose down && docker compose up -d

Verify the backend can reach Ollama:

docker exec -it pentest-copilot-backend-1 wget -qO- http://host-gateway:11434/api/tags

You should get a JSON list of your models.

Configure in Settings → Models

Base URL:  http://host-gateway:11434/v1
API Key:   ollama
Model:     your-model-name:tag

Make sure Ollama is listening on all interfaces

By default Ollama only binds to 127.0.0.1. Check:

ss -tlnp | grep 11434

If it shows 127.0.0.1:11434 only:

sudo systemctl edit ollama --force

Add:

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

Then restart:

sudo systemctl daemon-reload && sudo systemctl restart ollama

---

Scenario 2: Ollama running on a remote machine (Tailscale or LAN)

If Ollama is on a different machine — a home server, a remote GPU rig, etc. — you need a proxy to bridge the Docker network to that remote host.

Step 1: Confirm Ollama is reachable from your host

curl http://<remote-ip>:11434/api/tags

If using Tailscale, <remote-ip> is the Tailscale IP (100.x.x.x). On a LAN, use the local IP.

Step 2: Forward the port using socat

sudo apt install -y socat

socat TCP-LISTEN:11434,fork,reuseaddr TCP:<remote-ip>:11434 &

Step 3: Add host-gateway to docker-compose.yml (same as Scenario 1)

services:
  backend:
    extra_hosts:
      - "host-gateway:host-gateway"

docker compose down && docker compose up -d
docker exec -it pentest-copilot-backend-1 wget -qO- http://host-gateway:11434/api/tags

Step 4: Configure in Settings → Models

Base URL:  http://host-gateway:11434/v1
API Key:   ollama

Make socat persistent across reboots

echo 'socat TCP-LISTEN:11434,fork,reuseaddr TCP:<remote-ip>:11434 &' >> ~/.bashrc

---

Troubleshooting

Symptom	Likely Cause	Fix
Connection refused on localhost	Container loopback ≠ host	Use host-gateway
Host is unreachable on host IP	Docker iptables isolation	Use host-gateway instead of hardcoded IP
Resource temporarily unavailable	Proxy found but upstream unreachable	Check socat is running and remote Ollama is accessible
Agent gives steps instead of executing	Model safety alignment	Try a larger model or an abliterated variant

---

Why host-gateway Works

Docker's host-gateway keyword resolves to the host machine's internal IP from within any container on that network. Adding it to extra_hosts gives the backend container a reliable, stable hostname that always points to your host — no hardcoded IPs, no extra proxies needed for the local scenario.

[Hackerbone]

@Pseudosance thanks for the elaborate explanation and debugging for local models network resolution issue, do you wish to implement this fix and create a PR?