initial commit: extract llm-gateway as standalone OSS project

Author: simonmorley

.env.example

@@ -0,0 +1,22 @@
+# Provider selection: auto, deepseek, gemini, openai, anthropic
+LLM_PROVIDER=auto
+
+# DeepSeek
+DEEPSEEK_API_KEY=
+DEEPSEEK_MODEL=deepseek-chat
+
+# Google Gemini
+GEMINI_API_KEY=
+GEMINI_MODEL=gemini-2.0-flash
+
+# OpenAI (also required for /embed endpoint)
+OPENAI_API_KEY=
+OPENAI_MODEL=gpt-4o-mini
+
+# Anthropic
+ANTHROPIC_API_KEY=
+ANTHROPIC_MODEL=claude-3-5-sonnet-20241022
+
+# Server
+PORT=8090
+LOG_LEVEL=INFO

.github/workflows/test.yml

@@ -0,0 +1,18 @@
+name: Tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install -r requirements.txt
+      - run: pytest --cov=. --cov-report=term-missing --cov-fail-under=90

.gitignore

@@ -0,0 +1,12 @@
+venv/
+__pycache__/
+*.pyc
+*.pyo
+.pytest_cache/
+.coverage
+htmlcov/
+*.egg-info/
+dist/
+build/
+.env
+.env.local

CONTRIBUTING.md

@@ -0,0 +1,30 @@
+# Contributing
+
+## Getting Started
+
+1. Fork the repository
+2. Create a feature branch: `git checkout -b my-feature`
+3. Install dependencies: `pip install -r requirements.txt`
+
+## Development Workflow
+
+This project follows test-driven development:
+
+1. **Write a failing test first** — describe the behaviour you want
+2. **Write minimal code to pass** — no more than needed
+3. **Refactor** — clean up with tests still passing
+
+## Running Tests
+
+```bash
+pytest -v
+pytest --cov=. --cov-report=term-missing
+```
+
+Coverage must stay above 90% for new code.
+
+## Submitting a PR
+
+- Keep changes focused — one feature or fix per PR
+- Ensure all tests pass and coverage does not drop
+- Write a clear PR description explaining what and why

Dockerfile

@@ -0,0 +1,18 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# Install dependencies
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY . .
+
+# Create non-root user
+RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
+USER appuser
+
+EXPOSE 8090
+
+CMD ["python", "main.py"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 NullRabbit
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,210 @@
+# LLM Gateway
+
+Multi-provider LLM gateway with automatic fallback and cost tracking. Provides a single HTTP API that routes requests across DeepSeek, Gemini, OpenAI, and Anthropic — trying cheaper providers first and falling back automatically on failure.
+
+## Quick Start
+
+```bash
+# Install dependencies
+python -m venv venv
+source venv/bin/activate
+pip install -r requirements.txt
+
+# Set up at least one provider
+export LLM_PROVIDER=deepseek
+export DEEPSEEK_API_KEY=your-key
+export DEEPSEEK_MODEL=deepseek-chat
+
+# Start the server
+python main.py
+```
+
+The server runs on `http://localhost:8090` by default.
+
+## API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/classify` | POST | Classify items using AI (returns JSON) |
+| `/plan` | POST | Generate structured plans using AI (returns JSON) |
+| `/embed` | POST | Generate text embeddings (requires OPENAI_API_KEY) |
+| `/v1/chat/completions` | POST | OpenAI-compatible chat with optional tool call support |
+| `/health` | GET | Health check with provider status |
+
+### POST /classify
+
+Send a prompt, get back a JSON classification response.
+
+```bash
+curl -X POST http://localhost:8090/classify \
+  -H "Content-Type: application/json" \
+  -d '{"prompt": "Classify these items: ..."}'
+```
+
+### POST /plan
+
+Generate a structured plan from context and a system prompt.
+
+```bash
+curl -X POST http://localhost:8090/plan \
+  -H "Content-Type: application/json" \
+  -d '{
+    "context": {"task": "...", "constraints": []},
+    "system_prompt": "You are a planner. Return JSON."
+  }'
+```
+
+### POST /embed
+
+Generate text embeddings using OpenAI's embedding models.
+
+```bash
+curl -X POST http://localhost:8090/embed \
+  -H "Content-Type: application/json" \
+  -d '{"text": "text to embed"}'
+```
+
+Request body:
+- `text`: String or list of strings to embed
+- `model`: Embedding model (default: `text-embedding-ada-002`)
+
+Response:
+```json
+{
+  "embeddings": [[0.1, 0.2, ...]],
+  "model": "text-embedding-ada-002",
+  "dimensions": 1536,
+  "ai_call_log": {
+    "provider": "openai",
+    "model": "text-embedding-ada-002",
+    "prompt_tokens": 5,
+    "completion_tokens": 0,
+    "cost_microcents": 1,
+    "latency_ms": 150,
+    "success": true
+  }
+}
+```
+
+### POST /v1/chat/completions
+
+OpenAI-compatible endpoint supporting optional tool calls. Provider-specific translation (e.g. Anthropic tool format) is handled transparently.
+
+```bash
+curl -X POST http://localhost:8090/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "messages": [{"role": "user", "content": "Hello"}]
+  }'
+```
+
+### GET /health
+
+Check service health and provider status.
+
+```bash
+curl http://localhost:8090/health
+```
+
+Response:
+```json
+{
+  "status": "healthy",
+  "providers": [{"name": "deepseek", "model": "deepseek-chat"}],
+  "embeddings_available": true
+}
+```
+
+## Configuration
+
+All configuration is via environment variables. Copy `.env.example` to `.env` and fill in your keys.
+
+### Provider Selection
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `LLM_PROVIDER` | `auto` | Provider: `auto`, `deepseek`, `gemini`, `openai`, `anthropic` |
+
+When `LLM_PROVIDER=auto`, providers are tried in cost-effectiveness order:
+1. DeepSeek — $0.12/1M input, $0.20/1M output
+2. Gemini — $0.10/1M input, $0.40/1M output
+3. OpenAI — $0.15/1M input, $0.60/1M output
+4. Anthropic — $3/1M input, $15/1M output
+
+### Provider API Keys
+
+| Variable | Description |
+|----------|-------------|
+| `DEEPSEEK_API_KEY` | DeepSeek API key |
+| `DEEPSEEK_MODEL` | DeepSeek model (e.g., `deepseek-chat`) |
+| `GEMINI_API_KEY` | Google Gemini API key |
+| `GEMINI_MODEL` | Gemini model (e.g., `gemini-2.0-flash`) |
+| `OPENAI_API_KEY` | OpenAI API key (also required for `/embed`) |
+| `OPENAI_MODEL` | OpenAI model (e.g., `gpt-4o-mini`) |
+| `ANTHROPIC_API_KEY` | Anthropic API key |
+| `ANTHROPIC_MODEL` | Anthropic model (e.g., `claude-3-5-sonnet-20241022`) |
+
+At least one provider must have both API key and model configured.
+
+### Service Settings
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `PORT` | `8090` | HTTP port |
+| `LOG_LEVEL` | `INFO` | Logging level |
+
+## Development
+
+### Running Tests
+
+```bash
+# Run all tests
+pytest -v
+
+# Run with coverage
+pytest --cov=. --cov-report=term-missing
+
+# Run specific test file
+pytest tests/test_providers.py -v
+```
+
+### Docker
+
+```bash
+# Build
+docker build -t llm-gateway .
+
+# Run
+docker run -p 8090:8090 \
+  -e LLM_PROVIDER=auto \
+  -e DEEPSEEK_API_KEY=key \
+  -e DEEPSEEK_MODEL=deepseek-chat \
+  llm-gateway
+```
+
+## Architecture
+
+```
+┌─────────────┐     ┌─────────────┐     ┌─────────────┐
+│ Your Svc A  │     │ Your Svc B  │     │ Your Svc C  │
+│             │     │             │     │             │
+└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
+       │ HTTP              │ HTTP              │ HTTP
+       ▼                   ▼                   ▼
+┌──────────────────────────────────────────────────────┐
+│                  llm-gateway (Python)                 │
+│  ┌────────────────────────────────────────────────┐  │
+│  │ Providers: DeepSeek | Gemini | OpenAI | Anthropic│ │
+│  │ Features: Auto-fallback, Cost tracking, Retries  │ │
+│  │ Endpoints: /plan, /classify, /embed, /health    │ │
+│  └────────────────────────────────────────────────┘  │
+└──────────────────────────────────────────────────────┘
+```
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## License
+
+MIT — see [LICENSE](LICENSE).