Skip to content

Commit f7ed623

Browse files
committed
feat: Implement the new agent-gateway application, including its source code, tests, and deployment configurations.
1 parent 052de9a commit f7ed623

22 files changed

Lines changed: 1869 additions & 0 deletions

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,6 +258,16 @@ KnowCode enables token-efficient IDE agent workflows. When your IDE agent needs
258258
}
259259
```
260260

261+
## AI Gateway Scaffold (Extractable)
262+
263+
For OpenAPI-to-tool orchestration through LiteLLM, this repository now includes a self-contained gateway app at `apps/agent-gateway/`.
264+
265+
- Source code: `apps/agent-gateway/src/agent_gateway/`
266+
- Setup + local run: `apps/agent-gateway/README.md`
267+
- Clean repo split playbook: `apps/agent-gateway/EXTRACTION.md`
268+
269+
The gateway intentionally integrates with KnowCode only over HTTP (`/openapi.json` and `/api/v1/*`) so it can be moved to a separate repository without code changes.
270+
261271
**Token Savings:**
262272
- Simple "locate" queries → **100% savings** (answered locally)
263273
- Code explanations → **60-80% savings** (precise context only)

apps/agent-gateway/.env.example

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
KNOWCODE_API_BASE_URL=http://127.0.0.1:8000
2+
LITELLM_BASE_URL=http://127.0.0.1:4000
3+
LITELLM_API_KEY=sk-local-proxy
4+
AGENT_DEFAULT_MODEL=gemini/gemini-3-flash-preview
5+
AGENT_MAX_TOOL_ROUNDS=4
6+
AGENT_TOOL_TIMEOUT_SECONDS=30
7+
AGENT_OPENAPI_CACHE_TTL_SECONDS=300
8+
AGENT_ALLOWED_TOOL_NAMES=query_context,search,get_context,trace_calls
9+
AGENT_DEFAULT_TAGS=knowcode,context
10+
HOST=127.0.0.1
11+
PORT=8081

apps/agent-gateway/Dockerfile

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
FROM python:3.12-slim AS builder
2+
WORKDIR /app
3+
4+
RUN pip install --no-cache-dir uv
5+
COPY pyproject.toml README.md ./
6+
COPY src ./src
7+
RUN uv pip install --system --no-cache -e .
8+
9+
FROM python:3.12-slim AS runner
10+
WORKDIR /app
11+
12+
RUN useradd -r -u 1001 appuser
13+
14+
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
15+
COPY --from=builder /usr/local/bin /usr/local/bin
16+
COPY src ./src
17+
COPY pyproject.toml README.md ./
18+
19+
ENV PYTHONUNBUFFERED=1
20+
EXPOSE 8081
21+
22+
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8081/health')" || exit 1
23+
24+
USER appuser
25+
CMD ["python", "-m", "agent_gateway.main"]

apps/agent-gateway/EXTRACTION.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Clean Separation Playbook
2+
3+
## Why this structure is clean
4+
5+
- No imports from the parent project codebase
6+
- No dependence on KnowCode internals or files
7+
- All coupling is network-level contract (`/openapi.json`, `/api/v1/*`)
8+
9+
## Repository split steps
10+
11+
1. Copy folder contents to a new repository root.
12+
2. Rename project package if desired (optional).
13+
3. Create CI:
14+
- `pytest tests`
15+
- `ruff check src tests`
16+
4. Deploy with the same env variables.
17+
5. Point `KNOWCODE_API_BASE_URL` to remote KnowCode service.
18+
19+
## Anti-patterns to avoid
20+
21+
- Importing `src/knowcode` directly.
22+
- Reading parent repo files at runtime.
23+
- Sharing secrets/config through relative files outside this app.

apps/agent-gateway/README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Agent Gateway (Extractable App)
2+
3+
This app is a standalone orchestration layer that sits between your client and KnowCode.
4+
5+
It is intentionally isolated so you can move it into a separate GitHub repository later
6+
without changing application logic.
7+
8+
## Responsibilities
9+
10+
- Fetch KnowCode OpenAPI schema (`/openapi.json`)
11+
- Translate selected endpoints into OpenAI/LiteLLM tool schemas
12+
- Call LiteLLM (`/v1/chat/completions`) with tagged metadata
13+
- Execute tool calls against KnowCode REST endpoints
14+
- Return final answer, usage data, response cost, and tool execution trace
15+
16+
## Explicit Boundaries
17+
18+
- `apps/agent-gateway` imports **nothing** from `src/knowcode`
19+
- Integration happens only over HTTP:
20+
- KnowCode API (`KNOWCODE_API_BASE_URL`)
21+
- LiteLLM proxy (`LITELLM_BASE_URL`)
22+
- Runtime config comes only from environment variables
23+
24+
These boundaries are what make repo extraction clean.
25+
26+
## Local Run
27+
28+
```bash
29+
# 1) Start KnowCode API in another terminal (from repo root)
30+
uvicorn knowcode.api.main:create_app --factory --port 8000
31+
32+
# 2) Install gateway deps
33+
uv sync --project apps/agent-gateway --extra dev
34+
35+
# 3) Run gateway
36+
uv run --project apps/agent-gateway agent-gateway
37+
```
38+
39+
Gateway endpoints:
40+
- `GET /health`
41+
- `GET /api/v1/tools`
42+
- `POST /api/v1/chat`
43+
44+
## Docker Run
45+
46+
```bash
47+
cd apps/agent-gateway
48+
docker compose up --build
49+
```
50+
51+
This starts LiteLLM + the gateway. KnowCode is expected at
52+
`http://host.docker.internal:8000` by default.
53+
54+
## Extraction Checklist (To New Repo)
55+
56+
1. Copy `apps/agent-gateway` into a new repository root.
57+
2. Keep env variables and names unchanged.
58+
3. Move CI workflow to run tests in `tests/`.
59+
4. Deploy as an independent service (no shared filesystem needed).
60+
5. Set `KNOWCODE_API_BASE_URL` to deployed KnowCode API URL.
61+
62+
No code changes should be required if you keep the env contract.
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
services:
2+
litellm:
3+
image: ghcr.io/berriai/litellm:main-stable
4+
command: ["--config", "/app/config.yaml", "--port", "4000"]
5+
environment:
6+
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-local-proxy}
7+
LITELLM_MAX_BUDGET: ${LITELLM_MAX_BUDGET:-10}
8+
GOOGLE_API_KEY_1: ${GOOGLE_API_KEY_1}
9+
volumes:
10+
- ./litellm.config.yaml:/app/config.yaml:ro
11+
ports:
12+
- "4000:4000"
13+
14+
agent-gateway:
15+
build: .
16+
environment:
17+
KNOWCODE_API_BASE_URL: ${KNOWCODE_API_BASE_URL:-http://host.docker.internal:8000}
18+
LITELLM_BASE_URL: http://litellm:4000
19+
LITELLM_API_KEY: ${LITELLM_MASTER_KEY:-sk-local-proxy}
20+
AGENT_DEFAULT_MODEL: ${AGENT_DEFAULT_MODEL:-gemini/gemini-3-flash-preview}
21+
PORT: 8081
22+
depends_on:
23+
- litellm
24+
extra_hosts:
25+
- "host.docker.internal:host-gateway"
26+
ports:
27+
- "8081:8081"
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
model_list:
2+
- model_name: gemini/gemini-3-flash-preview
3+
litellm_params:
4+
model: gemini/gemini-3-flash-preview
5+
api_key: "os.environ/GOOGLE_API_KEY_1"
6+
7+
- model_name: ollama/llama3.1:8b
8+
litellm_params:
9+
model: ollama/llama3.1:8b
10+
api_base: "http://host.docker.internal:11434"
11+
12+
router_settings:
13+
fallbacks:
14+
- gemini/gemini-3-flash-preview:
15+
- ollama/llama3.1:8b
16+
17+
general_settings:
18+
master_key: "os.environ/LITELLM_MASTER_KEY"
19+
max_budget: "os.environ/LITELLM_MAX_BUDGET"

apps/agent-gateway/pyproject.toml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
[project]
2+
name = "knowcode-agent-gateway"
3+
version = "0.1.0"
4+
description = "Self-contained AI gateway/orchestrator for KnowCode + LiteLLM"
5+
readme = "README.md"
6+
requires-python = ">=3.10, <3.13"
7+
dependencies = [
8+
"fastapi>=0.100.0",
9+
"uvicorn>=0.22.0",
10+
]
11+
12+
[project.optional-dependencies]
13+
dev = [
14+
"pytest>=7.4",
15+
"ruff>=0.1.5",
16+
]
17+
18+
[project.scripts]
19+
agent-gateway = "agent_gateway.main:main"
20+
21+
[build-system]
22+
requires = ["hatchling"]
23+
build-backend = "hatchling.build"
24+
25+
[tool.hatch.build.targets.wheel]
26+
packages = ["src/agent_gateway"]
27+
28+
[tool.ruff]
29+
line-length = 88
30+
target-version = "py310"
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""KnowCode agent gateway package."""
2+
3+
from agent_gateway.app import create_app
4+
5+
__all__ = ["create_app"]
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
"""FastAPI application for the extractable KnowCode agent gateway."""
2+
3+
from __future__ import annotations
4+
5+
from typing import Any, Dict, List, Optional
6+
7+
from fastapi import FastAPI, HTTPException, Query
8+
9+
from agent_gateway.litellm_client import LiteLLMError
10+
from agent_gateway.models import ChatRequest, ChatResponse, ErrorResponse
11+
from agent_gateway.openapi_tools import OpenAPIFetchError, OpenAPITranslationError
12+
from agent_gateway.orchestrator import AgentOrchestrator
13+
from agent_gateway.settings import GatewaySettings
14+
15+
16+
17+
def create_app() -> FastAPI:
18+
"""Create and wire the gateway API application."""
19+
20+
settings = GatewaySettings.from_env()
21+
orchestrator = AgentOrchestrator(settings=settings)
22+
23+
app = FastAPI(
24+
title="KnowCode Agent Gateway",
25+
description="LiteLLM-backed tool execution gateway for KnowCode",
26+
version="0.1.0",
27+
)
28+
29+
@app.get("/health")
30+
def health() -> Dict[str, str]:
31+
return {"status": "ok"}
32+
33+
@app.get("/api/v1/config")
34+
def config_snapshot() -> Dict[str, Any]:
35+
return {
36+
"knowcode_api_base_url": settings.knowcode_api_base_url,
37+
"litellm_base_url": settings.litellm_base_url,
38+
"default_model": settings.default_model,
39+
"max_tool_rounds": settings.max_tool_rounds,
40+
"allowed_tool_names": list(settings.allowed_tool_names),
41+
"default_tags": list(settings.default_tags),
42+
}
43+
44+
@app.get("/api/v1/tools")
45+
def list_tools(
46+
names: Optional[List[str]] = Query(
47+
default=None,
48+
description="Optional repeated query arg to filter tool names",
49+
)
50+
) -> Dict[str, Any]:
51+
try:
52+
tools = orchestrator.list_tools(requested_tool_names=names)
53+
except (OpenAPIFetchError, OpenAPITranslationError, ValueError) as exc:
54+
raise HTTPException(status_code=502, detail=str(exc)) from exc
55+
return {"tools": tools, "count": len(tools)}
56+
57+
@app.post(
58+
"/api/v1/chat",
59+
response_model=ChatResponse,
60+
responses={
61+
400: {"model": ErrorResponse},
62+
502: {"model": ErrorResponse},
63+
500: {"model": ErrorResponse},
64+
},
65+
)
66+
def chat(request: ChatRequest) -> ChatResponse:
67+
try:
68+
return orchestrator.run(request)
69+
except ValueError as exc:
70+
raise HTTPException(status_code=400, detail=str(exc)) from exc
71+
except (OpenAPIFetchError, OpenAPITranslationError, LiteLLMError, RuntimeError) as exc:
72+
raise HTTPException(status_code=502, detail=str(exc)) from exc
73+
except Exception as exc: # pragma: no cover
74+
raise HTTPException(status_code=500, detail=str(exc)) from exc
75+
76+
return app

0 commit comments

Comments
 (0)