Skip to content

Commit f6e2e76

Browse files
authored
feat: add LiteLLM unified LLM provider (RFC-012) (#108)
Introduces a new provider: litellm that routes to 100+ LLM backends through LiteLLM's model-name prefix matching. Single provider replaces the need for separate openai_compat, anthropic, bedrock, vertex, and similar dedicated providers. Changes: - Create src/zettelforge/llm_providers/litellm_provider.py with LiteLLMProvider class implementing the LLMProvider protocol - Register 'litellm' provider in __init__.py (conditional on SDK) - Add litellm>=1.60.0 optional dependency to pyproject.toml - Update config.default.yaml with provider docs and examples - Add 11 unit tests: construction, generate, api_key, system prompt, json_mode, protocol satisfaction, registration check - Remove placeholder lines for openai_compat and anthropic (now superseded by LiteLLM) - Create docs/rfcs/RFC-012-litellm-unified-provider.md (369 lines) RFC: docs/rfcs/RFC-012-litellm-unified-provider.md
1 parent e41dca4 commit f6e2e76

6 files changed

Lines changed: 725 additions & 2 deletions

File tree

config.default.yaml

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,9 +148,8 @@ embedding:
148148
# Provider selects the backend registered in zettelforge.llm_providers:
149149
# local — in-process LLM inference (see local_backend below; requires zettelforge[local] or zettelforge[local-onnx])
150150
# ollama — Ollama HTTP API
151+
# litellm — LiteLLM routing to 100+ providers (RFC-012; requires zettelforge[litellm])
151152
# mock — test-only canned responses
152-
# openai_compat — any /v1/chat/completions endpoint (Phase 2, upcoming)
153-
# anthropic — Anthropic native SDK (Phase 3, upcoming)
154153
#
155154
# local_backend selects the in-process inference engine when provider=local:
156155
# llama-cpp-python — GGUF models via llama-cpp-python (default; requires zettelforge[local])
@@ -174,6 +173,19 @@ embedding:
174173
# filename: phi3-mini-4k-instruct-q4.onnx
175174
# provider: rocm # cpu, cuda, dml, rocm, openvino, coreml
176175
#
176+
# # LiteLLM — unified routing (requires pip install zettelforge[litellm])
177+
# # Model name determines the provider. Examples:
178+
# # gpt-4o, gpt-4o-mini → OpenAI
179+
# # claude-sonnet-4-20250514, claude-3-opus → Anthropic
180+
# # gemini/gemini-2.0-flash → Google Gemini
181+
# # groq/llama-3.3-70b-versatile → Groq
182+
# # together_ai/meta-llama/Llama-3.3-70B → Together AI
183+
# # openrouter/anthropic/claude-3.5-sonnet → OpenRouter
184+
# # bedrock/anthropic.claude-3-sonnet → AWS Bedrock
185+
# provider: litellm
186+
# model: gpt-4o
187+
# api_key: ${OPENAI_API_KEY}
188+
#
177189
# api_key supports ${ENV_VAR} references — never commit raw API keys.
178190
# api_key: ${OPENAI_API_KEY}
179191
#

0 commit comments

Comments
 (0)