Skip to content

Commit e895430

Browse files
committed
Merge Installation / Introduction docs
1 parent d5b07ff commit e895430

File tree

4 files changed

+235
-245
lines changed

4 files changed

+235
-245
lines changed
Lines changed: 63 additions & 128 deletions
Original file line numberDiff line numberDiff line change
@@ -1,163 +1,98 @@
11
---
2-
title: Introduction
3-
description: Introducing llms.py - Lightweight OpenAI compatible CLI and server gateway
2+
title: Installation
3+
description: How to install llms.py
44
---
55

6-
# Introducing llms.py 🚀
6+
## Using pip
77

8-
We're excited to announce **[llms.py](https://github.com/ServiceStack/llms)** - a super lightweight CLI tool and OpenAI-compatible server that acts as a **configurable gateway** over multiple configurable Large Language Model (LLM) providers.
8+
The easiest way to install llms.py is using pip:
99

10-
## 🎯 OpenRouter but Local
10+
<ShellCommand>pip install llms-py</ShellCommand>
1111

12-
**llms.py** is designed as a **unified gateway** that seamlessly connects you to multiple LLM providers through a single, consistent interface. Whether using cloud APIs or local models, `llms` provides intelligent routing and automatic failover to ensure your AI workflows connect to your chosen providers in your preferred priority - whether optimizing for cost, performance, or availability.
12+
## Using Docker
1313

14-
### ⚡ Ultra-Lightweight Architecture
14+
### Pre-built Docker Images
1515

16+
Pull and run the latest image:
1617

17-
1. **Simplicity**: Just one [llms.py](https://github.com/ServiceStack/llms/blob/main/llms/main.py) file (easily customizable)
18-
2. **Single Dependency**: Only requires `aiohttp` (Pillow optional)
19-
- Zero Dependencies for ComfyUI - Ideal for use in Custom Nodes
20-
3. **Flexibility**: Works with any OpenAI-compatible client or framework
21-
4. **Reliability**: Automatic failover ensures your workflows never break
22-
5. **Economy**: Intelligent routing minimizes API costs
23-
6. **Privacy**: Mix local and cloud models based on your data sensitivity
24-
7. **Future-Proof**: Easily add new providers as they emerge
25-
8. **No Setup**: Just download and use, configure preferred LLMs in [llms.json](https://github.com/ServiceStack/llms/blob/main/llms/llms.json)
18+
<ShellCommand>docker pull ghcr.io/servicestack/llms:latest</ShellCommand>
2619

27-
**llms.py** transforms the complexity of managing multiple LLM providers into a simple, unified experience.
28-
Whether you're researching capabilities of new models, building the next breakthrough AI application,
29-
or just want reliable access to the best models available, llms.py has you covered.
20+
<ShellCommand>docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latest</ShellCommand>
3021

31-
Get started today and avoid expensive cloud lock-ins with the freedom of provider-agnostic AI development! 🎉
22+
### Using docker compose (Recommended)
3223

33-
## 🌐 Configurable Multi-Provider Gateway
24+
1. Download the docker-compose.yml file:
3425

35-
Acts as an intelligent gateway that can route requests for 160+ models across:
26+
<ShellCommand>curl -O https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml</ShellCommand>
3627

37-
### Cloud Providers with Free Tiers
28+
2. Create a `.env` file with your API keys:
3829

39-
- OpenRouter
40-
- Groq
41-
- Codestral
42-
- Google
30+
```bash
31+
OPENROUTER_API_KEY=sk-or-...
32+
GROQ_API_KEY=gsk_...
33+
GOOGLE_FREE_API_KEY=AIza...
34+
```
4335

44-
### Premium Cloud Providers
36+
3. Start the service:
4537

46-
- OpenAI
47-
- Anthropic
48-
- Google
49-
- Grok
50-
- Qwen
51-
- Mistral
38+
<ShellCommand>docker compose up -d</ShellCommand>
5239

53-
### Local Providers
40+
### Build from Source
5441

55-
- Ollama
56-
- Restrict access to custom models
57-
- Or auto-discovery of installed models
42+
If you've cloned the repository:
5843

59-
### Custom Providers
44+
<ShellCommand>git clone https://github.com/ServiceStack/llms</ShellCommand>
6045

61-
Use JSON config to add any OpenAI-compatible API endpoints and models
46+
<ShellCommand>cd llms</ShellCommand>
6247

63-
## 🔄 Intelligent Request Routing
48+
<ShellCommand>docker compose -f docker-compose.local.yml up -d --build</ShellCommand>
6449

65-
- **Automatic Failover**: If one provider fails, automatically retry with the next available provider
66-
- **Cost Optimization**: Define free/cheap/local providers first to minimize costs
67-
- **Model Mapping**: Use unified model names that map to different provider-specific names
50+
## Setting Up API Keys
6851

69-
## 🚀 Key Features
52+
Set environment variables for the providers you want to use:
7053

71-
### 🎨 ChatGPT-like Web UI
54+
```bash
55+
export OPENROUTER_API_KEY="sk-or-..."
56+
export GROQ_API_KEY="gsk_..."
57+
export GOOGLE_FREE_API_KEY="AIza..."
58+
export ANTHROPIC_API_KEY="sk-ant-..."
59+
export OPENAI_API_KEY="sk-..."
60+
export GROK_API_KEY="xai-..."
61+
export DASHSCOPE_API_KEY="sk-..." # Qwen
62+
export ZAI_API_KEY="sk-..."
63+
export MISTRAL_API_KEY="..."
64+
```
7265

73-
A modern, fast, and privacy-focused web interface for interacting with all your LLMs.
66+
### API Key Reference
7467

75-
- **Offline & Private**: All data stored locally in SQLite
76-
- **Dark Mode**: Automatic or manual dark mode toggle
77-
- **Rich Markdown**: Full markdown support with syntax highlighting
78-
- **Search**: Quickly find past conversations
79-
- **Export/Import**: Backup and transfer chat histories
68+
| Provider | Environment Variable | Description |
69+
|----------|---------------------|-------------|
70+
| openrouter_free | `OPENROUTER_API_KEY` | OpenRouter FREE models |
71+
| groq | `GROQ_API_KEY` | Groq API key |
72+
| google_free | `GOOGLE_FREE_API_KEY` | Google FREE API key |
73+
| codestral | `CODESTRAL_API_KEY` | Codestral API key |
74+
| ollama | N/A | No API key required |
75+
| openrouter | `OPENROUTER_API_KEY` | OpenRouter API key |
76+
| google | `GOOGLE_API_KEY` | Google API key |
77+
| anthropic | `ANTHROPIC_API_KEY` | Anthropic API key |
78+
| openai | `OPENAI_API_KEY` | OpenAI API key |
79+
| grok | `GROK_API_KEY` | Grok (X.AI) API key |
80+
| qwen | `DASHSCOPE_API_KEY` | Qwen (Alibaba) key |
81+
| z.ai | `ZAI_API_KEY` | Z.ai API key |
82+
| mistral | `MISTRAL_API_KEY` | Mistral API key |
8083

81-
[Learn more about the Web UI →](/docs/features/web-ui)
84+
## Verification
8285

83-
### 🖥️ CLI Interface
86+
After installation, verify it's working:
8487

85-
- Interactive command-line tool
86-
- Support for all modalities (text, image, audio, files)
87-
- Custom system prompts
88-
- Raw JSON output mode
88+
<ShellCommand>llms ls</ShellCommand>
8989

90-
[Learn more about the CLI →](/docs/features/cli)
90+
## Updating
9191

92-
### 🎯 Multimodal Support
92+
To update to the latest version:
9393

94-
Process text, images, audio, and documents with capable models.
94+
<ShellCommand>pip install llms-py --upgrade</ShellCommand>
9595

96-
- **Text Generation**: Chat completions with any supported model
97-
- **Vision Models**: Process images through vision-capable models (GPT-4V, Gemini Vision, etc.)
98-
- **Audio Processing**: Handle audio inputs through audio-capable models
99-
- **Document Processing**: Analyze PDFs and documents with capable models
100-
- **Drag & Drop**: Easy file attachments in the UI
96+
For Docker:
10197

102-
[Learn more about Multimodal Support →](/docs/features/multimodal)
103-
104-
### Flexible Deployment Options
105-
- **CLI Tool**: Interactive command-line interface for quick queries
106-
- **HTTP Server**: OpenAI-compatible server at `http://localhost:{PORT}/v1/chat/completions`
107-
- **Python Module**: Import and use programmatically in your applications
108-
- **ComfyUI Node**: Embed directly in ComfyUI workflows
109-
110-
### Simple and Customizable
111-
- **Environment Variables**: Secure API key management
112-
- **Provider Management**: Easy enable/disable of providers
113-
- **Custom Models**: Define your own model aliases and mappings
114-
- **Unified Configuration**: Single [llms.json](https://github.com/ServiceStack/llms/blob/main/llms/llms.json) to configure all providers and models
115-
- Custom model aliases and mappings
116-
- Flexible chat templates
117-
- Environment variable support
118-
119-
[Learn more about Configuration →](docs/configuration)
120-
121-
### 🌐 OpenAI-Compatible API
122-
123-
- Drop-in replacement for OpenAI API
124-
- Works with any OpenAI-compatible client
125-
- Streaming support
126-
- Custom parameters
127-
128-
### 📊 Analytics & Monitoring
129-
130-
Track costs, usage, and performance across all providers.
131-
132-
- **Cost Tracking**: Per-message, per-thread, and monthly cost analytics
133-
- **Token Metrics**: Input/output token tracking
134-
- **Activity Logs**: Detailed request history
135-
- **Provider Stats**: Response times and reliability metrics
136-
137-
[Learn more about Analytics →](/docs/features/analytics)
138-
139-
### 🔄 Intelligent Provider Routing
140-
141-
Automatic failover and cost optimization across providers.
142-
143-
- **Multi-Provider**: Support for 10+ provider types with 160+ models
144-
- **Auto-Failover**: If one provider fails, automatically try the next
145-
- **Cost Optimization**: Route to free/cheap providers first
146-
- **Model Mapping**: Unified model names across providers
147-
148-
[Learn more about Providers →](/docs/features/providers)
149-
150-
### 🔒 Security
151-
152-
- Optional GitHub OAuth authentication
153-
- User access restrictions
154-
155-
[Learn more about GitHub OAuth →](/docs/deployment/github-oauth)
156-
157-
---
158-
159-
## Links
160-
161-
- 📚 [GitHub Repository](https://github.com/ServiceStack/llms)
162-
- 📦 [PyPI Package](https://pypi.org/project/llms-py/)
163-
- 🔧 [Source Code](https://github.com/ServiceStack/llms)
98+
<ShellCommand>docker pull ghcr.io/servicestack/llms:latest</ShellCommand>

content/docs/getting-started/installation.mdx

Lines changed: 0 additions & 106 deletions
This file was deleted.

content/docs/getting-started/meta.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
"defaultOpen": true,
44
"pages": [
55
"index",
6-
"installation",
76
"quick-start"
87
]
98
}

0 commit comments

Comments
 (0)