Skip to content

Commit 0670736

Browse files
committed
Add custom build docs
1 parent 6bcb5bf commit 0670736

File tree

3 files changed

+94
-0
lines changed

3 files changed

+94
-0
lines changed
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
title: Custom llms .py Build
3+
description: How to create a custom build of llms .py with only the extensions you need
4+
---
5+
6+
## Lean Core + Extensibility
7+
8+
To reduce bloat, **llms .py** was intentionally built around a lean core with support for extensibility. Most major features are encapsulated in **extension folders** that can be layered on to create a custom build with only the features you need.
9+
10+
### Benefits of This Approach
11+
12+
- **Minimal Footprint** - Deploy only what you need, reducing dependencies and attack surface
13+
- **Faster Startup** - Fewer modules to load means quicker initialization times
14+
- **Easier Maintenance** - Isolated extensions are simpler to update, debug, and replace
15+
- **Flexible Deployment** - Create purpose-built distributions for different use cases (e.g: CLI-only, API server)
16+
- **Cleaner Codebase** - Separation of concerns keeps the core focused and extensions self-contained
17+
18+
## Creating a Custom Build
19+
20+
The [custom-build](https://github.com/ServiceStack/llms/tree/main/custom-build) directory demonstrates how to create a custom build of the llms .py CLI/Server with a curated set of extensions. This allows you to deploy a lightweight version of the application with only the specific functionality you need.
21+
22+
## Custom Builder Script
23+
24+
Use `./build.sh` to create a custom build of llms .py with only the extensions you want, where running `./build.sh` without any arguments:
25+
26+
```bash
27+
cd custom-build
28+
29+
./build.sh
30+
```
31+
32+
Generates a minimal custom build inside a new `llms` directory containing just the [providers extension](/docs/extensions/built-in#providers-extension) and necessary configuration files:
33+
34+
```
35+
llms/
36+
├── .llms/ # New ~/.llms home for this build
37+
├── extensions/ # Built-in Extensions directory
38+
│ └── providers/ # Built-in providers extension
39+
├── llms.json # llms.json config
40+
├── providers.json # models.dev providers.json config
41+
├── main.py # llms.py implementation
42+
├── llms.sh # llms.sh startup script
43+
```
44+
45+
This minimal encapsulated build doesn't contain any UI or tools, but does contain the [providers extension](/docs/extensions/built-in#providers-extension) with support for all **24 providers** and their **530+ models** in both [CLI mode](/docs/features/cli) and its Open AI Compatible `/v1/chat/completions` endpoint.
46+
47+
### Startup Script
48+
49+
Once built you would use the `llms.sh` startup script for all commands instead of the `llms` command, e.g:
50+
51+
```bash
52+
./llms.sh "What's the Capital of France?"
53+
```
54+
55+
### Minimal UI Build
56+
57+
Set the `UI` environment variable to `1` for the minimal UI build which includes the [app](/docs/extensions/built-in#app-extension) and [tools](/docs/extensions/tools) extensions.
58+
59+
```bash
60+
UI=1 ./build.sh
61+
```
62+
63+
### Custom Extensions
64+
65+
For a custom build set the `EXTENSIONS` environment variable to a comma separated list of extensions to include in the build. E.g. this includes the same as as the Minimal UI build:
66+
67+
```bash
68+
EXTENSIONS="providers,app,tools" ./build.sh
69+
```
70+
71+
> Including any extension that requires a UI will automatically include it in the build.
72+
73+
### Add external extensions
74+
75+
With [full CLI support](/docs/features/cli), your custom build can use the [extension management commands](/docs/extensions) to manage installed extensions, e.g., you can add external extensions to your custom build by using the `llms.sh --add` command.
76+
77+
Preview available extensions:
78+
79+
```bash
80+
./llms.sh --add
81+
```
82+
83+
Add a specific extension:
84+
85+
```bash
86+
./llms.sh --add xmas
87+
```
88+
89+
## Built-in Extensions
90+
91+
You can copy any of these built-in extensions from the source [llms/extensions](https://github.com/ServiceStack/llms/tree/main/llms/extensions) directory - see [Built-in Extensions](/docs/extensions/built-in) for the list of built-in extensions you might want to include in your custom build.

content/docs/deployment/meta.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
"pages": [
55
"index",
66
"docker",
7+
"custom-build",
78
"github-oauth"
89
]
910
}

content/docs/v3.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,8 @@ UI components are now registered and referenced as Global Vue components, which
143143

144144
This approach allows [main.py](https://github.com/ServiceStack/llms/blob/main/llms/main.py) to retain a **lean functional core in a single file** whilst still being fully extensible and lays the foundation for **rapid development of new features** - both from the core team and external 3rd party extensions - enabling the community to extend llms.py in new unanticipated ways.
145145

146+
For deployments requiring minimal footprint, the [Custom Build](/docs/deployment/custom-build) docs shows how to create a tailored distribution with only the specific extensions you need - perfect for CLI-only or lightweight API server deployments.
147+
146148
## Extensions System
147149

148150
To keep the core lightweight while enabling limitless enhancements, we've implemented a flexible **Extensions system** inspired by ComfyUI Custom Nodes. This allows adding new features, pages and toolbar icons, register new provider implementations, extend, replace, and customize the UI with your own custom features, just by adding new extension folders.

0 commit comments

Comments
 (0)