Skip to content

Commit c2c3ec6

Browse files
committed
Create README.md
1 parent a72870f commit c2c3ec6

1 file changed

Lines changed: 88 additions & 0 deletions

File tree

custom-build/README.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Custom llms .py Build
2+
3+
## Lean Core + Extensibility
4+
5+
To reduce bloat, **llms .py** was intentionally built around a lean core with support for extensibility. Most major features are encapsulated in **extension folders** that can be layered on to create a custom build with only the features you need.
6+
7+
### Benefits of This Approach
8+
9+
- **Minimal Footprint** - Deploy only what you need, reducing dependencies and attack surface
10+
- **Faster Startup** - Fewer modules to load means quicker initialization times
11+
- **Easier Maintenance** - Isolated extensions are simpler to update, debug, and replace
12+
- **Flexible Deployment** - Create purpose-built distributions for different use cases (e.g., CLI-only, API server, full UI)
13+
- **Cleaner Codebase** - Separation of concerns keeps the core focused and extensions self-contained
14+
15+
## Creating a Custom Build
16+
17+
This directory demonstrates how to create a custom build of the llms .py CLI/Server with a curated set of extensions. This allows you to deploy a lightweight version of the application with only the specific functionality you need.
18+
19+
## Custom Builder
20+
21+
Use `./build.sh` inside the [custom-build](https://github.com/ServiceStack/llms/tree/main/custom-build) directory to create a custom build of llms .py with only the extensions you want, where running `./build.sh` without any arguments:
22+
23+
```bash
24+
cd custom-build
25+
26+
./build.sh
27+
```
28+
29+
Generates a minimal custom build inside a new `llms` directory containing just the [providers extension](https://llmspy.org/docs/extensions/built-in#providers-extension) and necessary configuration files:
30+
31+
```
32+
llms/
33+
├── .llms/ # .llms/ home for this build
34+
├── extensions/ # Extensions directory
35+
│ └── providers/ # Providers extension
36+
├── llms.json # llms.json config
37+
├── providers.json # models.dev providers.json config
38+
├── main.py # llms.py implementation
39+
├── llms.sh # llms.sh startup script
40+
```
41+
42+
This minimal build doesn't contain any UI or tools, but does contain the [providers extension](https://llmspy.org/docs/extensions/built-in#providers-extension) with support for all 24 supported providers and their 530+ models in both [CLI mode](https://llmspy.org/docs/features/cli) and its Open AI Compatible `/v1/chat/completions` endpoint.
43+
44+
### Startup Script
45+
46+
Once built you would use the `llms.sh` startup script for all commands instead of the `llms` command, e.g:
47+
48+
```bash
49+
./llms.sh "What's the Capital of France?"
50+
```
51+
52+
### Minimal UI Build
53+
54+
Set the `UI` environment variable to `1` for the minimal UI build which includes the [app](https://llmspy.org/docs/extensions/built-in#app-extension) and [tools](https://llmspy.org/docs/extensions/tools) extensions.
55+
56+
```bash
57+
UI=1 ./build.sh
58+
```
59+
60+
### Custom Extensions
61+
62+
For a custom build set the `EXTENSIONS` environment variable to a comma separated list of extensions to include in the build. E.g. this includes the same as as the Minimal UI build:
63+
64+
```bash
65+
EXTENSIONS="providers,app,tools" ./build.sh
66+
```
67+
68+
> Including any extension that requires a UI will automatically include it in the build.
69+
70+
### Add external extensions
71+
72+
With [full CLI support](https://llmspy.org/docs/features/cli), your custom build can use the [extension management commands](https://llmspy.org/docs/extensions) to manage installed extensions, e.g., you can add external extensions to your custom build by using the `llms.sh --add` command.
73+
74+
Preview available extensions:
75+
76+
```bash
77+
./llms.sh --add
78+
```
79+
80+
Add a specific extension:
81+
82+
```bash
83+
./llms.sh --add xmas
84+
```
85+
86+
## Built-in Extensions
87+
88+
You can copy any of these built-in extensions from the source [llms/extensions](https://github.com/ServiceStack/llms/tree/main/llms/extensions) directory - see [Built-in Extensions](http://llmspy.org/docs/extensions/built-in) for the list of built-in extensions you might want to include in your custom build.

0 commit comments

Comments
 (0)