1- # PUMA
1+ < div align = " center " >
22
3- A lightweight, high-performance inference engine for local AI. * Play for fun.*
3+ <picture >
4+ <source media =" (prefers-color-scheme: dark) " srcset =" docs/images/logo.png " >
5+ <source media =" (prefers-color-scheme: light) " srcset =" docs/images/logo.png " >
6+ <img alt =" PUMA Logo " src =" docs/images/puma-logo-light.svg " width =" 240 " >
7+ </picture >
48
5- ## Features
9+ ** A lightweight, high-performance inference engine for local AI **
610
7- - ** Model Management** - Download and manage AI models from model providers like Hugging Face
8- - ** System Detection** - Automatic GPU detection and system information reporting
9- - ** Local Caching** - Efficient model storage with custom cache directories
10- - ** Multiple Providers** - Support for Hugging Face with ModelScope coming soon
11+ [ ![ Stability: Active] ( https://img.shields.io/badge/stability-active-brightgreen.svg )] ( https://github.com/InftyAI/PUMA )
12+ [ ![ Latest Release] ( https://img.shields.io/github/v/release/InftyAI/PUMA )] ( https://github.com/InftyAI/PUMA/releases )
1113
12- ## Installation
14+ </div >
15+
16+ ## ✨ Features
17+
18+ 🔧 ** Model Management** - Download, cache, and organize AI models from Hugging Face
19+
20+ 🔍 ** Advanced Filtering** - Search models with regex patterns and SQL-style queries
1321
14- ### From Source
22+ 💻 ** System Detection** - Automatic GPU detection and resource reporting
23+
24+ ## Installation
1525
1626``` bash
1727make build
@@ -21,76 +31,108 @@ The binary will be available as `./puma`.
2131
2232## Quick Start
2333
24- ### 1. Download a Model
25-
2634``` bash
27- # From Hugging Face (default)
28- puma pull InftyAI/tiny-random-gpt2
29- ```
35+ # Download a model
36+ puma pull inftyai/tiny-random-gpt2
3037
31- ### 2. List Downloaded Models
32-
33- ``` bash
38+ # List all models
3439puma ls
35- ```
3640
37- ### 3. Check System Information
41+ # Inspect model details
42+ puma inspect inftyai/tiny-random-gpt2
3843
39- ``` bash
44+ # Check system info
4045puma info
41- ```
4246
43- Example output:
44- ```
45- System Information:
46- Operating System: Darwin
47- Architecture: arm64
48- CPU Cores: 14
49- Total Memory: 36.00 GiB
50- GPU: Apple M4 Max (Metal) - 32 GPU cores
51-
52- PUMA Information:
53- PUMA Version: 0.0.1
54- Cache Directory: ~/.puma/cache
55- Cache Size: 799.88 MiB
56- Models: 1
57- Running Models: 0
47+ # Remove a model
48+ puma rm inftyai/tiny-random-gpt2
5849```
5950
6051## Commands
6152
62- | Command | Status | Description | Example |
63- | ---------| --------| -------------| ---------|
64- | ` pull ` | ✅ | Download a model from a provider | ` puma pull InftyAI/tiny-random-gpt2 ` |
65- | ` ls ` | ✅ | List local models | ` puma ls ` |
66- | ` ps ` | 🚧 | List running models | ` puma ps ` |
67- | ` run ` | 🚧 | Create and run a model | ` puma run InftyAI/tiny-random-gpt2 ` |
68- | ` stop ` | 🚧 | Stop a running model | ` puma stop <model-id> ` |
69- | ` rm ` | ✅ | Remove a model | ` puma rm InftyAI/tiny-random-gpt2 ` |
70- | ` info ` | ✅ | Display system-wide information | ` puma info ` |
71- | ` inspect ` | ✅ | Return detailed information about a model or service | ` puma inspect InftyAI/tiny-random-gpt2 ` |
72- | ` version ` | ✅ | Show PUMA version | ` puma version ` |
73- | ` help ` | ✅ | Show help information | ` puma help ` |
53+ | Command | Status | Description |
54+ | ---------| --------| -------------|
55+ | ` pull <model> ` | ✅ | Download model from provider |
56+ | ` ls ` | ✅ | List models (supports regex, label filters) |
57+ | ` inspect <model> ` | ✅ | Show detailed model information |
58+ | ` rm <model> ` | ✅ | Remove model and cache |
59+ | ` info ` | ✅ | Display system information |
60+ | ` version ` | ✅ | Show PUMA version |
61+ | ` ps ` | 🚧 | List running models |
62+ | ` run ` | 🚧 | Start model inference |
63+ | ` stop ` | 🚧 | Stop running model |
7464
75- ## Configuration
65+ ## Advanced Usage
7666
77- PUMA stores models in ` ~/.puma/cache ` by default. This location is used for all downloaded models and metadata.
67+ ### Pattern Matching
7868
79- ## Supported Providers
69+ ``` bash
70+ # Substring match
71+ puma ls qwen
8072
81- - ** Hugging Face** - Full support with custom cache directories
73+ # Prefix match
74+ puma ls " ^inftyai/"
8275
83- ## Development
76+ # Alternation
77+ puma ls " llama-(2|3)"
78+ ```
8479
85- ### Build
80+ ### Label Filtering
8681
8782``` bash
88- make build
83+ # Single filter
84+ puma ls -l author=inftyai
85+
86+ # Multiple filters (AND condition)
87+ puma ls -l author=inftyai,license=mit
88+
89+ # Combine pattern + filter
90+ puma ls llama -l author=meta
8991```
9092
91- ### Test
93+ ** Available filters:** ` author ` , ` task ` , ` license ` , ` provider ` , ` model_series `
94+
95+ ### Inspect Output
9296
9397``` bash
98+ $ puma inspect inftyai/tiny-random-gpt2
99+
100+ name: inftyai/tiny-random-gpt2
101+ kind: model
102+ spec:
103+ author: inftyai
104+ task: text-generation
105+ license: MIT
106+ model_series: gpt2
107+ context_window: 2.05K
108+ safetensors:
109+ total: 7.00B
110+ parameters:
111+ f32: 7.00B
112+ artifact:
113+ provider: huggingface
114+ revision: abc123de
115+ size: 1.24 GB
116+ cache_path: ~ /.puma/cache/...
117+ status:
118+ created: 2 hours ago
119+ updated: 2 hours ago
120+ ```
121+
122+ ## Model Management
123+
124+ - ** Database:** ` ~/.puma/models.db ` (SQLite)
125+ - ** Cache:** ` ~/.puma/cache/ ` (model files)
126+
127+ Models are stored with lowercase names for case-insensitive matching.
128+
129+ ## Development
130+
131+ ``` bash
132+ # Build
133+ make build
134+
135+ # Run tests (67 unit + 22 integration)
94136make test
95137```
96138
@@ -99,19 +141,21 @@ make test
99141```
100142puma/
101143├── src/
102- │ ├── cli/ # Command-line interface
103- │ ├── downloader/ # Model download logic
104- │ ├── registry/ # Model registry management
105- │ ├── system/ # System detection (CPU, GPU, memory)
106- │ └── utils/ # Utility functions
107- ├── Cargo.toml # Rust dependencies
108- └── Makefile # Build commands
144+ │ ├── cli/ # Command implementations (ls, rm, inspect)
145+ │ ├── downloader/ # HuggingFace download logic
146+ │ ├── registry/ # Model registry & metadata
147+ │ ├── storage/ # SQLite storage backend
148+ │ ├── system/ # System info detection
149+ │ └── utils/ # Formatting & helpers
150+ ├── tests/ # Integration tests
151+ ├── Cargo.toml # Rust dependencies
152+ └── Makefile # Build commands
109153```
110154
111155## License
112156
113157Apache-2.0
114158
115- ## Contributing
159+ ## Star History
116160
117161[ ![ Star History Chart] ( https://api.star-history.com/svg?repos=inftyai/puma&type=Date )] ( https://www.star-history.com/#inftyai/puma&Date )
0 commit comments