Skip to content

Commit 0de9cfe

Browse files
committed
release: v0.4.9 - revert to Apache-2.0 license and add deployment guides
Changes: - Revert license from AGPL-3.0 to Apache-2.0 in pyproject.toml - Bump version from 0.4.8 to 0.4.9 - Update classifier to Apache Software License - Update all v0.4.8 references to v0.4.9 in README - Add Gunicorn, uWSGI, and systemd deployment examples - Add Docker Compose configuration - Add System Requirements section with OS compatibility - Add comprehensive Troubleshooting guide: * Database locked errors with standard mode * Library not found errors * Gunicorn worker issues and --preload flag * FastAPI/Uvicorn multi-worker setup * Performance monitoring and optimization - Document file descriptor limits and memory requirements
1 parent 5f05342 commit 0de9cfe

File tree

2 files changed

+245
-5
lines changed

2 files changed

+245
-5
lines changed

README.md

Lines changed: 242 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Choose the deployment mode that fits your needs.
3838
- ✅ Edge deployments without network
3939
- ✅ No server setup required
4040

41-
**Embedded Concurrent Mode (NEW in v0.4.8):**
41+
**Embedded Concurrent Mode (NEW in v0.4.9):**
4242
- ✅ Web applications (Flask, FastAPI, Django)
4343
- ✅ Multi-process workers (Gunicorn, uWSGI)
4444
- ✅ Hot reloading development servers
@@ -54,7 +54,7 @@ Choose the deployment mode that fits your needs.
5454

5555
---
5656

57-
## Concurrent Embedded Mode (v0.4.8+)
57+
## Concurrent Embedded Mode (v0.4.9+)
5858

5959
For web applications that need multiple processes to access the same database:
6060

@@ -105,6 +105,246 @@ def update_user(user_id):
105105
| Write | ~5ms (fsync) | ~60µs (amortized) |
106106
| Max concurrent readers | 1 | 1024 |
107107

108+
### Gunicorn Deployment
109+
110+
```bash
111+
# Install Gunicorn
112+
pip install gunicorn
113+
114+
# Run with 4 worker processes (all can access same DB concurrently)
115+
gunicorn -w 4 -b 0.0.0.0:8000 app:app
116+
117+
# Workers automatically share the database in concurrent mode
118+
```
119+
120+
### uWSGI Deployment
121+
122+
```bash
123+
# Install uWSGI
124+
pip install uwsgi
125+
126+
# Run with 4 processes
127+
uwsgi --http :8000 --wsgi-file app.py --callable app --processes 4
128+
```
129+
130+
### Systemd Service Example
131+
132+
```ini
133+
# /etc/systemd/system/myapp.service
134+
[Unit]
135+
Description=MyApp with SochDB
136+
After=network.target
137+
138+
[Service]
139+
Type=notify
140+
User=appuser
141+
WorkingDirectory=/opt/myapp
142+
ExecStart=/opt/myapp/venv/bin/gunicorn -w 4 -b 0.0.0.0:8000 app:app
143+
Restart=always
144+
145+
[Install]
146+
WantedBy=multi-user.target
147+
```
148+
149+
```bash
150+
# Enable and start service
151+
sudo systemctl enable myapp
152+
sudo systemctl start myapp
153+
sudo systemctl status myapp
154+
```
155+
156+
### Docker Compose Example
157+
158+
```yaml
159+
version: '3.8'
160+
services:
161+
app:
162+
build: .
163+
environment:
164+
- WORKERS=4
165+
volumes:
166+
- ./data:/app/data # Shared database volume
167+
ports:
168+
- "8000:8000"
169+
command: gunicorn -w 4 -b 0.0.0.0:8000 app:app
170+
```
171+
172+
---
173+
174+
## System Requirements
175+
176+
### For Concurrent Mode
177+
178+
- **SochDB Core**: v0.4.4 or later
179+
- **Python**: 3.9+ (3.11+ recommended)
180+
- **Native Library**: `libsochdb_storage.{dylib,so}` v0.4.4+
181+
- **FFI**: ctypes (built-in to Python)
182+
183+
**Operating Systems:**
184+
- ✅ Linux (Ubuntu 20.04+, RHEL 8+)
185+
- ✅ macOS (10.15+, both Intel and Apple Silicon)
186+
- ⚠️ Windows (requires native builds)
187+
188+
**File Descriptors:**
189+
- Default limit: 1024 (sufficient for most workloads)
190+
- For high concurrency with Gunicorn: `ulimit -n 4096`
191+
192+
**Memory:**
193+
- Standard mode: ~50MB base + data
194+
- Concurrent mode: +4KB per concurrent reader slot (1024 slots = ~4MB overhead)
195+
- Gunicorn: Each worker has independent memory
196+
197+
---
198+
199+
## Troubleshooting
200+
201+
### "Database is locked" Error (Standard Mode)
202+
203+
```
204+
OperationalError: database is locked
205+
```
206+
207+
**Solution**: Use concurrent mode for multi-process access:
208+
209+
```python
210+
# ❌ Standard mode - Gunicorn workers will conflict
211+
db = Database.open("./data.db")
212+
213+
# ✅ Concurrent mode - all workers can access
214+
db = Database.open_concurrent("./data.db")
215+
```
216+
217+
### Library Not Found Error
218+
219+
```
220+
OSError: libsochdb_storage.dylib not found
221+
```
222+
223+
**macOS**:
224+
```bash
225+
# Build and install library
226+
cd /path/to/sochdb
227+
cargo build --release
228+
sudo cp target/release/libsochdb_storage.dylib /usr/local/lib/
229+
```
230+
231+
**Linux**:
232+
```bash
233+
cd /path/to/sochdb
234+
cargo build --release
235+
sudo cp target/release/libsochdb_storage.so /usr/local/lib/
236+
sudo ldconfig
237+
```
238+
239+
**Development Mode** (no install):
240+
```bash
241+
export DYLD_LIBRARY_PATH=/path/to/sochdb/target/release # macOS
242+
export LD_LIBRARY_PATH=/path/to/sochdb/target/release # Linux
243+
```
244+
245+
### Gunicorn Worker Issues
246+
247+
**Symptom**: Workers crash with "database locked"
248+
249+
**Solution 1** - Ensure concurrent mode is used:
250+
```python
251+
# app.py
252+
import os
253+
from sochdb import Database
254+
255+
# Use environment variable to control mode
256+
USE_CONCURRENT = os.getenv('USE_CONCURRENT_MODE', 'true').lower() == 'true'
257+
258+
if USE_CONCURRENT:
259+
db = Database.open_concurrent('./db')
260+
else:
261+
db = Database.open('./db')
262+
263+
print(f"Concurrent mode: {db.is_concurrent}") # Should be True
264+
```
265+
266+
```bash
267+
# Start with concurrent mode enabled
268+
USE_CONCURRENT_MODE=true gunicorn -w 4 -b 0.0.0.0:8000 app:app
269+
```
270+
271+
**Solution 2** - Check preload settings:
272+
```bash
273+
# Don't use --preload with concurrent mode
274+
# ❌ This will cause issues:
275+
gunicorn --preload -w 4 app:app
276+
277+
# ✅ Let each worker open the database:
278+
gunicorn -w 4 app:app
279+
```
280+
281+
### FastAPI with Uvicorn Workers
282+
283+
**Symptom**: `RuntimeError: Concurrent mode requires multi-process access`
284+
285+
**Solution**: Use Uvicorn workers correctly:
286+
```bash
287+
# ❌ Single worker (async) - doesn't need concurrent mode
288+
uvicorn app:app --workers 1
289+
290+
# ✅ Multiple workers - needs concurrent mode
291+
uvicorn app:app --workers 4
292+
```
293+
294+
```python
295+
# main.py
296+
from fastapi import FastAPI
297+
from sochdb import Database
298+
import multiprocessing
299+
300+
app = FastAPI()
301+
302+
# Detect if running in multi-worker mode
303+
workers = multiprocessing.cpu_count()
304+
if workers > 1:
305+
db = Database.open_concurrent("./db")
306+
else:
307+
db = Database.open("./db")
308+
```
309+
310+
### Performance Issues
311+
312+
**Symptom**: Concurrent reads slower than expected
313+
314+
**Check 1** - Verify concurrent mode is active:
315+
```python
316+
import logging
317+
logging.basicConfig(level=logging.INFO)
318+
319+
db = Database.open_concurrent("./db")
320+
if not db.is_concurrent:
321+
logging.error("Database is not in concurrent mode!")
322+
raise RuntimeError("Expected concurrent mode")
323+
logging.info(f"Concurrent mode active: {db.is_concurrent}")
324+
```
325+
326+
**Check 2** - Monitor worker processes:
327+
```bash
328+
# Watch Gunicorn workers
329+
watch -n 1 'ps aux | grep gunicorn'
330+
331+
# Monitor file descriptors
332+
lsof | grep libsochdb_storage
333+
```
334+
335+
**Check 3** - Batch writes:
336+
```python
337+
# ❌ Slow - individual writes with fsync
338+
for item in items:
339+
db.put(key, value)
340+
341+
# ✅ Fast - batch in transaction
342+
tx = db.begin_txn()
343+
for item in items:
344+
tx.put(key, value)
345+
tx.commit() # Single fsync for entire batch
346+
```
347+
108348
---
109349

110350
## Installation

pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "sochdb"
7-
version = "0.4.8"
7+
version = "0.4.9"
88
description = "SochDB is an AI-native database with token-optimized output, O(|path|) lookups, built-in vector search, and durable transactions."
99
readme = "README.md"
10-
license = {text = "AGPL-3.0-or-later"}
10+
license = {text = "Apache-2.0"}
1111
authors = [
1212
{name = "Sushanth Reddy Vanagala", email = "sushanth@sochdb.dev"}
1313
]
@@ -18,7 +18,7 @@ keywords = ["database", "llm", "ai", "vector-search", "embedded", "key-value", "
1818
classifiers = [
1919
"Development Status :: 4 - Beta",
2020
"Intended Audience :: Developers",
21-
"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
21+
"License :: OSI Approved :: Apache Software License",
2222
"Programming Language :: Python :: 3",
2323
"Programming Language :: Python :: 3.9",
2424
"Programming Language :: Python :: 3.10",

0 commit comments

Comments
 (0)