Skip to content

Commit 8dca773

Browse files
hjk-makerhjk-maker
authored
Revamp README with comprehensive project details
Updated project title, overview, and added detailed sections on features, installation, usage, configuration, integration, security, known issues, roadmap, contributing, license, and support.
1 parent 1764fb1 commit 8dca773

1 file changed

Lines changed: 238 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 238 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,238 @@
1-
# clathsql-pro
2-
AI AGENT BASED DATABASE SOFTWARE
1+
# ⚡ ClathSQL Pro — Intelligent Data Routing Engine
2+
3+
**HJK-INC Enterprise Core | Virtual SQL Organisation**
4+
*Automate Your Storage Decisions with AI-Powered Intelligence*
5+
6+
---
7+
8+
![License](https://img.shields.io/badge/License-MIT-blue.svg)
9+
![Python](https://img.shields.io/badge/Python-3.8+-green.svg)
10+
![Status](https://img.shields.io/badge/Status-Production%20Ready-brightgreen)
11+
12+
---
13+
14+
## 📖 Overview
15+
16+
**ClathSQL Pro** is an autonomous data ingestion and routing platform designed for high-volume, multi-format data pipelines. By leveraging TinyLlama-based neural inference, it classifies incoming unstructured content into three execution paths:
17+
18+
| Route | Technology | Use Case |
19+
|-------|------------|----------|
20+
| **SQL** | SQLite (Relational) | Structured tabular data, queryable records |
21+
| **NOSQL** | JSON Document Store | Unstructured documents, nested schemas |
22+
| **PLOT** | Matplotlib Visualization | Time-series, numerical sequences, metrics |
23+
24+
Built by **hjk-inc**, this system embodies our philosophy: *"Build once, automate everywhere."* No manual classification needed—the brain decides, the vault stores, the GUI visualizes.
25+
26+
---
27+
28+
## 🎯 Key Features
29+
30+
### 🔹 Hybrid AI Routing Logic
31+
- **Neural Inference**: TinyLlama-1.1B Chat model for semantic understanding
32+
- **Regex Bypass**: Zero-latency pattern matching for obvious cases (JSON → NOSQL, SELECT → SQL)
33+
- **Warmup Sequence**: Pre-loaded model weights eliminate first-inference lag
34+
35+
### 🔹 Dual-Storage Vault
36+
- **SQLite Backend**: Indexed temporal database with auto-timestamps
37+
- **Append-Only JSON**: Lock-safe NoSQL document storage with PID concurrency control
38+
39+
### 🔹 Enterprise Dashboard
40+
- **CustomTkinter UI**: Dark-mode optimized for extended operations
41+
- **Real-time Monitoring**: CPU/RAM telemetry via `psutil`
42+
- **Embedded Visualization**: Matplotlib plots rendered directly in GUI
43+
- **Export Automation**: One-click CSV export of vault contents
44+
45+
### 🔹 Thread-Safe Architecture
46+
- All heavy operations run on background threads
47+
- Non-blocking UI during AI inference (5-15s models)
48+
- Graceful shutdown with resource cleanup
49+
50+
---
51+
52+
## 🚀 Installation & Setup
53+
54+
### Prerequisites
55+
```bash
56+
# Python 3.8+ required
57+
python --version
58+
59+
# Install dependencies
60+
pip install customtkinter torch transformers pandas matplotlib psutil
61+
```
62+
63+
### Quick Start
64+
```bash
65+
git clone https://github.com/virtualsql/clathsql-pro.git
66+
cd clathsql-pro
67+
python clathsql_pro.py
68+
```
69+
70+
### First Run Checklist
71+
1. ✅ Model downloads automatically (~1GB TinyLlama cache)
72+
2. ✅ Watch status indicator turn green ("Brain Online")
73+
3. ✅ Paste test data → Click "CLUTCH DATA"
74+
4. ✅ Verify vault files created (`hjk_clath_vault.db`, `hjk_clath_docs.json`)
75+
76+
---
77+
78+
---
79+
80+
## 🧪 Usage Examples
81+
82+
### 1. Routing to SQL
83+
```text
84+
Paste: "SELECT * FROM users WHERE id > 100"
85+
→ AI detects SQL keywords → Stored in relational table
86+
```
87+
88+
### 2. Routing to NoSQL
89+
```json
90+
Paste: {"user_id": 42, "action": "purchase", "amount": 99.99}
91+
→ Regex bypasses AI → Appended as JSON document
92+
```
93+
94+
### 3. Routing to Plot
95+
```text
96+
Paste:
97+
10.5
98+
20.3
99+
15.8
100+
25.1
101+
30.4
102+
→ AI identifies numerical sequence → Generated trend plot
103+
```
104+
105+
---
106+
107+
## 🛠️ Configuration
108+
109+
### Environment Variables (Optional)
110+
```bash
111+
# .env file at project root
112+
MODEL_ID=TinyLlama/TinyLlama-1.1B-Chat-v1.0 # Switch to Phi-3 or Llama-3
113+
DEVICE=0 # 0 for GPU, -1 for CPU
114+
MAX_TOKENS=5 # Response length limit
115+
```
116+
117+
### Customizing AI Model
118+
Edit `ClathSQL_Brain.__init__`:
119+
```python
120+
model_id = "mistralai/Mistral-7B-Instruct-v0.1" # Higher accuracy
121+
torch_dtype=torch.float16 # Reduce VRAM usage
122+
```
123+
124+
---
125+
126+
## 📊 Vault Statistics
127+
128+
Access real-time metrics via the sidebar panel:
129+
130+
| Metric | Description |
131+
|--------|-------------|
132+
| **SQL Entries** | Count of records in `clath_stream` table |
133+
| **NoSQL Docs** | Lines written to document store |
134+
| **CPU Power** | Real-time processor utilization |
135+
| **RAM Usage** | System memory consumption percentage |
136+
137+
Refresh manually or watch auto-updates every 30 seconds.
138+
139+
---
140+
141+
## 🤝 Integration Guide
142+
143+
### Connect to Maker's Newspaper Platform
144+
Expose ClathSQL routing logic via FastAPI for cross-service automation:
145+
146+
```python
147+
from fastapi import FastAPI
148+
app = FastAPI()
149+
150+
@app.post("/route-data")
151+
def route_data(data: str):
152+
decision = brain.clutch_logic(data)
153+
return {"route": decision}
154+
```
155+
156+
Then consume from your other HJK-INC microservices without duplicating infrastructure.
157+
158+
### Embed in Existing Applications
159+
Import core classes directly:
160+
```python
161+
from clathsql_pro import ClathSQL_Brain, ClathSQL_Vault
162+
163+
brain = ClathSQL_Brain(lambda x, y: None) # Disable UI callback
164+
vault = ClathSQL_Vault()
165+
decision = brain.clutch_logic(user_input)
166+
vault.store_sql(user_input) if decision == "SQL" else vault.store_nosql(user_input)
167+
```
168+
169+
---
170+
171+
## 🔒 Security Considerations
172+
173+
| Risk | Mitigation |
174+
|------|------------|
175+
| **Path Injection** | Input sanitized before file I/O |
176+
| **Concurrent Writes** | PID lock file prevents race conditions |
177+
| **Memory Leaks** | Explicit thread cleanup on app close |
178+
| **Model Exploits** | Temperature=0.1 limits adversarial outputs |
179+
180+
> ⚠️ **Warning**: For production use, add input size limits (max 1MB per payload) and authentication middleware.
181+
182+
---
183+
184+
## 🐛 Known Issues
185+
186+
| Issue | Status | Workaround |
187+
|-------|--------|------------|
188+
| First inference takes ~15s | Expected | Warmup runs automatically on launch |
189+
| Plot fails with <2 numbers | Intentional | Display warning instead of fake data |
190+
| Large JSON slows append | Ongoing | Batch writes planned v2.1 |
191+
192+
---
193+
194+
## 📅 Roadmap
195+
196+
- [x] V2.0 — Stable release with hybrid routing
197+
- [ ] V2.1 — Multi-threaded batch processing
198+
- [ ] V2.2 — REST API layer for remote routing
199+
- [ ] V2.3 — PostgreSQL/Redis adapters
200+
- [ ] V2.4 — Cloud deployment templates (Docker/Kubernetes)
201+
202+
---
203+
204+
## 👥 Contributing
205+
206+
Pull requests welcome! Please follow these guidelines:
207+
208+
1. Fork the repository
209+
2. Create feature branch (`git checkout -b feature/amazing-feature`)
210+
3. Commit with conventional messages (`feat:`, `fix:`, `docs:`)
211+
4. Push to branch and open Pull Request
212+
5. Ensure all tests pass before merge
213+
214+
---
215+
216+
## 📄 License
217+
218+
MIT License © 2024 **HJK-INC**. See [LICENSE](LICENSE) for full details.
219+
220+
---
221+
222+
## 📞 Support
223+
224+
For enterprise support, integration consulting, or bulk deployment:
225+
226+
📧 **Contact**: support@hjk-inc.com
227+
🌐 **Website**: https://hjk-inc.com
228+
💬 **Discord**: join.hjk-inc/community
229+
230+
---
231+
232+
<div align="center">
233+
234+
### Built with ❤️ by the HJK-INC Engineering Team
235+
236+
*"Lazy by design, powerful by necessity"*
237+
238+
</div>

0 commit comments

Comments
 (0)