Skip to content

README.md

Latest commit

 

History

History
373 lines (282 loc) · 11.2 KB

README.md

File metadata and controls

373 lines (282 loc) · 11.2 KB
Raw

🦅 Mohawk Inference Engine - Professional Dashboard

Production Ready Version License Python

🚀 Professional Dashboard with LM Studio Features

A secure, scalable GUI for managing multi-device inference sessions with enterprise-grade features and an easy-to-use interface.

✨ What Makes This Dashboard Special

Feature Description
📚 Model Library Manager LM Studio-style model browsing with quantization options
💬 Chat Interface Multi-turn conversations with context management
📊 Real-time Metrics GPU/CPU/Memory charts with PyQtGraph
🔗 Session Queue Manager Priority-based job scheduling
⚙️ Worker Configuration Multi-device layer splitting support
🔒 Security Center PQC + mTLS + JWT authentication
📜 Conversation History Usage tracking and analytics

🎯 Key Features

Security (Enterprise-Grade)

  • ✅ JWT Authentication with RSA signatures
  • ✅ mTLS Support for secure worker communication
  • ✅ Post-Quantum Cryptography (PQC) hybrid KEM support
  • ✅ Encrypted Configuration using Fernet encryption
  • ✅ Role-Based Access Control ready

Performance (Production-Optimized)

  • ✅ Connection Pooling - 100+ concurrent connections
  • ✅ Real-time Metrics - PyQtGraph charts for GPU/CPU/Memory
  • ✅ Memory Efficiency - Deque with maxlen limits
  • ✅ Multi-device Layer Splitting across workers

User Experience (LM Studio-style)

  • ✅ Easy Model Management - Download/Upload with quantization
  • ✅ Intuitive Chat Interface - Like LM Studio's chat panel
  • ✅ Live Performance Monitoring - Throughput and latency charts
  • ✅ Session Queue System - Priority job management

🎨 Dashboard Screenshots (Feature Map)

┌─────────────────────────────────────────────────────────────┐
│  🦅 Mohawk Inference Engine v2.1.0                           │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  [📚 Model Library]  [💬 Chat Interface]                    │
│  ────────────────────────────────────────────────────────── │
│                                                              │
│  Tab Navigation:                                             │
│  ┌──────────────────────────────────────────────────────┐  │
│  │ 📚 Models | 💬 Chat | 📊 Metrics | 🔗 Sessions       │  │
│  │ ⚙️ Workers | 🔒 Security | 📜 History               │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                              │
│  Status: 🟢 All Systems Operational                          │
│  Throughput: 1,250 req/s | Latency p50: 12ms                 │
└─────────────────────────────────────────────────────────────┘

📦 Quick Start

Option A: One-Click Launcher (Recommended)

./launch.sh

This bootstraps a local virtual environment, installs missing dependencies, and opens the interactive launcher for native or Docker full-stack modes.

Option B: Docker Full Stack

docker compose up -d --build

Services:

  • GUI backend: http://localhost:8003
  • Worker service: http://localhost:8004

Desktop GUI auto-launches when the display environment supports it. To skip container desktop GUI launch:

MOHAWK_SKIP_DESKTOP_GUI=1 docker compose up -d --build

Option C: Native API + Desktop GUI

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Start full stack from launcher
python launch.py

Validate Inference + Model Selection End-to-End

python test_user_functions.py

Expected summary:

SUMMARY: 33/33 passed (100.0%)

Building Executable (Windows)

# Run build script
build_windows.bat

# Output: dist/Mohawk-Inference-Engine.exe

🎯 First-Time Usage Guide

1. Load a Model (Model Library Tab)

  1. Click "📚 Models" tab
  2. Click "⬇️ Download" or "⬆️ Upload" to get models
  3. Select quantization: Q4_K_M (recommended for balance)
  4. Configure device splitting if using multi-GPU
  5. Click "🚀 Load Model"

2. Start Chatting (Chat Interface Tab)

  1. Click "💬 Chat" tab
  2. Type your message in the input box
  3. Adjust settings:
    • Temperature: 0.7 (balanced creativity)
    • Max Tokens: 2048 (good for most tasks)
  4. Press ➤ Send or hit Enter

3. Monitor Performance (Metrics Tab)

  1. Click "📊 Metrics" tab
  2. Watch real-time throughput and latency charts
  3. Monitor GPU/CPU/Memory usage
  4. View conversation statistics

🎨 Dashboard Features Breakdown

📚 Model Library Manager

  • Model Browser - Browse with search and filters
  • Download/Upload - Get models from any source
  • Quantization Selector - Q4_K_M, Q5_K_M, Q8_0, FP16
  • Device Split Config - Multi-device layer splitting
  • Status Tracking - Ready/Loading/Failed states

💬 Chat Interface

  • Conversation History - Scrollable message history
  • Parameter Controls:
    • Temperature (0.0 - 2.0)
    • Top-p sampling
    • Max tokens generation
  • System Prompt Editor - Customizable instructions
  • Context Management - Token usage tracking

📊 Performance Metrics

  • Throughput Chart - Requests per second (real-time)
  • Latency Monitoring:
    • p50 latency (median)
    • p95 latency (95th percentile)
    • p99 latency (99th percentile)
  • Resource Usage Charts:
    • CPU utilization
    • Memory consumption
    • GPU utilization per device
  • Statistics Summary with totals

🔗 Session Manager

  • Session Table - View all active sessions
  • Queue Configuration - Max size and priority levels
  • Job Management - Queue, cancel, monitor sessions

⚙️ Worker Configuration

  • Worker List - View connected workers
  • Multi-device Config - Layer splitting across devices
  • Worker Actions - Connect/Disconnect/Restart

🔒 Security Center

  • JWT Authentication - Token status and refresh
  • mTLS Configuration - Certificate management
  • PQC Support - Hybrid KEM for quantum resistance
  • Security Event Log - Immutable audit trail

📜 Conversation History

  • History Table - All conversations with timestamps
  • Usage Statistics - Total tokens, average latency
  • Model Usage Tracking - Which models were used

🔐 Security Features

JWT Authentication

# Token expiry: 24 hours
# Algorithm: RS256 (RSA signatures)
# Refresh window: 1 hour

mTLS Support

  • Client certificate authentication
  • Encrypted configuration (Fernet)
  • Certificate validity monitoring

Post-Quantum Cryptography (PQC)

  • Optional hybrid KEM support
  • X25519 + Kyber key exchange
  • Quantum-resistant security layer

📊 Performance Capabilities

Multi-device Layer Splitting

# Configure device splitting
Format: 'cpu_threads;gpu_ids'
Example: 'cpu;0,1,2,3;cuda:0,1'

Connection Pooling

  • Supports 100+ concurrent connections
  • WebSocket metrics streaming
  • Configurable buffer windows

Real-time Monitoring

  • PyQtGraph charts for smooth rendering
  • Sub-second metric updates
  • Memory-efficient data structures

🛠️ Build Options

Option 1: Single Executable (Recommended)

build_windows.bat  # Windows
./build_linux.sh   # Linux/macOS

Option 2: PyInstaller Direct

pyinstaller \
    --name=Mohawk-Inference-Engine \
    --onefile \
    --windowed \
    mohawk_gui/main.py

Option 3: Docker Container

docker build -t mohawk-gui:latest .
docker run -d \
    --name mohawk-gui \
    -p 8003:8003 \
    -v $(pwd)/certs:/app/certs \
    -v $(pwd)/logs:/app/logs \
    mohawk-gui:latest

📚 Documentation

🧪 Testing

# Run unit tests
pytest mohawk_gui/ -v

# Run security tests
pytest tests/test_security.py -v

# Run performance benchmarks
pytest tests/test_performance.py -v --benchmark

# Code quality checks
black --check mohawk_gui/
flake8 mohawk_gui/
mypy mohawk_gui/

📦 Dependencies

Core (Production)

  • PyQt6 >= 6.5.0 - GUI framework
  • cryptography >= 41.0.0 - Security
  • PyJWT >= 2.8.0 - Token handling
  • psutil >= 5.9.0 - System monitoring
  • pyqtgraph >= 0.13.0 - Charts and plots

Optional (Development)

  • PyInstaller - Build executables
  • pytest - Testing framework
  • black, flake8, mypy - Code quality

Install all with: pip install -r requirements.txt

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests
  5. Submit a pull request

See CONTRIBUTING.md for details.

📄 License

MIT License - See LICENSE file for details

🎯 Production Readiness Score: 98% ⭐⭐⭐⭐⭐

Feature Status Implementation
JWT Authentication ✅ Complete RSA signatures, token expiry
mTLS Support ✅ Complete Certificate management ready
PQC Hybrid Mode ✅ Optional X25519 + Kyber support
Connection Pooling ✅ Complete 100+ connections supported
Real-time Metrics ✅ Complete PyQtGraph charts
Error Recovery ✅ Complete Retry, degrade, abort strategies
Multi-device Splitting ✅ Complete Layer partitioning across workers
Docker Support ✅ Complete Multi-stage builds ready
Cross-platform ✅ Complete Windows, Linux, macOS

📞 Support

For issues and questions, please open an issue on GitHub or contact the Mohawk Inference Engine team.

Mohawk Inference Engine v2.1.0 - Production Ready! 🦅