You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+26-6Lines changed: 26 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,19 +50,39 @@ GhostCoder is the first coding assistant that validates its own output.
50
50
51
51
---
52
52
53
-
## Hardware Compatibility & local LLMs
53
+
## Model Agnostic & GPU Auto-Scaling
54
54
55
-
GhostCoder runs entirely on your local machine. It is designed to work efficiently even on lightweight GPUs (e.g., **NVIDIA GTX 1650 4GB VRAM**):
56
-
1.**Qwen2.5-0.5B (1GB VRAM)**: Kept loaded in memory for fast situation classification.
57
-
2.**Qwen2.5-Coder-1.5B (2GB VRAM)**: Loaded on-demand for code generation, and automatically unloaded after 30 seconds of idle time.
58
-
3.**CPU Fallback**: Gracefully falls back to CPU execution if GPU memory headroom (default 500MB) is exceeded.
55
+
GhostCoder is **completely model-agnostic** and works with any Large Language Model supported by your local Ollama instance (including Llama 3, DeepSeek, Mistral, Qwen, Codellama, etc.).
56
+
57
+
By default, GhostCoder automatically detects your local GPU resources and dynamically routes requests to the largest model that fits comfortably in your VRAM budget (up to 85% utilization).
58
+
59
+
### GPU Tiers & Recommendations
60
+
61
+
| VRAM Capacity | Tier Profile | Classifier Model | Coder Model | Reasoner Model | Skeptic Model |
0 commit comments