Skip to content

Commit 3fad261

Browse files
solderzzcsolderzzc
committed
feat: YOLO 2026 detection skill with zero-assumption installer
- Register yolo-detection-2026 in skills.json with FPS presets and model sizes - Rewrite SKILL.md: model_size selection (nano/small/medium/large), FPS presets - Rewrite detect.py: model_size→ultralytics mapping, frame_id echo, AEGIS_SKILL_PARAMS - Create deploy.sh: zero-assumption bootstrapper (Python/conda/pyenv, CUDA/ROCm/MPS/CPU) - Per-backend requirements files (cuda/rocm/mps/cpu) - Create config.yaml with default parameters - Create docs/detection-protocol.md: full JSONL protocol spec, bbox format, backpressure
1 parent 8884320 commit 3fad261

10 files changed

Lines changed: 454 additions & 82 deletions

File tree

‎docs/detection-protocol.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Detection Skill Protocol
2+
3+
Communication protocol for DeepCamera detection skills integrated with SharpAI Aegis.
4+
5+
## Transport
6+
7+
- **stdin** (Aegis → Skill): frame events and commands
8+
- **stdout** (Skill → Aegis): detection results, ready/error events
9+
- **stderr**: logging only — ignored by Aegis data parser
10+
11+
Format: **JSON Lines** (one JSON object per line, newline-delimited).
12+
13+
## Events
14+
15+
### Ready (Skill → Aegis)
16+
17+
Emitted after model loads successfully. `fps` reflects the skill's configured processing rate. `available_sizes` lists the model variants the skill supports.
18+
19+
```jsonl
20+
{"event": "ready", "model": "yolo2026n", "device": "mps", "classes": 80, "fps": 5, "available_sizes": ["nano", "small", "medium", "large"]}
21+
```
22+
23+
### Frame (Aegis → Skill)
24+
25+
Instruction to analyze a specific frame. `frame_id` is an incrementing integer used to correlate request/response.
26+
27+
```jsonl
28+
{"event": "frame", "frame_id": 42, "camera_id": "front_door", "timestamp": "2026-03-01T14:30:00Z", "frame_path": "/tmp/aegis_detection/frame_front_door.jpg", "width": 1920, "height": 1080}
29+
```
30+
31+
### Detections (Skill → Aegis)
32+
33+
Results of frame analysis. Must echo the same `frame_id` received in the frame event.
34+
35+
```jsonl
36+
{"event": "detections", "frame_id": 42, "camera_id": "front_door", "timestamp": "2026-03-01T14:30:00Z", "objects": [
37+
{"class": "person", "confidence": 0.92, "bbox": [100, 50, 300, 400]},
38+
{"class": "car", "confidence": 0.87, "bbox": [500, 200, 900, 500]}
39+
]}
40+
```
41+
42+
### Error (Skill → Aegis)
43+
44+
Indicates a processing error. `retriable: true` means Aegis can send the next frame.
45+
46+
```jsonl
47+
{"event": "error", "frame_id": 42, "message": "Inference error: ...", "retriable": true}
48+
```
49+
50+
### Stop (Aegis → Skill)
51+
52+
Graceful shutdown command.
53+
54+
```jsonl
55+
{"command": "stop"}
56+
```
57+
58+
## Data Formats
59+
60+
### Bounding Boxes
61+
62+
**Format**: `[x_min, y_min, x_max, y_max]` — pixel coordinates (xyxy).
63+
64+
| Field | Type | Description |
65+
|-------|------|-------------|
66+
| `x_min` | int | Left edge (pixels) |
67+
| `y_min` | int | Top edge (pixels) |
68+
| `x_max` | int | Right edge (pixels) |
69+
| `y_max` | int | Bottom edge (pixels) |
70+
71+
Coordinates are in the original image space (not normalized).
72+
73+
### Timestamps
74+
75+
ISO 8601 format: `2026-03-01T14:30:00Z`
76+
77+
### Frame Transfer
78+
79+
Frames are written to `/tmp/aegis_detection/frame_{camera_id}.jpg` as JPEG files with recycled per-camera filenames (overwritten each cycle). The `frame_path` in the frame event is the absolute path to the JPEG file.
80+
81+
## FPS Presets
82+
83+
| Preset | FPS | Use Case |
84+
|--------|-----|----------|
85+
| Ultra Low | 0.2 | Battery saver |
86+
| Low | 0.5 | Passive surveillance |
87+
| Normal | 1 | Standard monitoring |
88+
| Active | 3 | Active area monitoring |
89+
| High | 5 | Security-critical zones |
90+
| Real-time | 15 | Live tracking |
91+
92+
## Backpressure
93+
94+
The protocol is **request-response**: Aegis sends one frame, waits for the detection result, then sends the next. This provides natural backpressure — if the skill is slow, Aegis automatically drops frames (always uses the latest available frame).

‎skills.json‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,54 @@
4848
"ui_unlocks": [
4949
"benchmark_report"
5050
]
51+
},
52+
{
53+
"id": "yolo-detection-2026",
54+
"name": "YOLO 2026 Object Detection",
55+
"description": "State-of-the-art real-time object detection — 80+ COCO classes, bounding box overlays, multi-size model selection.",
56+
"version": "1.0.0",
57+
"category": "detection",
58+
"path": "skills/detection/yolo-detection-2026",
59+
"tags": [
60+
"detection",
61+
"yolo",
62+
"object-detection",
63+
"real-time",
64+
"coco"
65+
],
66+
"platforms": [
67+
"linux-x64",
68+
"linux-arm64",
69+
"darwin-arm64",
70+
"darwin-x64",
71+
"win-x64"
72+
],
73+
"requirements": {
74+
"python": ">=3.9",
75+
"ram_gb": 2
76+
},
77+
"capabilities": [
78+
"live_detection",
79+
"bbox_overlay"
80+
],
81+
"ui_unlocks": [
82+
"detection_overlay",
83+
"detection_results"
84+
],
85+
"fps_presets": [
86+
0.2,
87+
0.5,
88+
1,
89+
3,
90+
5,
91+
15
92+
],
93+
"model_sizes": [
94+
"nano",
95+
"small",
96+
"medium",
97+
"large"
98+
]
5199
}
52100
]
53101
}

‎skills/detection/yolo-detection-2026/SKILL.md‎

Lines changed: 47 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,16 @@
11
---
22
name: yolo-detection-2026
3-
description: "State-of-the-art real-time object detection using YOLO"
3+
description: "YOLO 2026 — state-of-the-art real-time object detection"
44
version: 1.0.0
55
icon: assets/icon.png
66

77
parameters:
8-
- name: model
9-
label: "Model"
8+
- name: model_size
9+
label: "Model Size"
1010
type: select
11-
options: ["yolov11n", "yolov11s", "yolov11m", "yolov10n", "yolov10s", "yolov8n"]
12-
default: "yolov11n"
11+
options: ["nano", "small", "medium", "large"]
12+
default: "nano"
13+
description: "Larger models are more accurate but slower"
1314
group: Model
1415

1516
- name: confidence
@@ -29,18 +30,18 @@ parameters:
2930

3031
- name: fps
3132
label: "Processing FPS"
32-
type: number
33-
min: 1
34-
max: 30
33+
type: select
34+
options: [0.2, 0.5, 1, 3, 5, 15]
3535
default: 5
36+
description: "Frames per second — higher = more CPU/GPU usage"
3637
group: Performance
3738

3839
- name: device
3940
label: "Inference Device"
4041
type: select
41-
options: ["auto", "cpu", "cuda", "mps"]
42+
options: ["auto", "cpu", "cuda", "mps", "rocm"]
4243
default: "auto"
43-
description: "auto = GPU if available, else CPU"
44+
description: "auto = best available GPU, else CPU"
4445
group: Performance
4546

4647
capabilities:
@@ -49,78 +50,59 @@ capabilities:
4950
description: "Real-time object detection on live camera frames"
5051
---
5152

52-
# YOLO Object Detection (2026)
53-
54-
Real-time object detection using state-of-the-art YOLO models. Detects 80+ COCO object classes including people, vehicles, animals, and everyday objects. Outputs bounding boxes with labels and confidence scores that SharpAI Aegis renders as overlays on the live camera feed.
55-
56-
## What You Get
57-
58-
When installed in SharpAI Aegis, this skill unlocks:
59-
- **Live detection overlays** on camera feeds — bounding boxes around detected objects
60-
- **Smart alert triggers** — configure alerts when specific objects are detected
61-
- **Detection history** — searchable log of all detections
62-
63-
## Models
64-
65-
| Model | Size | Speed (FPS) | Accuracy (mAP) | Best For |
66-
|-------|------|-------------|-----------------|----------|
67-
| YOLOv11n | 6 MB | 30+ | 39.5 | Real-time on CPU |
68-
| YOLOv11s | 22 MB | 20+ | 47.0 | Balanced |
69-
| YOLOv11m | 68 MB | 12+ | 51.5 | High accuracy |
70-
| YOLOv10n | 7 MB | 28+ | 38.5 | Ultra-fast |
71-
| YOLOv10s | 24 MB | 18+ | 46.3 | Balanced (v10) |
72-
| YOLOv8n | 6 MB | 30+ | 37.3 | Legacy compatible |
53+
# YOLO 2026 Object Detection
7354

74-
## Setup
55+
Real-time object detection using the latest YOLO 2026 models. Detects 80+ COCO object classes including people, vehicles, animals, and everyday objects. Outputs bounding boxes with labels and confidence scores.
7556

76-
1. Create a Python virtual environment:
77-
```bash
78-
python3 -m venv .venv && source .venv/bin/activate
79-
```
57+
## Model Sizes
8058

81-
2. Install dependencies:
82-
```bash
83-
pip install -r requirements.txt
84-
```
85-
86-
3. Download model weights (automatic on first run, or manually):
87-
```bash
88-
python scripts/download_models.py --model yolov11n
89-
```
59+
| Size | Speed | Accuracy | Best For |
60+
|------|-------|----------|----------|
61+
| nano | Fastest | Good | Real-time on CPU, edge devices |
62+
| small | Fast | Better | Balanced speed/accuracy |
63+
| medium | Moderate | High | Accuracy-focused deployments |
64+
| large | Slower | Highest | Maximum detection quality |
9065

9166
## Protocol
9267

93-
This skill communicates with SharpAI Aegis via **JSON lines** over stdin/stdout.
94-
95-
### Aegis → Skill (stdin): frames to process
68+
Communicates via **JSON lines** over stdin/stdout.
9669

70+
### Aegis → Skill (stdin)
9771
```jsonl
98-
{"event": "frame", "camera_id": "front_door", "timestamp": "2026-03-01T14:30:00Z", "frame_path": "/tmp/frame_001.jpg", "width": 1920, "height": 1080}
72+
{"event": "frame", "frame_id": 42, "camera_id": "front_door", "timestamp": "...", "frame_path": "/tmp/aegis_detection/frame_front_door.jpg", "width": 1920, "height": 1080}
9973
```
10074

101-
### Skill → Aegis (stdout): detection results
102-
75+
### Skill → Aegis (stdout)
10376
```jsonl
104-
{"event": "ready", "model": "yolov11n", "device": "mps", "classes": 80}
105-
{"event": "detections", "camera_id": "front_door", "timestamp": "2026-03-01T14:30:00Z", "objects": [
106-
{"class": "person", "confidence": 0.92, "bbox": [100, 50, 300, 400]},
107-
{"class": "car", "confidence": 0.87, "bbox": [500, 200, 900, 500]}
77+
{"event": "ready", "model": "yolo2026n", "device": "mps", "classes": 80, "fps": 5}
78+
{"event": "detections", "frame_id": 42, "camera_id": "front_door", "timestamp": "...", "objects": [
79+
{"class": "person", "confidence": 0.92, "bbox": [100, 50, 300, 400]}
10880
]}
81+
{"event": "error", "message": "...", "retriable": true}
10982
```
11083

11184
### Bounding Box Format
85+
`[x_min, y_min, x_max, y_max]` — pixel coordinates (xyxy).
11286

113-
`[x_min, y_min, x_max, y_max]` in pixel coordinates.
87+
### Stop Command
88+
```jsonl
89+
{"command": "stop"}
90+
```
91+
92+
## Hardware Support
11493

115-
## Hardware Requirements
94+
| Platform | Backend | Performance |
95+
|----------|---------|-------------|
96+
| Apple Silicon (M1+) | MPS | 20-30 FPS |
97+
| NVIDIA GPU | CUDA | 25-60 FPS |
98+
| AMD GPU | ROCm | 15-40 FPS |
99+
| CPU (modern x86) | CPU | 5-15 FPS |
100+
| Raspberry Pi 5 | CPU | 2-5 FPS |
116101

117-
| Device | Performance |
118-
|--------|------------|
119-
| Apple Silicon (M1+) | 20-30 FPS with MPS acceleration |
120-
| NVIDIA GPU | 25-60 FPS with CUDA |
121-
| CPU (modern x86) | 5-15 FPS |
122-
| Raspberry Pi 5 | 2-5 FPS |
102+
## Installation
123103

124-
## Contributing
104+
The `deploy.sh` bootstrapper handles everything — Python environment, GPU backend detection, and dependency installation. No manual setup required.
125105

126-
This skill is part of the [DeepCamera](https://github.com/SharpAI/DeepCamera) open-source project. Contributions welcome — see [Contributions.md](../../Contributions.md).
106+
```bash
107+
./deploy.sh
108+
```

‎skills/detection/yolo-detection-2026/config.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# YOLO 2026 Detection Skill — Default Configuration
2+
# These values can be overridden via AEGIS_SKILL_PARAMS or CLI arguments.
3+
4+
model_size: nano # nano | small | medium | large
5+
confidence: 0.5 # 0.1 – 1.0
6+
classes: "person,car,dog,cat" # Comma-separated COCO class names
7+
device: auto # auto | cpu | cuda | mps | rocm
8+
fps: 5 # 0.2 | 0.5 | 1 | 3 | 5 | 15

0 commit comments

Comments
 (0)