|
| 1 | +# Depth Estimation — Privacy Transform |
| 2 | + |
| 3 | +Transform camera feeds into **colorized depth maps** using [Depth Anything v2](https://github.com/DepthAnything/Depth-Anything-V2), providing real-time privacy protection for security monitoring. |
| 4 | + |
| 5 | +In **privacy mode** (`depth_only`), the scene is fully anonymized — no faces, no clothing, no identifying features — while preserving spatial layout and activity patterns for security awareness. |
| 6 | + |
| 7 | + |
| 8 | + |
| 9 | + |
| 10 | +## How It Works |
| 11 | + |
| 12 | +``` |
| 13 | +Camera Frame → Depth Anything v2 → Colorized Depth Map → Aegis Overlay |
| 14 | + (BGR) (monocular) (warm=near, cool=far) (0.5 FPS) |
| 15 | +``` |
| 16 | + |
| 17 | +The depth model converts each frame into a distance map where **warm colors** (red/orange) indicate nearby objects and **cool colors** (blue/purple) indicate distant ones. This preserves enough spatial information to understand activity (someone approaching, car in driveway, etc.) without revealing identity. |
| 18 | + |
| 19 | +## Hardware Support |
| 20 | + |
| 21 | +Auto-detected via `HardwareEnv` from `skills/lib/env_config.py`: |
| 22 | + |
| 23 | +| Platform | Backend | Notes | |
| 24 | +|----------|---------|-------| |
| 25 | +| **NVIDIA** | CUDA | FP16 on GPU | |
| 26 | +| **AMD** | ROCm | PyTorch HIP | |
| 27 | +| **Apple Silicon** | MPS | Unified memory, leaves Neural Engine free | |
| 28 | +| **Intel** | OpenVINO | CPU + NPU support | |
| 29 | +| **CPU** | PyTorch | Fallback, slower | |
| 30 | + |
| 31 | +## Models |
| 32 | + |
| 33 | +| Model | Size | Speed | Quality | |
| 34 | +|-------|------|-------|---------| |
| 35 | +| `depth-anything-v2-small` | 25MB | Fast | Good (default) | |
| 36 | +| `depth-anything-v2-base` | 98MB | Medium | Better | |
| 37 | +| `depth-anything-v2-large` | 335MB | Slow | Best | |
| 38 | + |
| 39 | +Weights are downloaded from HuggingFace Hub on first run and cached locally. |
| 40 | + |
| 41 | +## Display Modes |
| 42 | + |
| 43 | +- **`depth_only`** (default) — Full anonymization. Only the depth map is shown. |
| 44 | +- **`overlay`** — Depth map blended on top of the original feed (adjustable opacity). |
| 45 | +- **`side_by_side`** — Original and depth map shown next to each other. |
| 46 | + |
| 47 | +## Setup |
| 48 | + |
| 49 | +```bash |
| 50 | +python3 -m venv .venv && source .venv/bin/activate |
| 51 | +pip install -r requirements.txt |
| 52 | +``` |
| 53 | + |
| 54 | +## Integration with Aegis |
| 55 | + |
| 56 | +This skill communicates with Aegis via **JSONL over stdin/stdout**. Aegis sends frame events, the skill returns transformed frames (base64 JPEG). See [SKILL.md](SKILL.md) for the full protocol specification and the `TransformSkillBase` interface for building new privacy skills. |
| 57 | + |
| 58 | +## Creating New Privacy Skills |
| 59 | + |
| 60 | +Subclass `TransformSkillBase` and implement two methods: |
| 61 | + |
| 62 | +```python |
| 63 | +from transform_base import TransformSkillBase |
| 64 | + |
| 65 | +class MyPrivacySkill(TransformSkillBase): |
| 66 | + def load_model(self, config): |
| 67 | + self.model = load_my_model() |
| 68 | + return {"model": "my-model", "device": self.device} |
| 69 | + |
| 70 | + def transform_frame(self, image, metadata): |
| 71 | + return self.model.anonymize(image) |
| 72 | + |
| 73 | +if __name__ == "__main__": |
| 74 | + MyPrivacySkill().run() |
| 75 | +``` |
| 76 | + |
| 77 | +The base class handles JSONL protocol, performance tracking, hardware detection, rate limiting, and graceful shutdown. |
0 commit comments