Skip to content

Commit 6cf85cd

Browse files
solderzzcsolderzzc
committed
refactor(depth-estimation): improve ONNX support, update deploy and benchmark scripts
- transform.py: add ONNX model download and inference path - deploy.bat: refine Python discovery and dependency flow - requirements: update dependency specs - benchmark.py: unify inference routing for CoreML/ONNX/PyTorch
1 parent 95c2268 commit 6cf85cd

7 files changed

Lines changed: 304 additions & 205 deletions

File tree

‎.agents/workflows/command-execution.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
description: Best practices for running terminal commands to prevent stuck "Running.." states
3+
---
4+
5+
# Command Execution Best Practices
6+
7+
These rules prevent commands from getting stuck in a "Running.." state due to the IDE
8+
failing to detect command completion. Apply these on EVERY `run_command` call.
9+
10+
## Rule 1: Use High `WaitMsBeforeAsync` for Fast Commands
11+
12+
For commands expected to finish within a few seconds (git status, git log, git diff --stat,
13+
ls, cat, echo, pip show, python --version, etc.), ALWAYS set `WaitMsBeforeAsync` to **5000**.
14+
15+
This gives the command enough time to complete synchronously so the IDE never sends it
16+
to background monitoring (where completion detection can fail).
17+
18+
```
19+
WaitMsBeforeAsync: 5000 # for fast commands (< 5s expected)
20+
WaitMsBeforeAsync: 500 # ONLY for long-running commands (servers, builds, installs)
21+
```
22+
23+
## Rule 2: Limit Output to Prevent Truncation Cascades
24+
25+
When output gets truncated, the IDE may auto-trigger follow-up commands (like `git status --short`)
26+
that can get stuck. Prevent this by limiting output upfront:
27+
28+
- Use `--short`, `--stat`, `--oneline`, `-n N` flags on git commands
29+
- Pipe through `head -n 50` for potentially long output
30+
- Use `--no-pager` explicitly on git commands
31+
- Prefer `git diff --stat` over `git diff` when full diff isn't needed
32+
33+
Examples:
34+
```bash
35+
# GOOD: limited output
36+
git log -n 5 --oneline
37+
git diff --stat
38+
git diff -- path/to/file.py | head -n 80
39+
40+
# BAD: unbounded output that may truncate
41+
git log
42+
git diff
43+
```
44+
45+
## Rule 3: Batch Related Quick Commands
46+
47+
Instead of running multiple fast commands sequentially (which can cause race conditions),
48+
batch them into a single call with separators:
49+
50+
```bash
51+
# GOOD: one call, no race conditions
52+
git status --short && echo "---" && git log -n 3 --oneline && echo "---" && git diff --stat
53+
54+
# BAD: three separate rapid calls
55+
# Call 1: git status --short
56+
# Call 2: git log -n 3 --oneline
57+
# Call 3: git diff --stat
58+
```
59+
60+
## Rule 4: Always Follow Up Async Commands with `command_status`
61+
62+
If a command goes async (returns a background command ID), immediately call `command_status`
63+
with `WaitDurationSeconds: 30` to block until completion rather than leaving it in limbo.
64+
65+
## Rule 5: Terminate Stuck Commands
66+
67+
If a command appears stuck in "Running.." but should have completed, use `send_command_input`
68+
with `Terminate: true` to force-kill it, then re-run with a higher `WaitMsBeforeAsync`.

‎skills/transformation/depth-estimation/deploy.bat‎

Lines changed: 84 additions & 184 deletions
Original file line numberDiff line numberDiff line change
@@ -1,223 +1,123 @@
11
@echo off
2-
REM deploy.bat — Zero-assumption bootstrapper for Depth Estimation Skill (Windows)
3-
REM
4-
REM Probes the system for Python, GPU backends, and installs the minimum
5-
REM viable stack. Called by Aegis skill-runtime-manager during installation.
2+
setlocal enabledelayedexpansion
3+
REM ═══════════════════════════════════════════════════════════════════
4+
REM Depth Estimation Skill — Windows Deployment (ONNX Runtime)
65
REM
7-
REM Uses skills\lib\env_config.py for hardware detection.
6+
REM GPU detection cascade:
7+
REM 1. nvidia-smi found → onnxruntime-gpu (CUDA + TensorRT EPs)
8+
REM 2. Non-NVIDIA GPU found (WMI) → onnxruntime-directml
9+
REM 3. No GPU → onnxruntime (CPU)
810
REM
9-
REM Exit codes:
10-
REM 0 = success
11-
REM 1 = fatal error (no Python found)
12-
REM 2 = partial success (CPU-only fallback)
13-
14-
setlocal enabledelayedexpansion
15-
16-
set "SKILL_DIR=%~dp0"
17-
REM Remove trailing backslash
18-
if "%SKILL_DIR:~-1%"=="\" set "SKILL_DIR=%SKILL_DIR:~0,-1%"
19-
set "VENV_DIR=%SKILL_DIR%\.venv"
20-
set "LOG_PREFIX=[Depth-deploy]"
21-
22-
REM Resolve lib dir (two levels up + lib)
23-
set "LIB_DIR="
24-
if exist "%SKILL_DIR%\..\..\lib\env_config.py" (
25-
pushd "%SKILL_DIR%\..\..\lib"
26-
set "LIB_DIR=!CD!"
27-
popd
28-
)
11+
REM Then downloads ONNX model from HuggingFace.
12+
REM ═══════════════════════════════════════════════════════════════════
2913

30-
REM ─── Step 1: Find Python ───────────────────────────────────────────────────
31-
32-
echo %LOG_PREFIX% Searching for Python...>&2
14+
echo [DepthDeploy] Starting depth-estimation skill deployment...
15+
echo [DepthDeploy] Platform: Windows (%PROCESSOR_ARCHITECTURE%)
3316

17+
REM ── 1. Find Python ─────────────────────────────────────────────────
3418
set "PYTHON_CMD="
3519

36-
REM Try the Windows Python launcher (py.exe) first — ships with python.org installer
37-
for %%V in (3.12 3.11 3.10 3.9) do (
38-
if not defined PYTHON_CMD (
39-
py -%%V --version >nul 2>&1
40-
if !errorlevel! equ 0 (
41-
set "PYTHON_CMD=py -%%V"
42-
)
43-
)
20+
REM Try py launcher first (most reliable on Windows)
21+
py --version >nul 2>&1
22+
if %ERRORLEVEL% equ 0 (
23+
set "PYTHON_CMD=py"
24+
goto :found_python
4425
)
4526

46-
REM Fallback: bare python3 / python on PATH
47-
if not defined PYTHON_CMD (
48-
python3 --version >nul 2>&1
49-
if !errorlevel! equ 0 (
50-
REM Verify version >= 3.9
51-
for /f "tokens=2 delims= " %%A in ('python3 --version 2^>^&1') do set "_pyver=%%A"
52-
for /f "tokens=1,2 delims=." %%A in ("!_pyver!") do (
53-
if %%A geq 3 if %%B geq 9 set "PYTHON_CMD=python3"
54-
)
55-
)
27+
REM Try python (could be Python 3 on PATH)
28+
python --version >nul 2>&1
29+
if %ERRORLEVEL% equ 0 (
30+
set "PYTHON_CMD=python"
31+
goto :found_python
5632
)
5733

58-
if not defined PYTHON_CMD (
59-
python --version >nul 2>&1
60-
if !errorlevel! equ 0 (
61-
for /f "tokens=2 delims= " %%A in ('python --version 2^>^&1') do set "_pyver=%%A"
62-
for /f "tokens=1,2 delims=." %%A in ("!_pyver!") do (
63-
if %%A geq 3 if %%B geq 9 set "PYTHON_CMD=python"
64-
)
65-
)
66-
)
34+
echo [DepthDeploy] ERROR: Python not found. Install Python 3.9+ from python.org
35+
exit /b 1
6736

68-
if not defined PYTHON_CMD (
69-
echo %LOG_PREFIX% ERROR: No Python ^>=3.9 found. Install Python 3.9+ and retry.>&2
70-
echo {"event": "error", "stage": "python", "message": "No Python >=3.9 found"}
71-
exit /b 1
72-
)
73-
74-
for /f "tokens=*" %%A in ('!PYTHON_CMD! --version 2^>^&1') do set "PY_VERSION=%%A"
75-
echo %LOG_PREFIX% Using Python: %PYTHON_CMD% (%PY_VERSION%)>&2
76-
echo {"event": "progress", "stage": "python", "message": "Found %PY_VERSION%"}
37+
:found_python
38+
echo [DepthDeploy] Using Python: %PYTHON_CMD%
39+
%PYTHON_CMD% --version
7740

78-
REM ─── Step 2: Create virtual environment ────────────────────────────────────
79-
80-
if not exist "%VENV_DIR%\Scripts\python.exe" (
81-
echo %LOG_PREFIX% Creating virtual environment...>&2
82-
%PYTHON_CMD% -m venv "%VENV_DIR%"
83-
if !errorlevel! neq 0 (
84-
echo %LOG_PREFIX% ERROR: Failed to create virtual environment>&2
85-
echo {"event": "error", "stage": "venv", "message": "Failed to create venv"}
41+
REM ── 2. Create venv ─────────────────────────────────────────────────
42+
if not exist ".venv\Scripts\python.exe" (
43+
echo [DepthDeploy] Creating virtual environment...
44+
%PYTHON_CMD% -m venv .venv
45+
if %ERRORLEVEL% neq 0 (
46+
echo [DepthDeploy] ERROR: Failed to create venv
8647
exit /b 1
8748
)
8849
)
8950

90-
set "PIP=%VENV_DIR%\Scripts\pip.exe"
91-
set "VPYTHON=%VENV_DIR%\Scripts\python.exe"
51+
set "VENV_PIP=.venv\Scripts\pip.exe"
52+
set "VENV_PYTHON=.venv\Scripts\python.exe"
9253

93-
"%PIP%" install --upgrade pip -q >nul 2>&1
54+
echo [DepthDeploy] Upgrading pip...
55+
%VENV_PYTHON% -m pip install --upgrade pip >nul 2>&1
9456

95-
echo {"event": "progress", "stage": "venv", "message": "Virtual environment ready"}
57+
REM ── 3. Detect GPU ──────────────────────────────────────────────────
58+
echo [DepthDeploy] Detecting GPU hardware...
9659

97-
REM ─── Step 2.5: Bundle env_config.py alongside transform.py ─────────────────
60+
set "GPU_BACKEND=cpu"
61+
set "REQUIREMENTS_FILE=requirements_cpu.txt"
9862

99-
if defined LIB_DIR (
100-
if exist "%LIB_DIR%\env_config.py" (
101-
copy /Y "%LIB_DIR%\env_config.py" "%SKILL_DIR%\scripts\env_config.py" >nul 2>&1
102-
echo %LOG_PREFIX% Bundled env_config.py into scripts\>&2
103-
)
63+
REM 3a. Check for NVIDIA GPU via nvidia-smi
64+
nvidia-smi --query-gpu=name --format=csv,noheader,nounits >nul 2>&1
65+
if %ERRORLEVEL% equ 0 (
66+
echo [DepthDeploy] NVIDIA GPU detected:
67+
nvidia-smi --query-gpu=name,memory.total --format=csv,noheader,nounits
68+
set "GPU_BACKEND=cuda"
69+
set "REQUIREMENTS_FILE=requirements_cuda.txt"
70+
goto :gpu_detected
10471
)
10572

106-
REM ─── Step 3: Detect hardware via env_config ────────────────────────────────
107-
108-
set "BACKEND=cpu"
109-
110-
REM Find env_config.py — bundled copy or repo lib\
111-
set "ENV_CONFIG_DIR="
112-
if exist "%SKILL_DIR%\scripts\env_config.py" (
113-
set "ENV_CONFIG_DIR=%SKILL_DIR%\scripts"
114-
) else if defined LIB_DIR (
115-
if exist "%LIB_DIR%\env_config.py" (
116-
set "ENV_CONFIG_DIR=%LIB_DIR%"
117-
)
73+
REM 3b. Check for any GPU via WMI (AMD, Intel, Qualcomm)
74+
for /f "tokens=*" %%G in ('powershell -NoProfile -Command "Get-CimInstance Win32_VideoController | Where-Object { $_.Name -notlike '*Microsoft*' -and $_.Name -notlike '*Remote*' } | Select-Object -ExpandProperty Name" 2^>nul') do (
75+
echo [DepthDeploy] GPU found: %%G
76+
set "GPU_BACKEND=directml"
77+
set "REQUIREMENTS_FILE=requirements_directml.txt"
11878
)
11979

120-
if defined ENV_CONFIG_DIR (
121-
echo %LOG_PREFIX% Detecting hardware via env_config.py...>&2
122-
123-
REM Run env_config detection via Python (temp file avoids for /f quoting issues)
124-
set "ENV_CONFIG_DIR=!ENV_CONFIG_DIR!"
125-
set "_TMPBACK=%TEMP%\_aegis_detect_backend.txt"
126-
"%VPYTHON%" -c "import sys,os; sys.path.insert(0, os.environ['ENV_CONFIG_DIR']); from env_config import HardwareEnv; env = HardwareEnv.detect(); print(env.backend)" > "!_TMPBACK!" 2>nul
127-
if exist "!_TMPBACK!" (
128-
set /p DETECTED_BACKEND=<"!_TMPBACK!"
129-
del "!_TMPBACK!" 2>nul
130-
)
80+
:gpu_detected
81+
echo [DepthDeploy] Selected backend: %GPU_BACKEND%
82+
echo [DepthDeploy] Requirements: %REQUIREMENTS_FILE%
13183

132-
REM Validate backend value (Windows: only cuda, intel, cpu are realistic)
133-
if "!DETECTED_BACKEND!"=="cuda" (
134-
set "BACKEND=cuda"
135-
) else if "!DETECTED_BACKEND!"=="intel" (
136-
set "BACKEND=intel"
137-
) else if "!DETECTED_BACKEND!"=="cpu" (
138-
set "BACKEND=cpu"
139-
) else (
140-
echo %LOG_PREFIX% env_config returned '!DETECTED_BACKEND!', falling back to heuristic>&2
141-
set "BACKEND=cpu"
142-
)
143-
144-
echo %LOG_PREFIX% env_config detected backend: !BACKEND!>&2
145-
) else (
146-
echo %LOG_PREFIX% env_config.py not found, using heuristic detection...>&2
147-
148-
REM Fallback: inline GPU detection via nvidia-smi
149-
where nvidia-smi >nul 2>&1
150-
if !errorlevel! equ 0 (
151-
for /f "tokens=*" %%G in ('nvidia-smi --query-gpu^=driver_version --format^=csv^,noheader 2^>nul') do (
152-
if not "%%G"=="" (
153-
set "BACKEND=cuda"
154-
echo %LOG_PREFIX% Detected NVIDIA GPU ^(driver: %%G^)>&2
155-
)
156-
)
157-
)
158-
)
159-
160-
echo {"event": "progress", "stage": "gpu", "backend": "!BACKEND!", "message": "Compute backend: !BACKEND!"}
161-
162-
REM ─── Step 4: Install requirements ──────────────────────────────────────────
163-
164-
set "REQ_FILE=%SKILL_DIR%\requirements_!BACKEND!.txt"
165-
166-
if not exist "!REQ_FILE!" (
167-
echo %LOG_PREFIX% WARNING: !REQ_FILE! not found, falling back to CPU>&2
168-
set "REQ_FILE=%SKILL_DIR%\requirements_cpu.txt"
169-
set "BACKEND=cpu"
170-
)
171-
172-
echo %LOG_PREFIX% Installing dependencies from !REQ_FILE! ...>&2
173-
echo {"event": "progress", "stage": "install", "message": "Installing !BACKEND! dependencies..."}
174-
175-
if "!BACKEND!"=="cuda" (
176-
REM CUDA on Windows: install torch with CUDA index, then remaining deps
177-
"%PIP%" install torch torchvision --index-url https://download.pytorch.org/whl/cu124 -q 2>&1 | findstr /V "^$" >nul
178-
if !errorlevel! neq 0 (
179-
echo %LOG_PREFIX% WARNING: CUDA torch install failed, trying cu121...>&2
180-
"%PIP%" install torch torchvision --index-url https://download.pytorch.org/whl/cu121 -q 2>&1 | findstr /V "^$" >nul
181-
)
182-
REM Install remaining requirements (depth-anything-v2, tensorrt, etc.)
183-
"%PIP%" install -r "!REQ_FILE!" -q 2>&1 | findstr /V "^$" >nul
184-
) else (
185-
"%PIP%" install -r "!REQ_FILE!" -q 2>&1 | findstr /V "^$" >nul
84+
REM ── 4. Install dependencies ────────────────────────────────────────
85+
if not exist "%REQUIREMENTS_FILE%" (
86+
echo [DepthDeploy] WARNING: %REQUIREMENTS_FILE% not found, falling back to requirements_cpu.txt
87+
set "REQUIREMENTS_FILE=requirements_cpu.txt"
18688
)
18789

188-
REM ─── Step 5: Pre-build TensorRT engine (CUDA only) ─────────────────────────
189-
190-
if "!BACKEND!"=="cuda" (
191-
echo %LOG_PREFIX% Pre-building TensorRT FP16 engine ^(30-120s one-time build^)...>&2
192-
echo {"event": "progress", "stage": "optimize", "message": "Building TensorRT FP16 engine (~30-120s)..."}
193-
194-
"%VPYTHON%" -c "import sys,os; sys.path.insert(0, '%SKILL_DIR%\\scripts'); from transform import DepthEstimationSkill, PYTORCH_CONFIGS, TRT_CACHE_DIR; import torch; gpu_tag = torch.cuda.get_device_name(0).replace(' ', '_').lower(); cfg = PYTORCH_CONFIGS['depth-anything-v2-small']; engine_path = TRT_CACHE_DIR / f'{cfg[\"filename\"].replace(\".pth\", \"\")}_fp16_{gpu_tag}.trt'; print(f'Engine exists: {engine_path.exists()}') if engine_path.exists() else None; skill = DepthEstimationSkill(); config = {'model': 'depth-anything-v2-small', 'device': 'auto', 'colormap': 'inferno', 'blend_mode': 'depth_only'}; info = skill._load_tensorrt('depth-anything-v2-small', config); print(f'TensorRT engine built: {info}')" 2>&1
195-
196-
if !errorlevel! equ 0 (
197-
echo {"event": "progress", "stage": "optimize", "message": "TensorRT engine built successfully"}
198-
) else (
199-
echo %LOG_PREFIX% WARNING: TensorRT build failed, will use PyTorch CUDA at runtime>&2
200-
echo {"event": "progress", "stage": "optimize", "message": "TensorRT build failed — PyTorch CUDA fallback"}
90+
echo [DepthDeploy] Installing %REQUIREMENTS_FILE%...
91+
%VENV_PIP% install -r %REQUIREMENTS_FILE%
92+
if %ERRORLEVEL% neq 0 (
93+
echo [DepthDeploy] WARNING: Install failed for %REQUIREMENTS_FILE%
94+
if not "%GPU_BACKEND%"=="cpu" (
95+
echo [DepthDeploy] Falling back to CPU requirements...
96+
%VENV_PIP% install -r requirements_cpu.txt
20197
)
20298
)
20399

204-
REM ─── Step 6: Verify installation ───────────────────────────────────────────
205-
206-
echo %LOG_PREFIX% Verifying installation...>&2
100+
REM ── 5. Download ONNX model ─────────────────────────────────────────
101+
echo [DepthDeploy] Downloading ONNX model from HuggingFace...
207102

208-
if defined ENV_CONFIG_DIR (
209-
"%VPYTHON%" -c "import sys,os,json; sys.path.insert(0, os.environ['ENV_CONFIG_DIR']); from env_config import HardwareEnv; env = HardwareEnv.detect(); print(json.dumps(env.to_dict(), indent=2))" 2>&1
103+
%VENV_PYTHON% -c "from huggingface_hub import snapshot_download; snapshot_download('onnx-community/depth-anything-v2-small', local_dir='models/onnx/depth-anything-v2-small', allow_patterns=['onnx/*.onnx', 'onnx/*.json', 'preprocessor_config.json', 'config.json'])"
104+
if %ERRORLEVEL% equ 0 (
105+
echo [DepthDeploy] ONNX model downloaded successfully
210106
) else (
211-
"%VPYTHON%" -c "import torch; print(f'torch={torch.__version__}, CUDA={torch.cuda.is_available()}')" 2>&1
107+
echo [DepthDeploy] WARNING: Model download failed — will retry on first run
212108
)
213109

214-
"%VPYTHON%" -c "import sys; sys.path.insert(0, '%SKILL_DIR%\\scripts'); from transform import DepthEstimationSkill; print('transform.py: OK')" 2>&1
215-
if !errorlevel! neq 0 (
216-
echo %LOG_PREFIX% WARNING: transform.py import check failed>&2
110+
REM ── 6. Verify installation ─────────────────────────────────────────
111+
echo [DepthDeploy] Verifying ONNX Runtime installation...
112+
113+
%VENV_PYTHON% -c "import onnxruntime as ort; eps = ort.get_available_providers(); print(f'[DepthDeploy] Available EPs: {eps}')"
114+
if %ERRORLEVEL% neq 0 (
115+
echo [DepthDeploy] ERROR: ONNX Runtime import failed
116+
exit /b 1
217117
)
218118

219-
echo {"event": "complete", "backend": "!BACKEND!", "message": "Depth Estimation skill installed (!BACKEND! backend)"}
220-
echo %LOG_PREFIX% Done! Backend: !BACKEND!>&2
119+
REM Log detected execution providers
120+
%VENV_PYTHON% -c "import onnxruntime as ort; eps = ort.get_available_providers(); cuda = 'CUDAExecutionProvider' in eps; trt = 'TensorrtExecutionProvider' in eps; dml = 'DmlExecutionProvider' in eps; print(f'[DepthDeploy] CUDA EP: {cuda}, TensorRT EP: {trt}, DirectML EP: {dml}')"
221121

222-
endlocal
122+
echo [DepthDeploy] Deployment complete (%GPU_BACKEND% backend)
223123
exit /b 0

‎skills/transformation/depth-estimation/requirements_cpu.txt‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
1-
# Depth Estimation — CPU-only requirements
2-
# Smallest install — no GPU acceleration
3-
--extra-index-url https://download.pytorch.org/whl/cpu
4-
torch~=2.7.0
5-
torchvision~=0.22.0
6-
depth-anything-v2>=0.1.0
1+
# Depth Estimation — ONNX Runtime CPU-only
2+
# Installed by deploy.bat when no GPU is detected.
3+
#
4+
# Smallest install footprint. No GPU acceleration.
5+
6+
onnxruntime>=1.17.0
7+
8+
# ── Common dependencies ─────────────────────────────────────────────
79
huggingface_hub>=0.20.0
810
numpy>=1.24.0
911
opencv-python-headless>=4.8.0

0 commit comments

Comments
 (0)