|
| 1 | +# Overview |
| 2 | + |
| 3 | +DeepLabCut Live GUI is a **PySide6-based desktop application** for running real-time DeepLabCut pose estimation experiments with **one or multiple cameras**, optional **processor plugins**, and **video recording** (with or without overlays). |
| 4 | + |
| 5 | +This page gives you a **guided tour of the main window**, explains the **core workflow**, and introduces the key concepts used throughout the user guide. |
| 6 | + |
| 7 | +--- |
| 8 | + |
| 9 | +## Main Window at a Glance |
| 10 | + |
| 11 | +When you launch the application (`dlclivegui`), you will see: |
| 12 | + |
| 13 | +- A **Controls panel** (left) for configuring cameras, inference, recording, and overlays |
| 14 | +- A **Video panel** (right) showing the live preview (single or tiled multi-camera) |
| 15 | +- A **Stats area** (below the video) summarizing camera, inference, and recorder performance |
| 16 | +- A **Status bar** (bottom) for short messages and warnings |
| 17 | + |
| 18 | +--- |
| 19 | + |
| 20 | +## Typical Workflow (Recommended) |
| 21 | + |
| 22 | +Most users will follow this sequence: |
| 23 | + |
| 24 | +1. **Configure Cameras** |
| 25 | + Use **Configure Cameras…** to select one or more cameras and their parameters. |
| 26 | + |
| 27 | +2. **Start Preview** |
| 28 | + Click **Start Preview** to begin streaming. |
| 29 | + - If multiple cameras are active, the preview becomes a **tiled view**. |
| 30 | + |
| 31 | +3. *(If ready)* **Start Pose Inference** |
| 32 | + Choose a **Model file**, optional **Processor**, select the **Inference Camera**, then click **Start pose inference**. |
| 33 | + - Toggle **Display pose predictions** to show or hide overlays. |
| 34 | + |
| 35 | +4. *(If ready)* **Start Recording** |
| 36 | + Choose an **Output directory**, session/run naming options, and encoding settings, then click **Start recording**. |
| 37 | + - Recording supports **all active cameras** in multi-camera mode. |
| 38 | + |
| 39 | +5. **Stop** |
| 40 | + Use **Stop Preview**, **Stop pose inference**, and/or **Stop recording** as needed. |
| 41 | + |
| 42 | +```{note} |
| 43 | +Pose inference requires the camera preview to be running. |
| 44 | +If you start pose inference while preview is stopped, the GUI will automatically start the preview first. |
| 45 | +``` |
| 46 | + |
| 47 | +--- |
| 48 | + |
| 49 | +## Main control panel |
| 50 | + |
| 51 | +:::{figure} ../_static/images/main_window/main_window_startup.png |
| 52 | +:label: fig:main_window_startup |
| 53 | +:alt: Screenshot of the main window |
| 54 | +:width: 100% |
| 55 | +:align: center |
| 56 | + |
| 57 | + The main window on startup, showing the Controls panel (left), Video panel (right), and Stats area (below video). |
| 58 | +::: |
| 59 | + |
| 60 | + |
| 61 | +### Camera settings |
| 62 | + |
| 63 | +**Purpose:** Define which cameras are available and active. |
| 64 | + |
| 65 | +- **Configure Cameras…** |
| 66 | + Opens the camera configuration dialog where you can: |
| 67 | + - Add, enable, or disable cameras |
| 68 | + - Select backend and index |
| 69 | + - Adjust camera-specific properties |
| 70 | + - Switch between single- and multi-camera setups |
| 71 | + |
| 72 | +- **Active** |
| 73 | + Displays a summary of configured cameras: |
| 74 | + - **Single camera:** `Name [backend:index] @ fps` |
| 75 | + - **Multiple cameras:** `N cameras: camA, camB, …` |
| 76 | + |
| 77 | +```{important} |
| 78 | +In multi-camera mode, pose inference runs on **one selected camera at a time** (the *Inference Camera*), |
| 79 | +even though preview and recording may include multiple cameras. |
| 80 | +``` |
| 81 | + |
| 82 | +--- |
| 83 | + |
| 84 | +### DLCLive settings |
| 85 | + |
| 86 | +**Purpose:** Configure and run pose inference on the live stream. |
| 87 | + |
| 88 | +- **Model file** |
| 89 | + Path to an exported DeepLabCut-Live model file (e.g. `.pt`, `.pb`). |
| 90 | + |
| 91 | +- **Processor folder / Processor** *(optional)* |
| 92 | + Processor plugins extend functionality (for example, experiment logic or external control). |
| 93 | + This allows for e.g. closed-loop control of devices based on pose estimation results. |
| 94 | + |
| 95 | +- **Inference Camera** |
| 96 | + Select which active camera is used for pose inference. |
| 97 | + In multi-camera preview, pose overlays are drawn only on the corresponding tile. |
| 98 | + |
| 99 | +- **Start pose inference / Stop pose inference** |
| 100 | + The button indicates inference state: |
| 101 | + - *Initializing DLCLive!* → Model loading |
| 102 | + - *DLCLive running!* → Inference active |
| 103 | + |
| 104 | +- **Display pose predictions** |
| 105 | + Toggle visualization of predicted keypoints. |
| 106 | + |
| 107 | +- **Auto-record video on processor command** *(optional)* |
| 108 | + Allows compatible processors to start and stop recording automatically. |
| 109 | + |
| 110 | +- **Processor Status** |
| 111 | + Displays processor-specific status information when available. |
| 112 | + |
| 113 | +--- |
| 114 | + |
| 115 | +### Recording |
| 116 | + |
| 117 | +**Purpose:** Save videos from active cameras ! |
| 118 | + |
| 119 | +Core settings: |
| 120 | +- **Output directory**: Base directory for all recordings |
| 121 | +- **Session name**: Logical grouping of runs (e.g. `mouseA_day1`) |
| 122 | +- **Use timestamp for run folder name**: |
| 123 | + - Enabled → `run_YYYYMMDD_HHMMSS_mmm` |
| 124 | + - Disabled → `run_0001`, `run_0002`, … |
| 125 | + |
| 126 | +A live preview label shows the *approximate* output path, including camera placeholders. |
| 127 | + |
| 128 | +Encoding options: |
| 129 | +- **Container** (e.g. `mp4`, `avi`, `mov`) |
| 130 | +- **Codec** (availability depends on OS and hardware) |
| 131 | +- **CRF** (quality/compression tradeoff; lower values = higher quality) |
| 132 | + |
| 133 | +Additional controls: |
| 134 | +- **Record video with overlays** |
| 135 | + Include pose predictions and/or bounding boxes directly in the recorded video. |
| 136 | + :::{caution} |
| 137 | + This **cannot be easily undone** once the recording is saved.<br> |
| 138 | + Use with caution if you want to preserve raw footage. |
| 139 | + ::: |
| 140 | +- **Start recording / Stop recording** |
| 141 | +- **Open recording folder** |
| 142 | + |
| 143 | +--- |
| 144 | + |
| 145 | +### Bounding Box Visualization |
| 146 | + |
| 147 | +**Purpose:** Show a bounding box around the detected subject. |
| 148 | + |
| 149 | +- **Show bounding box**: Enable or disable overlay |
| 150 | +- **Coordinates**: `x0`, `y0`, `x1`, `y1` |
| 151 | + |
| 152 | +In multi-camera mode, the bounding box is applied relative to the **inference camera tile**, ensuring correct alignment in tiled previews. |
| 153 | + |
| 154 | +--- |
| 155 | + |
| 156 | +## Video Panel and Stats |
| 157 | + |
| 158 | +### Video preview |
| 159 | + |
| 160 | +- Displays a logo screen when idle |
| 161 | +- Shows live video when preview is running |
| 162 | +- Uses a tiled layout automatically when multiple cameras are active |
| 163 | + |
| 164 | +### Stats panel |
| 165 | + |
| 166 | +Three continuously updated sections: |
| 167 | + |
| 168 | +- **Camera**: Per-camera measured frame rate |
| 169 | +- **DLC Processor**: Inference throughput, latency, queue depth, and dropped frames |
| 170 | +- **Recorder**: Recording status and write performance |
| 171 | + |
| 172 | +```{tip} |
| 173 | +Stats text can be selected and copied directly, which is useful for debugging or reporting performance issues. |
| 174 | +``` |
| 175 | + |
| 176 | +--- |
| 177 | + |
| 178 | +## Menus |
| 179 | + |
| 180 | +### File |
| 181 | +- Load configuration… |
| 182 | +- Save configuration |
| 183 | +- Save configuration as… |
| 184 | +- Open recording folder |
| 185 | +- Close window |
| 186 | + |
| 187 | +### View → Appearance |
| 188 | +- System theme |
| 189 | +- Dark theme |
| 190 | + |
| 191 | +--- |
| 192 | + |
| 193 | +## Configuration and Persistence |
| 194 | + |
| 195 | +The GUI can restore settings across sessions using: |
| 196 | + |
| 197 | +- Explicitly saved **JSON configuration files** |
| 198 | +- A stored snapshot of the most recent configuration |
| 199 | +- Remembered paths (e.g. last-used model directory) |
| 200 | + |
| 201 | +```{tip} |
| 202 | +For reproducible experiments, prefer saving and versioning JSON configuration files |
| 203 | +rather than relying only on remembered application state. |
| 204 | +``` |
| 205 | + |
| 206 | +--- |
| 207 | + |
| 208 | +## Next Steps |
| 209 | + |
| 210 | +- **Camera Setup**: Add and validate cameras, start preview |
| 211 | +- **Inference Setup**: Select a model, start pose inference, interpret performance metrics |
| 212 | +- **First Recording**: Understand session/run structure and verify recorded output |
0 commit comments