Skip to content

Commit 37116b1

Browse files
committed
Reintroduce previous README
1 parent 48741fb commit 37116b1

File tree

1 file changed

+38
-288
lines changed

1 file changed

+38
-288
lines changed

README.md

Lines changed: 38 additions & 288 deletions
Original file line numberDiff line numberDiff line change
@@ -1,319 +1,69 @@
1-
<!-- FIXME @C-Achard out of date -->
2-
# DeepLabCut Live GUI
1+
# DeepLabCut-Live! GUI <img src="https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1596193544929-NHMVMXPVEYZ6R4I45DSR/ke17ZwdGBToddI8pDm48kOHwsIwndRGCkvek0XAcW4ZZw-zPPgdn4jUwVcJE1ZvWEtT5uBSRWt4vQZAgTJucoTqqXjS3CfNDSuuf31e0tVH0wqgmu6zkAOZ3crWCtkmLcPIuzHaxU8QRzZwtrVtHupu3E9Ef3XsXP1C_826c-iU/DLCLIVEGUI_LOGO.png?format=750w" width="350" title="DLC-live GUI" alt="DLC LIVE! GUI" align="right" vspace = "100">
32

4-
A modern PySide6 GUI for running [DeepLabCut-live](https://github.com/DeepLabCut/DeepLabCut-live) experiments with real-time pose estimation. The application streams frames from industrial or consumer cameras, performs DLCLive inference, and records high-quality video with synchronized pose data.
3+
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
4+
![PyPI - Python Version](https://img.shields.io/pypi/v/deeplabcut-live-gui)
5+
![PyPI - Downloads](https://img.shields.io/pypi/dm/deeplabcut-live-gui?color=purple)
6+
![Python package](https://github.com/DeepLabCut/DeepLabCut-live/workflows/Python%20package/badge.svg)
57

6-
## Features
8+
[![License](https://img.shields.io/pypi/l/deeplabcutcore.svg)](https://github.com/DeepLabCut/deeplabcutlive/raw/master/LICENSE)
9+
[![Image.sc forum](https://img.shields.io/badge/dynamic/json.svg?label=forum&amp;url=https%3A%2F%2Fforum.image.sc%2Ftags%2Fdeeplabcut.json&amp;query=%24.topic_list.tags.0.topic_count&amp;colorB=brightgreen&amp;&amp;suffix=%20topics&amp;logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAABPklEQVR42m3SyyqFURTA8Y2BER0TDyExZ+aSPIKUlPIITFzKeQWXwhBlQrmFgUzMMFLKZeguBu5y+//17dP3nc5vuPdee6299gohUYYaDGOyyACq4JmQVoFujOMR77hNfOAGM+hBOQqB9TjHD36xhAa04RCuuXeKOvwHVWIKL9jCK2bRiV284QgL8MwEjAneeo9VNOEaBhzALGtoRy02cIcWhE34jj5YxgW+E5Z4iTPkMYpPLCNY3hdOYEfNbKYdmNngZ1jyEzw7h7AIb3fRTQ95OAZ6yQpGYHMMtOTgouktYwxuXsHgWLLl+4x++Kx1FJrjLTagA77bTPvYgw1rRqY56e+w7GNYsqX6JfPwi7aR+Y5SA+BXtKIRfkfJAYgj14tpOF6+I46c4/cAM3UhM3JxyKsxiOIhH0IO6SH/A1Kb1WBeUjbkAAAAAElFTkSuQmCC)](https://forum.image.sc/tags/deeplabcut)
10+
[![Gitter](https://badges.gitter.im/DeepLabCut/community.svg)](https://gitter.im/DeepLabCut/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
11+
[![Twitter Follow](https://img.shields.io/twitter/follow/DeepLabCut.svg?label=DeepLabCut&style=social)](https://twitter.com/DeepLabCut)
712

8-
### Core Functionality
9-
- **Modern Python Stack**: Python 3.10+ compatible codebase with PySide6 interface
10-
- **Multi-Backend Camera Support**: OpenCV, GenTL (Harvesters), Aravis, and Basler (pypylon)
11-
- **Real-Time Pose Estimation**: Live DLCLive inference with configurable models (TensorFlow, PyTorch)
12-
- **High-Performance Recording**: Hardware-accelerated video encoding via FFmpeg
13-
- **Flexible Configuration**: Single JSON file for all settings with GUI editing
13+
GUI to run [DeepLabCut-live](https://github.com/DeepLabCut/DeepLabCut-live) on a video feed, record videos, and record external timestamps.
1414

15-
### Camera Features
16-
- **Multiple Backends**:
17-
- OpenCV - Universal webcam support
18-
- GenTL - Industrial cameras via Harvesters (Windows/Linux)
19-
- Aravis - GenICam/GigE cameras (Linux/macOS)
20-
- Basler - Basler cameras via pypylon
21-
- **Smart Device Detection**: Automatic camera enumeration without unnecessary probing
22-
- **Camera Controls**: Exposure time, gain, frame rate, and ROI cropping
23-
- **Live Preview**: Real-time camera feed with rotation support (0°, 90°, 180°, 270°)
15+
## [Installation Instructions](docs/install.md)
2416

25-
### DLCLive Features
26-
- **Model Support**: Only PyTorch models! (in theory also tensorflow models work)
27-
- **Processor System**: Plugin architecture for custom pose processing
28-
- **Auto-Recording**: Automatic video recording triggered by processor commands
29-
- **Performance Metrics**: Real-time FPS, latency, and queue monitoring
30-
- **Pose Visualization**: Optional overlay of detected keypoints on live feed
17+
## Getting Started
3118

32-
### Recording Features
33-
- **Hardware Encoding**: NVENC (NVIDIA GPU) and software codecs (libx264, libx265)
34-
- **Configurable Quality**: CRF-based quality control
35-
- **Multiple Formats**: MP4, AVI, MOV containers
36-
- **Timestamp Support**: Frame-accurate timestamps for synchronization
37-
- **Performance Monitoring**: Write FPS, buffer status, and dropped frame tracking
19+
#### Open DeepLabCut-live-GUI
3820

39-
### User Interface
40-
- **Intuitive Layout**: Organized control panels with clear separation of concerns
41-
- **Configuration Management**: Load/save settings, support for multiple configurations
42-
- **Status Indicators**: Real-time feedback on camera, inference, and recording status
43-
- **Bounding Box Tool**: Visual overlay for ROI definition
21+
In a terminal, activate the conda or virtual environment where DeepLabCut-live-GUI is installed, then run:
4422

45-
## Installation
46-
47-
### Basic Installation
48-
49-
```bash
50-
pip install deeplabcut-live-gui
51-
```
52-
53-
This installs the core package with OpenCV camera support.
54-
55-
### Full Installation with Optional Dependencies
56-
57-
```bash
58-
# Install with gentl support
59-
pip install deeplabcut-live-gui[gentl]
60-
```
61-
62-
### Platform-Specific Camera Backend Setup
63-
64-
#### Windows (GenTL for Industrial Cameras)
65-
1. Install camera vendor drivers and SDK
66-
2. Ensure GenTL producer (.cti) files are accessible
67-
3. Common locations:
68-
- `C:\Program Files\The Imaging Source Europe GmbH\IC4 GenTL Driver\bin\`
69-
- Check vendor documentation for CTI file location
70-
71-
#### Linux (Aravis for GenICam Cameras - Recommended)
72-
NOT tested
73-
```bash
74-
# Ubuntu/Debian
75-
sudo apt-get install gir1.2-aravis-0.8 python3-gi
76-
77-
# Fedora
78-
sudo dnf install aravis python3-gobject
7923
```
80-
81-
#### macOS (Aravis)
82-
NOT tested
83-
```bash
84-
brew install aravis
85-
pip install pygobject
24+
dlclivegui
8625
```
8726

88-
#### Basler Cameras (All Platforms)
89-
NOT tested
90-
```bash
91-
# Install Pylon SDK from Basler website
92-
# Then install pypylon
93-
pip install pypylon
94-
```
9527

96-
### Hardware Acceleration (Optional)
28+
#### Configurations
9729

98-
For NVIDIA GPU encoding (highly recommended for high-resolution/high-FPS recording):
99-
```bash
100-
# Ensure NVIDIA drivers are installed
101-
# FFmpeg with NVENC support will be used automatically
102-
```
10330

104-
## Quick Start
31+
First, create a configuration file: select the drop down menu labeled `Config`, and click `Create New Config`. All settings, such as details about cameras, DLC networks, and DLC-live Processors, will be saved into configuration files so that you can close and reopen the GUI without losing all of these details. You can create multiple configuration files on the same system, so that different users can save different camera options, etc on the same computer. To load previous settings from a configuration file, please just select the file from the drop-down menu. Configuration files are stored at `$HOME/Documents/DeepLabCut-live-GUI/config`. These files do not need to be edited manually, they can be entirely created and edited automatically within the GUI.
10532

106-
1. **Launch the GUI**:
107-
```bash
108-
dlclivegui
109-
```
33+
#### Set Up Cameras <img src= https://imagizer.imageshack.com/img924/626/acJhWD.png align="right">
11034

111-
2. **Select Camera Backend**: Choose from the dropdown (opencv, gentl, aravis, basler)
35+
To setup a new camera, select `Add Camera` from the dropdown menu, and then click `Init Cam`. This will be bring up a new window where you need to select the type of camera (see [Camera Support](docs/camera_support.md)), input a name for the camera, and click `Add Camera`. This will initialize a new `Camera` entry in the drop down menu. Now, select your camera from the dropdown menu and click`Edit Camera Settings` to setup your camera settings (i.e. set the serial number, exposure, cropping parameters, etc; the exact settings depend on the specific type of camera). Once you have set the camera settings, click `Init Cam` to start streaming. To stop streaming data, click `Close Camera`, and to remove a camera from the dropdown menu, click `Remove Camera`.
11236

113-
3. **Configure Camera**: Set FPS, exposure, gain, and other parameters
37+
#### Processor (optional)
11438

115-
4. **Start Preview**: Click "Start Preview" to begin camera streaming
39+
To write custom `Processors`, please see [here](https://github.com/DeepLabCut/DeepLabCut-live/tree/master/dlclive/processor). The directory that contains your custom `Processor` should be a python module -- this directory must contain an `__init__.py` file that imports your custom `Processor`. For examples of how to structure a custom `Processor` directory, please see [here](https://github.com/DeepLabCut/DeepLabCut-live/tree/master/example_processors).
11640

117-
5. **Optional - Load DLC Model**: Browse to your exported DLCLive model directory
41+
To use your processor in the GUI, you must first add your custom `Processor` directory to the dropdown menu: next to the `Processor Dir` label, click `Browse`, and select your custom `Processor` directory. Next, select the desired directory from the `Processor Dir` dropdown menu, then select the `Processor` you would like to use from the `Processor` menu. If you would like to edit the arguments for your processor, please select `Edit Proc Settings`, and finally, to use the processor, click `Set Proc`. If you have previously set a `Processor` and would like to clear it, click `Clear Proc`.
11842

119-
6. **Optional - Start Inference**: Click "Start pose inference" for real-time tracking
43+
#### Configure DeepLabCut Network
12044

121-
7. **Optional - Record Video**: Configure output path and click "Start recording"
45+
<img src= https://imagizer.imageshack.com/img923/9730/MNzr1J.png align="right">
12246

123-
## Configuration
47+
Select the `DeepLabCut` dropdown menu, and click `Add DLC`. This will bring up a new window to choose a name for the DeepLabCut configuration, choose the path to the exported DeepLabCut model, and set DeepLabCut-live settings, such as the cropping or resize parameters. Once configured, click `Update` to add this DeepLabCut configuration to the dropdown menu. You can edit the settings at any time by clicking `Edit DLC Settings`. Once configured, you can load the network and start performing inference by clicking `Start DLC`. If you would like to view the DeepLabCut pose estimation in real-time, select `Display DLC Keypoints`. You can edit the keypoint display settings (the color scheme, size of points, and the likelihood threshold for display) by selecting `Edit DLC Display Settings`.
12448

125-
The GUI uses a single JSON configuration file containing all experiment settings:
126-
127-
```json
128-
{
129-
"camera": {
130-
"name": "Camera 0",
131-
"index": 0,
132-
"fps": 60.0,
133-
"backend": "gentl",
134-
"exposure": 10000,
135-
"gain": 5.0,
136-
"crop_x0": 0,
137-
"crop_y0": 0,
138-
"crop_x1": 0,
139-
"crop_y1": 0,
140-
"max_devices": 3,
141-
"properties": {}
142-
},
143-
"dlc": {
144-
"model_path": "/path/to/exported-model",
145-
"model_type": "pytorch",
146-
},
147-
"recording": {
148-
"enabled": true,
149-
"directory": "~/Videos/deeplabcut-live",
150-
"filename": "session.mp4",
151-
"container": "mp4",
152-
"codec": "h264_nvenc",
153-
"crf": 23
154-
},
155-
"bbox": {
156-
"enabled": false,
157-
"x0": 0,
158-
"y0": 0,
159-
"x1": 200,
160-
"y1": 100
161-
}
162-
}
163-
```
164-
165-
### Configuration Management
166-
167-
- **Load**: File → Load configuration… (or Ctrl+O)
168-
- **Save**: File → Save configuration (or Ctrl+S)
169-
- **Save As**: File → Save configuration as… (or Ctrl+Shift+S)
170-
171-
All GUI fields are automatically synchronized with the configuration file.
172-
173-
## Camera Backends
174-
175-
### Backend Selection Guide
176-
177-
| Backend | Platform | Use Case | Auto-Detection |
178-
|---------|----------|----------|----------------|
179-
| **opencv** | All | Webcams, simple USB cameras | Basic |
180-
| **gentl** | Windows, Linux | Industrial cameras via CTI files | Yes |
181-
| **aravis** | Linux, macOS | GenICam/GigE cameras | Yes |
182-
| **basler** | All | Basler cameras specifically | Yes |
183-
184-
### Backend-Specific Configuration
185-
186-
#### OpenCV
187-
```json
188-
{
189-
"camera": {
190-
"backend": "opencv",
191-
"index": 0,
192-
"fps": 30.0
193-
}
194-
}
195-
```
196-
**Note**: Exposure and gain controls are disabled for OpenCV backend due to limited driver support.
197-
198-
#### GenTL (Harvesters)
199-
```json
200-
{
201-
"camera": {
202-
"backend": "gentl",
203-
"index": 0,
204-
"fps": 60.0,
205-
"exposure": 15000,
206-
"gain": 8.0,
207-
}
208-
}
209-
```
49+
If you want to stop performing inference at any time, just click `Stop DLC`, and if you want to remove a DeepLabCut configuration from the dropdown menu, click `Remove DLC`.
21050

51+
#### Set Up Session
21152

212-
See [Camera Backend Documentation](docs/camera_support.md) for detailed setup instructions.
53+
Sessions are defined by the subject name, the date, and an attempt number. Within the GUI, select a `Subject` from the dropdown menu, or to add a new subject, type the new subject name in to the entry box and click `Add Subject`. Next, select an `Attempt` from the dropdown menu. Then, select the directory that you would like to save data to from the `Directory` dropdown menu. To add a new directory to the dropdown menu, click `Browse`. Finally, click `Set Up Session` to initiate a new recording. This will prepare the GUI to save data. Once you click `Set Up Session`, the `Ready` button should turn blue, indicating a session is ready to record.
21354

214-
## DLCLive Integration
55+
#### Controlling Recording
21556

216-
### Model Types
57+
If the `Ready` button is selected, you can now start a recording by clicking `On`. The `On` button will then turn green indicating a recording is active. To stop a recording, click `Off`. This will cause the `Ready` button to be selected again, as the GUI is prepared to restart the recording and to save the data to the same file. If you're session is complete, click `Save Video` to save all files: the video recording (as .avi file), a numpy file with timestamps for each recorded frame, the DeepLabCut poses as a pandas data frame (hdf5 file) that includes the time of each frame used for pose estimation and the time that each pose was obtained, and if applicable, files saved by the `Processor` in use. These files will be saved into a new directory at `{YOUR_SAVE_DIRECTORY}/{CAMERA NAME}_{SUBJECT}_{DATE}_{ATTEMPT}`
21758

218-
The GUI supports PyTorch DLCLive models:
59+
- YOUR_SAVE_DIRECTORY : the directory chosen from the `Directory` dropdown menu.
60+
- CAMERA NAME : the name of selected camera (from the `Camera` dropdown menu).
61+
- SUBJECT : the subject chosen from the `Subject` drowdown menu.
62+
- DATE : the current date of the experiment.
63+
- ATTEMPT : the attempt number chosen from the `Attempt` dropdown.
21964

220-
1. **PyTorch**: PyTorch-based models (requires PyTorch installation)
65+
If you would not like to save the data from the session, please click `Delete Video`, and all data will be discarded. After you click `Save Video` or `Delete Video`, the `Off` button will be selected, indicating you can now set up a new session.
22166

222-
Select the model type from the dropdown before starting inference.
67+
#### References:
22368

224-
### Processor System
225-
226-
The GUI includes a plugin system for custom pose processing:
227-
228-
```python
229-
# Example processor
230-
class MyProcessor:
231-
def process(self, pose, timestamp):
232-
# Custom processing logic
233-
x, y = pose[0, :2] # First keypoint
234-
print(f"Position: ({x}, {y})")
235-
def save(self):
236-
pass
237-
```
238-
239-
Place processors in `dlclivegui/processors/` and refresh to load them.
240-
241-
See [Processor Plugin Documentation](docs/PLUGIN_SYSTEM.md) for details.
242-
243-
### Auto-Recording Feature
244-
245-
Enable "Auto-record video on processor command" to automatically start/stop recording based on processor signals. Useful for event-triggered recording in behavioral experiments.
246-
247-
## Performance Optimization
248-
249-
### High-Speed Camera Tips
250-
251-
1. **Use Hardware Encoding**: Select `h264_nvenc` codec for NVIDIA GPUs
252-
2. **Adjust Buffer Count**: Increase buffers for GenTL/Aravis backends
253-
```json
254-
"properties": {"n_buffers": 20}
255-
```
256-
3. **Optimize CRF**: Lower CRF = higher quality but larger files (default: 23)
257-
4. **Disable Visualization**: Uncheck "Display pose predictions" during recording
258-
5. **Crop Region**: Use cropping to reduce frame size before inference
259-
260-
### Project Structure
261-
262-
```
263-
dlclivegui/
264-
├── __init__.py
265-
├── gui.py # Main PySide6 application
266-
├── config.py # Configuration dataclasses
267-
├── camera_controller.py # Camera capture thread
268-
├── dlc_processor.py # DLCLive inference thread
269-
├── video_recorder.py # Video encoding thread
270-
├── cameras/ # Camera backend modules
271-
│ ├── base.py # Abstract base class
272-
│ ├── factory.py # Backend registry and detection
273-
│ ├── opencv_backend.py
274-
│ ├── gentl_backend.py
275-
│ ├── aravis_backend.py
276-
│ └── basler_backend.py
277-
└── processors/ # Pose processor plugins
278-
├── processor_utils.py
279-
└── dlc_processor_socket.py
280-
```
281-
282-
283-
## Documentation
284-
285-
- [Camera Support](docs/camera_support.md) - All camera backends and setup
286-
- [Aravis Backend](docs/aravis_backend.md) - GenICam camera setup (Linux/macOS)
287-
- [Processor Plugins](docs/PLUGIN_SYSTEM.md) - Custom pose processing
288-
- [Installation Guide](docs/install.md) - Detailed setup instructions
289-
- [Timestamp Format](docs/timestamp_format.md) - Timestamp synchronization
290-
291-
## System Requirements
292-
293-
294-
### Recommended
295-
- Python 3.10+
296-
- 8 GB RAM
297-
- NVIDIA GPU with CUDA support (for DLCLive inference and video encoding)
298-
- USB 3.0 or GigE network (for industrial cameras)
299-
- SSD storage (for high-speed recording)
300-
301-
### Tested Platforms
302-
- Windows 11
303-
304-
## License
305-
306-
This project is licensed under the GNU Lesser General Public License v3.0. See the [LICENSE](LICENSE) file for more information.
307-
308-
## Citation
309-
310-
Cite the original DeepLabCut-live paper:
311-
```bibtex
312-
@article{Kane2020,
313-
title={Real-time, low-latency closed-loop feedback using markerless posture tracking},
314-
author={Kane, Gary A and Lopes, Gonçalo and Saunders, Jonny L and Mathis, Alexander and Mathis, Mackenzie W},
315-
journal={eLife},
316-
year={2020},
317-
doi={10.7554/eLife.61909}
318-
}
319-
```
69+
If you use this code we kindly ask you to you please [cite Kane et al, eLife 2020](https://elifesciences.org/articles/61909). The preprint is available here: https://www.biorxiv.org/content/10.1101/2020.08.04.236422v2

0 commit comments

Comments
 (0)