|
1 | 1 | # Cheese3D |
2 | 2 |
|
| 3 | +[](https://hou-lab-cshl.github.io/cheese3d/) |
| 4 | + |
3 | 5 | Cheese3D is a pipeline for tracking mouse facial movements built on top of existing tools like [DeepLabCut](https://github.com/DeepLabCut/DeepLabCut) and [Anipose](https://github.com/lambdaloop/anipose). By tracking anatomically-informed keypoints using multiple cameras registered in 3D, our pipeline produces sensitive, high-precision facial movement data that can be related internal state (e.g., electrophysiology). |
4 | 6 |
|
5 | 7 | <p align="center"> |
6 | 8 | <img src="docs/source/_static/Cheese3D.gif" alt="Animation of Cheese3D pipeline", width=60%> |
7 | 9 | <img src="docs/source/_static/Cheese3DIcon.png" alt="Animation of Cheese3D pipeline", width=34%> |
8 | 10 | </p> |
9 | 11 |
|
| 12 | +Cheese3D output can be visualized interactively. |
| 13 | + |
| 14 | +<p align="center"> |
| 15 | + <img src="docs/source/_static/Cheese3DVisualizer.gif" alt="Animation of Cheese3D visualizer", width=50%> |
| 16 | + <img src="docs/source/_static/Cheese3DVisualizerStatic.png" alt="Animation of Cheese3D visualizer", width=50.2%> |
| 17 | +</p> |
| 18 | + |
10 | 19 | Using a combination of hardware synchronization signals and a multi-stage pipeline, we are able to precisely synchronize video and electrophysiology data. This allows us to relate spikes recorded in the brainstem to various facial movements (here, we highlight two example units correlated with ipsilateral ear movements). |
11 | 20 |
|
12 | | - |
| 21 | + |
| 22 | + |
| 23 | + |
13 | 24 |
|
14 | | -<!-- If you use Cheese3D, please cite our preprint: |
| 25 | + If you use Cheese3D, please cite our preprint: |
15 | 26 | ``` |
16 | 27 | @article {Daruwalla2024.05.07.593051, |
17 | | - author = {Daruwalla, Kyle and Martin, Irene Nozal and Frankel, Andrew and Nagli{\v c}, Diana and Ahmad, Zainab and Hou, Xun Helen}, |
18 | | - title = {A 3D whole-face movement analysis system to uncover underlying physiology in mice}, |
19 | | - elocation-id = {2024.05.07.593051}, |
20 | | - year = {2024}, |
21 | | - doi = {10.1101/2024.05.07.593051}, |
22 | | - publisher = {Cold Spring Harbor Laboratory}, |
23 | | - URL = {https://www.biorxiv.org/content/early/2024/05/08/2024.05.07.593051}, |
24 | | - eprint = {https://www.biorxiv.org/content/early/2024/05/08/2024.05.07.593051.full.pdf}, |
25 | | - journal = {bioRxiv} |
| 28 | + author = {Daruwalla, Kyle and Martin, Irene Nozal and Zhang, Linghua and Nagli{\v c}, Diana and Frankel, Andrew and Rasgaitis, Catherine and Zhang, Xinyan and Ahmad, Zainab and Borniger, Jeremy C. and Hou, Xun Helen}, |
| 29 | + title = {Cheese3D: Sensitive Detection and Analysis of Whole-Face Movement in Mice}, |
| 30 | + elocation-id = {2024.05.07.593051}, |
| 31 | + year = {2025}, |
| 32 | + doi = {10.1101/2024.05.07.593051}, |
| 33 | + publisher = {Cold Spring Harbor Laboratory}, |
| 34 | + URL = {https://www.biorxiv.org/content/early/2025/03/01/2024.05.07.593051}, |
| 35 | + eprint = {https://www.biorxiv.org/content/early/2025/03/01/2024.05.07.593051.full.pdf}, |
| 36 | + journal = {bioRxiv} |
26 | 37 | } |
27 | | -``` --> |
28 | | - |
29 | | -## Using Cheese3D |
30 | | - |
31 | | -Follow the documentation by running: |
32 | | - |
33 | | -1. Install [pixi](https://pixi.sh/latest/#installation) |
34 | | -2. Clone or download this repository |
35 | | - (`git clone https://github.com/Hou-Lab-CSHL/cheese3d`) |
36 | | -3. Change directories to the repository folder |
37 | | - (`cd <path to repo>/cheese3d`) |
38 | | -5. Open documentation (`pixi run docs-serve`) |
39 | | - |
40 | | -## Reproducing Cheese3D paper figures |
41 | | - |
42 | | -The following notebooks contain the code required to reproduce the figures in our paper. They also serve as a showcase of the type of analysis enabled by Cheese3D's output. You can find the complete collection under the `paper/` directory. |
43 | | - |
44 | | -| Example figure panel | Notebook | Description | |
45 | | -|:--------------------:|:---------|:------------| |
46 | | -| <img src="paper/Fig1Example.png" width=200> | [`paper/fig1-cheese3d-accuracy.ipynb`](paper/fig1-cheese3d-accuracy.ipynb) | Framework and validation of capturing face-wide movement as 3D geometric features in mice | |
47 | | -| <img src="paper/Fig2Example.png" width=200> | [`paper/fig2-cheese3d-jitter-analysis.ipynb`](paper/fig2-cheese3d-jitter-analysis.ipynb) | Reduction in keypoint tracking jitter due to 3D triangulation of data from six camera views | |
48 | | -| <img src="paper/Fig3Example.png" width=200> | [`paper/fig3-cheese3d-general-anesthesia-eeg.ipynb`](paper/fig3-cheese3d-general-anesthesia-eeg.ipynb) | Distinct facial patterns track time during induction and recovery from ketamine-induced anesthesia | |
49 | | -| <img src="paper/Fig4Example.png" width=200> | [`paper/fig4-cheese3d-chewing-whole-face-kinematics.ipynb`](paper/fig4-cheese3d-chewing-whole-face-kinematics.ipynb) | Chewing kinematics in mouth and surrounding facial areas | |
50 | | -| <img src="paper/Fig5-1Example.png" width=200> | [`paper/fig5-part1-cheese3d-stimulation-triggered-movement.ipynb`](paper/fig5-part1-cheese3d-stimulation-triggered-movement.ipynb) | Stimulation triggered facial movements in anesthetized mice | |
51 | | -| <img src="paper/Fig5-2Example.png" width=200> | [`paper/fig5-part2-cheese3d-synchronized-electrophysiology.ipynb`](paper/fig5-part2-cheese3d-synchronized-electrophysiology.ipynb) | Synchronized Cheese3D with electrophysiology relates motor control activity to subtle facial movements | |
| 38 | +``` |
0 commit comments