README.md

This code is adapted from HOC-Search for generating views and instance segmentations for individual objects.

[Dislaimer] We don't maintain code in this subdirectory.

---

Setup

---

0. cd into scannotate_preprocessing directory, assuming the current directory is the root of this repository :

cd scannotate_preprocessing

1. We suggest creating a new environment:

conda env create -f environment.yml
conda activate hoc_preprocessing

2. Install SAM

pip install git+https://github.com/facebookresearch/segment-anything.git

3. Download SAM weights:

• default or vit_h: ViT-H SAM model.
• vit_l: ViT-L SAM model.
• vit_b: ViT-B SAM model.

Extracting Views and Masks

---

0. Make sure you are in the scannotate_preprocessing directory.

1. Modify config script config/0_Scannotate_masks.ini and adapt paths.

SCANNOTATE_PATH should point to the base of SCANnotate dataset. out_folder is the folder containing SCANnotate annotations and where instance masks will be extracted. Structure should be as follows:

├── <SCANNOTATE_PATH>
│   ├── annotations
│   │    ├── <scene_name>
│   │    │    ├── <scene_name>.pkl
│   │    ├── ... 

SCANNET_base_path is the base path where ScanNet scenes are located with structure:

├── <SCANNET_base_path>
│   ├── <scene_name>
│   │    ├── color
│   │    ├── depth
│   │    ├── intrinsic
│   │    ├── pose
│   │    ├── <scene_name>.pkl
│   │    ├── ...
│   ├── ...

---

2. Render all 2D masks from 3D:

python ScanNet_renderer/Render_Scannotate_masks.py

Masks will be saved to SCANNOTATE_PATH:

├── <SCANNOTATE_PATH>
│   ├── annotations
│   │    ├── <scene_name>
│   │    │    ├── <scene_name>.pkl
│   │    │    ├── all_inst_seg_2d
│   │    │    ├── mask2d_from_3d
│   │    ├── ... 

---

3. Finally, calculate SAM masks for each object:

python ScanNet_renderer/Scannotate_2d_mask_predictor.py

Masks will be saved to SCANNOTATE_PATH:

├── <SCANNOTATE_PATH>
│   ├── annotations
│   │    ├── <scene_name>
│   │    │    ├── <scene_name>.pkl
│   │    │    ├── all_inst_seg_2d
│   │    │    ├── mask2d_from_3d
│   │    │    ├── sam_results_path
│   │    │    ├── valid_maps
│   │    ├── ...