Results on selval dataset of KITTI

[adityat4aa]

Hi, I am getting RMSE of 0.7176936 meters.

Did you also achieve the same result?

[Sharpiless]

Yes, my result was 717.5 on RMSE and 194.2 on MAE, tested on a single 4090. That’s quite close.

[adityat4aa]

I used the following script to train my own model -

torchrun --nproc_per_node=1 --master_port 4321 train_distill.py
gpus=[0] num_workers=32 name=DMD3D_FS_KITTI
++chpt=checkpoints/pretrained_mixed_singleview_256.pth
net=PMP data=KITTI
lr=5e-4 train_batch_size=2 test_batch_size=1
sched/lr=NoiseOneCycleCosMo sched.lr.policy.max_momentum=0.90
nepoch=50 test_epoch=45 ++net.sbn=true

But the best emea recorded was 0.9606 meters.

@Sharpiless Could you suggest how I could improve this metric to match your best metric of 0.717 meters of the model dmd3c_distillation_depth_anything_v2.pth on the sel val kitti dataset ?

Any leads would be much appreciated!

[Sharpiless]

Hi, would you like to share the code to save the dpeth map? I can check it for you.

[adityat4aa]

Ok here is the command that I am using to save the outputs

Command -

python test.py gpus="[0]"
name=test_kitti_FS =PMP data=KITTI
+data.data_root=/mnt/mnt_azure/KITTI/all/KITTI
+data.mode=selval +chpt=/home/g7pmrt8/Thesis/BP-Net/checkpoints/DMD3D_FS_KITTI/result_ema.pth
+save=true test_batch_size=1 +net.sbn=true

test.txt

.py files are not allowed hence I uploaded test.txt

Is there anything else I need to upload?

[Sharpiless]

What I meant was the code you used for generating the pseudo-depth labels with depth-anything during inference. There might be a scale issue (for example, whether the depth values were normalized). If it's not urgent, I can check the inference script I used with depth-anything tomorrow when I'm at my office computer.

[Sharpiless]

For fast validation, you can fine-tune the pre-trained BP-Net (kitti) from its official implementation with our distillation pipeline.

[adityat4aa]

@Sharpiless I have a quick question regarding the psuedo depth maps.

what images should be kept in the path - data_depth_annotated/train/proj_depth/groundtruth/image_02/*.png ?

Should I first run depthanything on all kitti images and then save the depth maps created by DepthAnything in the above mentioned path?

because the kitti depth completion dataset already contains the folder data_depth_annotated

[Sharpiless]

I'm a bit confused about your training pipeline. However, based on the code snippet self.depths = list(sorted(glob.iglob(self.depth_path + "/*.png", recursive=True))), it's loading the depth maps generated by Depth-Anything. You might want to double-check that to be sure. The depth map provided by kitti is sparse and should not be computed for our distillation loss, as the code did not filter the zero (invalid) values.
Let me know if I misunderstood your setup!

[adityat4aa]

Oh okay! Actually I did not run Depthanything on the raw images.

Am I supposed to do that?

[Sharpiless]

Oh I see — sorry about the confusion! My README was written a bit hastily, so I understand it might have been unclear. I'll update it once I get through my current papers and work.

Regarding the use of Depth-Anything, our paper includes a pipeline diagram that outlines the process — you can refer to that for guidance. In general, for KITTI-RAW images, you’re expected to save the corresponding Depth-Anything predictions.

[adityat4aa]

Oh okay! That clears the confusion. So basically, the path data_depth_annotated/train/proj_depth/groundtruth/image_02/*.png should contain the depth maps created by Depth-Anything instead of the sparse depth maps which KITTI gives us.

Then I should do the training. Is that correct?

[Sharpiless]

Yes, that’s right — you should save the Depth-Anything predicted depth maps corresponding to each raw image. Instead of placing them under data_depth_annotated/..., the depth maps should be stored directly under the corresponding raw path.

For reference, here's the path structure used in the code:

rgb_path = os.path.join(*file_names[:-7], 'raw', file_names[-5].split('_drive')[0], file_names[-5], 
                        file_names[-2], 'data', file_names[-1])
disp_path = os.path.join(*file_names[:-7], 'raw', file_names[-5].split('_drive')[0], file_names[-5],
                         file_names[-2], 'disp', file_names[-1])

Just make sure your Depth-Anything outputs go to the correct disp folder matching each image.

Also, sorry — I misspoke the code position here: #13 (comment). Please ignore that sentence (my brain was a bit fried after a long day of work 😅).

[adityat4aa]

Thanks for the reply! I just have a couple of questions for you to clarify everything.

1 - disp_path = os.path.join(*file_names[:-7], 'raw', file_names[-5].split('_drive')[0], file_names[-5],
file_names[-2], 'disp', file_names[-1])

What does disp_path exactly contain? The disparities or the depth_maps generated by DepthAnything?

2. What should be the format of the depth-maps? Grayscale or RGB?

3. This is what you have written in your Repo -

Runing monocular depth estimation for all KITTI-raw images. Data structure:

├── datas/kitti/raw/
│ ├── 2011_09_26
│ │ ├── 2011_09_26_drive_0001_sync
│ │ │ ├── image_02
│ │ │ │ ├── data/.png
│ │ │ │ ├── disp/.png
│ │ │ ├── image_03
│ │ ├── 2011_09_26_drive_0002_sync.......

Where disparity images are stored in gray-scale.

Does the folder disp actually contain the depth maps and not the disparities?

[Sharpiless]

The data structure you mentioned is correct. But, it’s important to note that although Depth-Anything is a monocular depth estimation model (and its name includes “Depth”), its output is actually disparity, not absolute depth. So in our pipeline, what we store in the disp/ folder is the raw output of Depth-Anything — which is disparity.

[adityat4aa]

Ok thanks, so in which folder do you save the pseudo depth maps after calculating the disparity?