Add nuScenes data processing into codebase (#6)

* conf(nus): update ground segmentation in nuscenes.

* conf(nus): update ground segmentation in nuscenes.

* feat(data): core extraction py file for nuscenes.

* Add NusMap to Argoverse 2 for consistent evaluation afterward.

* style(model): update a basemodel to avoid repeating the same fn, etc.

* will nus training need to update the dataloader, it will involved by DeltaFlow merge there.

* docs: fix typo on dataprocess README. tested successfully on all fn.

* Add flow check in nus from supervised training later.

---------

Co-authored-by: EdwardLeeLPZ <EdwardLeeLPZ@users.noreply.github.com>

Author: Kin-Zhang

CONTRIBUTING.md

@@ -1,10 +1,10 @@
-# Contributing to OpenSceneFlow
+# Contributing to [OpenSceneFlow](https://github.com/KTH-RPL/OpenSceneFlow)
 
 We want to make contributing to this project as easy and transparent as possible. We welcome any contributions, from bug fixes to new features. If you're interested in adding your own scene flow method, this guide will walk you through the process.
 
 ## Adding a New Method
 
-Here is a quick guide to integrating a new method into the OpenSceneFlow codebase.
+Here is a quick guide to integrating a new method into the [OpenSceneFlow](https://github.com/KTH-RPL/OpenSceneFlow) codebase.
 
 ### 1. Data Preparation
 

README.md

@@ -9,7 +9,7 @@
 💞 If you find [*OpenSceneFlow*](https://github.com/KTH-RPL/OpenSceneFlow) useful to your research, please cite [**our works** 📖](#cite-us) and [give a star 🌟](https://github.com/KTH-RPL/OpenSceneFlow) as encouragement. (੭ˊ꒳​ˋ)੭✧
 
 OpenSceneFlow is a codebase for point cloud scene flow estimation. 
-It is also an official implementation of the following papers (sored by the time of publication):
+It is also an official implementation of the following papers (sorted by the time of publication):
 
 - **HiMo: High-Speed Objects Motion Compensation in Point Clouds** (SeFlow++)   
 *Qingwen Zhang, Ajinkya Khoche, Yi Yang, Li Ling, Sina Sharif Mansouri, Olov Andersson, Patric Jensfelt*  
@@ -101,7 +101,7 @@ If you prefer to build the Docker image by yourself, Check [build-docker-image](
 
 ## 1. Data Preparation
 
-Refer to [dataprocess/README.md](dataprocess/README.md) for dataset download instructions. Currently, we support **Argoverse 2**, **Waymo**, and **custom datasets** (more datasets will be added in the future). 
+Refer to [dataprocess/README.md](dataprocess/README.md) for dataset download instructions. Currently, we support **Argoverse 2**, **Waymo**, **nuScenes** and **custom datasets** (more datasets will be added in the future). 
 
 After downloading, convert the raw data to `.h5` format for easy training, evaluation, and visualization. Follow the steps in [dataprocess/README.md#process](dataprocess/README.md#process). 
 

assets/docs/nuscenes.png

@@ -0,0 +1,31 @@
+
+[important]
+height = 1.85        # sensor height. 雷达传感器高度，主要是+offset
+
+# 整个雷点点云以自己为中心 分为多少个segment，每个segment又分成多少个bin
+[segments]
+r_min = 0.1          # minimum point distance. 感兴趣的区域
+r_max = 80           # maximum point distance.
+n_segments = 360     # number of radial segments.
+n_bins = 160         # number of radial bins.
+
+
+[ground]
+# 直线的 slope 斜率，一个seg里拟合的线斜率
+min_slope = 0.0      # minimum slope of a ground line. [T_m_small]
+max_slope = 0.1     # maximum slope of a ground line. [T_m]
+# 如果拟合点超过这个 long threshold，判断其 height 变化是否超过 max_long_height
+long_threshold = 2.0    # Distance at which points are considered far from each other. [T_d_prev]
+max_long_height = 0.3   # maximum height change to previous point in long line. [T_b]
+# 如果没有超过 long threshold，判断现在拟合的点与fitline高度变化是否超过 max_start_height
+max_start_height = 0.2  # Maximum heigh of starting line to be labelled ground.
+# 最大的拟合误差，超过这个误差的点不会被加入 line 拟合
+max_fit_error = 0.1  # maximum error of a point during line fit. [T_RMSE: will sqaure inside code.]
+
+# 判断是否是地面点的时候，需要满足的条件：
+max_dist_to_line = 0.2   # maximum vertical distance of point to line to be considered ground. [T_d_ground]
+line_search_angle = 2.0 # How far to search for a line in angular direction [rad].
+
+[general]
+n_threads = 8        # number of threads for parallel processing.
+verbose = false       # if you don't want to see every output, set this to false.

conf/others/nuscenes.toml

@@ -10,7 +10,7 @@ We've updated the process dataset for:
 
 - [x] Argoverse 2.0: check [here](#argoverse-20). The process script Involved from [DeFlow](https://github.com/KTH-RPL/DeFlow).
 - [x] Waymo: check [here](#waymo-dataset). The process script was involved from [SeFlow](https://github.com/KTH-RPL/SeFlow).
-- [ ] nuScenes: done coding, public after review. Will be involved later by another paper.
+- [x] nuScenes: check [here](#nuscenes), The process script was involved from [DeltaFlow](https://github.com/Kin-Zhang/DeltaFlow).
 - [ ] TruckScene: done coding, public after review. Will be involved later by another paper.
 - [ ] ZOD (w/o gt): done coding, public after review. Will be involved later by another paper.
 
@@ -77,6 +77,20 @@ You need sign up an account at [nuScenes](https://www.nuscenes.org/) to download
 ![](../assets/docs/nuscenes.png)
 
 
+Extracting & processing nuScenes require special handling:
+
+* Frame Rate: The raw LiDAR data is captured at 20Hz, while ground truth (GT) annotations are only available at 2Hz.
+* Resampling: To standardize the data for consistent evaluation, we downsample the LiDAR point clouds to 10Hz. It is a GT-guided process that guarantees all annotated 2Hz frames are preserved within the final 10Hz sequence.
+* The ground truth scene flow is generated using the official per-object velocity labels provided in the dataset, calculated between the resampled 10Hz frames.
+
+
+#### Dataset frames
+
+| Dataset | # Total Scene | # Total Frames |
+| ------- | ------------- | -------------- |
+| train   | 700           | 137575 / 27392 (w. gt)         |
+| val     | 150           | 29126 / 5798 (w.gt)          |
+
 ### Waymo Dataset
 
 To download the Waymo dataset, you need to register an account at [Waymo Open Dataset](https://waymo.com/open/). You also need to install gcloud SDK and authenticate your account. Please refer to [this page](https://cloud.google.com/sdk/docs/install) for more details. 
@@ -123,6 +137,9 @@ python dataprocess/extract_av2.py --av2_type sensor --data_mode train --argo_dir
 
 # waymo:
 python dataprocess/extract_waymo.py --mode train --flow_data_dir /home/kin/data/waymo/flowlabel --map_dir /home/kin/data/waymo/flowlabel/map --output_dir /home/kin/data/waymo/preprocess  --nproc 48
+
+# nus:
+python dataprocess/extract_nus.py --mode v1.0-trainval --output_dir /home/kin/data/nus/h5py/full --nproc 24
 ```