Skip to content

Commit 3a10c77

Browse files
committed
docs(README): update teflow, dogflow papers, wait for the merge.
hotfix(truckscene): bbx overlap on the same points, select the max one for assign flows.
1 parent d5ad83a commit 3a10c77

File tree

2 files changed

+50
-21
lines changed

2 files changed

+50
-21
lines changed

README.md

Lines changed: 28 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,10 @@
1111
OpenSceneFlow is a codebase for point cloud scene flow estimation.
1212
It is also an official implementation of the following papers (sorted by the time of publication):
1313

14-
<!-- - **TeFlow: An Efficient Multi-frame Scene Flow Estimation Method**
14+
- **TeFlow: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation**
1515
*Qingwen Zhang, Chenhan Jiang, Xiaomeng Zhu, Yunqi Miao, Yushan Zhang, Olov Andersson, Patric Jensfelt*
16-
Under Review
17-
[ Strategy ] [ Self-Supervised ] - [ [OpenReview](https://openreview.net/forum?id=h70FLgnIAw) ] [ [Project](https://github.com/Kin-Zhang/TeFlow) ]&rarr; [here](#teflow) -->
16+
Conference on Computer Vision and Pattern Recognition (**CVPR**) 2026
17+
[ Strategy ] [ Self-Supervised ] - [ [arXiv](https://arxiv.org/abs/2602.19053) ] [ [Project]() ]
1818

1919
- **DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method**
2020
*Qingwen Zhang, Xiaomeng Zhu, Yushan Zhang, Yixi Cai, Olov Andersson, Patric Jensfelt*
@@ -26,6 +26,11 @@ Conference on Neural Information Processing Systems (**NeurIPS**) 2025 - Spotlig
2626
IEEE Transactions on Robotics (**T-RO**) 2025
2727
[ Strategy ] [ Self-Supervised ] - [ [arXiv](https://arxiv.org/abs/2503.00803) ] [ [Project](https://kin-zhang.github.io/HiMo/) ] &rarr; [here](#seflow-1)
2828

29+
- **DoGFlow: Self-Supervised LiDAR Scene Flow via Cross-Modal Doppler Guidance**
30+
*Ajinkya Khoche, Qingwen Zhang, Yixi Cai, Sina Sharif Mansouri and Patric Jensfelt*
31+
IEEE Robotics and Automation Letters (**RA-L**) 2026
32+
[ Multi-Model ] [ Self-Supervised ] - [ [arXiv](https://arxiv.org/abs/2508.18506) ] [ [Project](https://ajinkyakhoche.github.io/DogFlow/) ]&rarr; [here](https://github.com/ajinkyakhoche/DoGFlow)
33+
2934
- **VoteFlow: Enforcing Local Rigidity in Self-Supervised Scene Flow**
3035
*Yancong Lin\*, Shiming Wang\*, Liangliang Nan, Julian Kooij, Holger Caesar*
3136
Conference on Computer Vision and Pattern Recognition (**CVPR**) 2025
@@ -46,7 +51,6 @@ International Conference on Robotics and Automation (**ICRA**) 2025
4651
European Conference on Computer Vision (**ECCV**) 2024
4752
[ Strategy ] [ Self-Supervised ] - [ [arXiv](https://arxiv.org/abs/2407.01702) ] [ [Project](https://github.com/KTH-RPL/SeFlow) ] &rarr; [here](#seflow)
4853

49-
5054
- **DeFlow: Decoder of Scene Flow Network in Autonomous Driving**
5155
*Qingwen Zhang, Yi Yang, Heng Fang, Ruoyu Geng, Patric Jensfelt*
5256
International Conference on Robotics and Automation (**ICRA**) 2024
@@ -62,6 +66,7 @@ Additionally, *OpenSceneFlow* integrates following excellent works: [ICLR'24 Zer
6266
- [x] [NSFP](https://arxiv.org/abs/2111.01253): NeurIPS 2021, faster 3x than original version because of [our CUDA speed up](assets/cuda/README.md), same (slightly better) performance.
6367
- [x] [FastNSF](https://arxiv.org/abs/2304.09121): ICCV 2023. SSL Optimization-based.
6468
- [x] [ICP-Flow](https://arxiv.org/abs/2402.17351): CVPR 2024. SSL Optimization-based.
69+
- [ ] [Floxels](https://arxiv.org/abs/2503.04718): CVPR 2025. SSL optimization-based. coding now but not yet ready for release as lower performance than reported. check [branch code](https://github.com/Kin-Zhang/OpenSceneFlow/tree/feature/floxels) for more details.
6570
- [ ] [EulerFlow](https://arxiv.org/abs/2410.02031): ICLR 2025. SSL optimization-based. In my plan, haven't coding yet.
6671

6772
</details>
@@ -144,7 +149,7 @@ Train DeltaFlow with the leaderboard submit config. [Runtime: Around 18 hours in
144149

145150
```bash
146151
# total bz then it's 10x2 under above training setup.
147-
python train.py model=deltaFlow optimizer.lr=2e-3 epochs=20 batch_size=2 num_frames=5 loss_fn=deflowLoss train_aug=True "voxel_size=[0.15, 0.15, 0.15]" "point_cloud_range=[-38.4, -38.4, -3.2, 38.4, 38.4, 3.2]" +optimizer.scheduler.name=WarmupCosLR +optimizer.scheduler.max_lr=2e-3 +optimizer.scheduler.total_steps=20000
152+
python train.py model=deltaFlow optimizer.lr=2e-3 epochs=20 batch_size=2 num_frames=5 loss_fn=deflowLoss train_aug=True "voxel_size=[0.15, 0.15, 0.15]" "point_cloud_range=[-38.4, -38.4, -3, 38.4, 38.4, 3]" +optimizer.scheduler.name=WarmupCosLR +optimizer.scheduler.max_lr=2e-3 +optimizer.scheduler.total_steps=20000
148153

149154
# Pretrained weight can be downloaded through (av2), check all other datasets in the same folder.
150155
wget https://huggingface.co/kin-zhang/OpenSceneFlow/resolve/main/deltaflow/deltaflow-av2.ckpt
@@ -211,7 +216,7 @@ python train.py model=deflow optimizer.lr=2e-4 epochs=9 batch_size=16 loss_fn=se
211216
wget https://huggingface.co/kin-zhang/OpenSceneFlow/resolve/main/seflow_best.ckpt
212217
```
213218

214-
#### VoteFLow
219+
#### VoteFlow
215220
Extra pakcges needed for VoteFlow, [pytorch3d](https://pytorch3d.org/) (prefer 0.7.7) and [torch-scatter](https://github.com/rusty1s/pytorch_scatter?tab=readme-ov-file) (prefer 2.1.2):
216221

217222
```bash
@@ -350,6 +355,13 @@ It is actively maintained and developed by the community (ref. below works).
350355
If you find it useful, please cite our works:
351356

352357
```bibtex
358+
@inproceedings{zhang2026teflow,
359+
title = {{TeFlow}: Enabling Multi-frame Supervision for Self-Supervised Feed-forward Scene Flow Estimation},
360+
author={Zhang, Qingwen and Jiang, Chenhan and Zhu, Xiaomeng and Miao, Yunqi and Zhang, Yushan and Andersson, Olov and Jensfelt, Patric},
361+
year = {2026},
362+
booktitle = {Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
363+
pages = {},
364+
}
353365
@inproceedings{zhang2024seflow,
354366
author={Zhang, Qingwen and Yang, Yi and Li, Peizheng and Andersson, Olov and Jensfelt, Patric},
355367
title={{SeFlow}: A Self-Supervised Scene Flow Method in Autonomous Driving},
@@ -383,17 +395,21 @@ If you find it useful, please cite our works:
383395
year={2025},
384396
url={https://openreview.net/forum?id=T9qNDtvAJX}
385397
}
386-
@misc{zhang2025teflow,
387-
title={{TeFlow}: Enabling Multi-frame Supervision for Feed-forward Scene Flow Estimation},
388-
author={Zhang, Qingwen and Jiang, Chenhan and Zhu, Xiaomeng and Miao, Yunqi and Zhang, Yushan and Andersson, Olov and Jensfelt, Patric},
389-
year={2025},
390-
url={https://openreview.net/forum?id=h70FLgnIAw}
391-
}
392398
```
393399

394400
And our excellent collaborators works contributed to this codebase also:
395401

396402
```bibtex
403+
@article{khoche2026dogflow,
404+
author={Khoche, Ajinkya and Zhang, Qingwen and Cai, Yixi and Mansouri, Sina Sharif and Jensfelt, Patric},
405+
journal = {IEEE Robotics and Automation Letters},
406+
title = {{DoGFlow}: Self-Supervised LiDAR Scene Flow via Cross-Modal Doppler Guidance},
407+
year = {2026},
408+
volume = {11},
409+
number = {3},
410+
pages = {3836-3843},
411+
doi = {10.1109/LRA.2026.3662592},
412+
}
397413
@article{kim2025flow4d,
398414
author={Kim, Jaeyeul and Woo, Jungwan and Shin, Ukcheol and Oh, Jean and Im, Sunghoon},
399415
journal={IEEE Robotics and Automation Letters},

dataprocess/extract_truckscenes.py

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,12 @@ def create_group_data(group, pc, pose, lidar_id, lidar_center, gm = None, flow_0
6868
def compute_flow_simple(data_fn, pc0, pose0, pose1, ts0, ts1, sample_ann_list, dclass, DataNameMap=ManNamMap):
6969
# compute delta transform between pose0 and pose1
7070
ego1_SE3_ego0 = npcal_pose0to1(pose0, pose1)
71-
# flow due to ego motion
72-
flow = np.zeros_like(pc0[:,:3])
73-
flow = pc0[:,:3] @ ego1_SE3_ego0[:3,:3].T + ego1_SE3_ego0[:3,3] - pc0[:,:3] # pose flow
71+
# flow due to ego motion (baseline for all points)
72+
ego_flow = pc0[:,:3] @ ego1_SE3_ego0[:3,:3].T + ego1_SE3_ego0[:3,3] - pc0[:,:3]
73+
74+
# object flow (without ego motion), used to track max flow magnitude
75+
obj_flow_all = np.zeros_like(pc0[:,:3])
76+
obj_flow_magnitude = np.zeros(len(pc0), dtype=np.float32)
7477

7578
valid = np.ones(len(pc0), dtype=np.bool_)
7679
classes = np.zeros(len(pc0), dtype=np.uint8)
@@ -96,13 +99,21 @@ def compute_flow_simple(data_fn, pc0, pose0, pose1, ts0, ts1, sample_ann_list, d
9699
classes[points_in_box_mask] = CATEGORY_TO_INDEX[DataNameMap[cls]]
97100

98101
if np.sum(points_in_box_mask) > 5:
99-
obj_flow = np.ones_like(pc0[points_in_box_mask,:3]) * ann_vel * delta_t
100-
flow[points_in_box_mask] += obj_flow
101-
instances[points_in_box_mask] = (dclass[id_]+1)
102+
obj_flow = ann_vel * delta_t
103+
obj_flow_mag = np.linalg.norm(obj_flow)
104+
105+
# For overlapping boxes, keep the flow with higher magnitude
106+
higher_flow_mask = points_in_box_mask & (obj_flow_mag > obj_flow_magnitude)
107+
obj_flow_all[higher_flow_mask] = obj_flow
108+
obj_flow_magnitude[higher_flow_mask] = obj_flow_mag
109+
instances[higher_flow_mask] = (dclass[id_]+1)
102110
id_ += 1
103111
else:
104112
valid[points_in_box_mask] = False
105113

114+
# Final flow = ego motion + object flow (with max magnitude selection)
115+
flow = ego_flow + obj_flow_all
116+
106117
return {'flow_0_1': flow, 'valid_0': valid, 'classes_0': classes,
107118
'ego_motion': ego1_SE3_ego0, 'flow_instance_id': instances}
108119

@@ -183,16 +194,18 @@ def compute_flow_simple(data_fn, pc0, pose0, pose1, ts0, ts1, sample_ann_list, d
183194
# lidar_dt = lidar_dt[not_close]
184195
is_ground_0 = np.array(mygroundseg.run(points[:, :3]))
185196

197+
# HARDCODE: for TruckScenes, add all points below 0.2m as ground
198+
is_ground_0 = is_ground_0 | (points[:,2] < 0.2)
199+
200+
group = f.create_group(str(ts0))
186201
if cnt == len(full_sweep_data_dict[SelectedSensor[-1]]) - 1:
187-
group = f.create_group(str(ts0))
188202
create_group_data(group=group, pc=points, gm=is_ground_0.astype(np.bool_), pose=pose0, \
189203
lidar_id=lidar_id, lidar_center=lidar_center)
190204
else:
191205
sweep_data_next = full_sweep_data_dict[SelectedSensor[-1]][cnt+1]
192206
ts1 = sweep_data_next['timestamp']
193207
pose1 = get_pose(mants, sweep_data_next, w2stf=False)
194208

195-
group = f.create_group(str(ts0))
196209
# annotated frame, compute flow
197210
if sweep_data['is_key_frame'] and sweep_data['prev'] != "":
198211
curr_scene_ann = mants.get_boxes(sweep_data['token'])
@@ -242,7 +255,7 @@ def process_logs(data_mode, data_dir: Path, scene_list: list, output_dir: Path,
242255
res = list(tqdm(p.imap_unordered(proc, args), total=len(scene_list), ncols=100))
243256

244257
def main(
245-
data_dir: str = "/home/kin/data/truckscenes/man-truckscenes",
258+
data_dir: str = "/home/kin/data/man-demo/man-truckscenes",
246259
mode: str = "v1.0-mini",
247260
output_dir: str ="/home/kin/data/truckscenes/h5py",
248261
nproc: int = (multiprocessing.cpu_count() - 1),

0 commit comments

Comments
 (0)