Skip to content

Commit 8d587ec

Browse files
Update README.md
1 parent 21d85ee commit 8d587ec

1 file changed

Lines changed: 3 additions & 2 deletions

File tree

README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ What can Think2Drive + Bench2Drive provide ? <b>Please click to view the video.<
3636
5. [Citation](#citation)
3737

3838
## News <a name="news"></a>
39+
- [2026/03/27] Check [Bench2Drive-Speed](https://github.com/Thinklab-SJTU/Bench2Drive-Speed), which estimates a new & useful functionality of AD systems - speed customization!
40+
- [2025/10/13] Check [Bench2Drive-VL](https://github.com/Thinklab-SJTU/Bench2Drive-VL), which enables closed-loop QA of VLM4AD. It also provides a efficient communication interface between VLM environment (python 3.9 or higher) and CARLA (python 3.7 and 3.7).
3941
- [2025/02/18] In our latest work [DriveTransformer (ICLR25)](https://openreview.net/forum?id=M42KR4W9P5), **a tiny validation set [Dev10](./leaderboard/data/drivetransformer_bench2drive_dev10.xml) is proposed for quick development of models.** The 10 clips are carefully selected from the official 220 routes, to be both difficult and representative with low variance. It is suggested to be used for ablation study to avoid overfitting the whole bench2drive220 routes.
4042
- [2025/02/05] As meticulously described in [CARLA_GARGE](https://github.com/autonomousvision/carla_garage/blob/leaderboard_2/docs/common_mistakes_in_benchmarking_ad.md), L2 error is not a meaningful indictor at all. We agree that **authors' should stop reporting results on nuScenes open-loop planning and reviewers should not ask for nuScenes open-loop planning results**. The number on nuScenes open-loop planning is not persuasive at all and could only lead to unjustified and wrong conclusions, which may impede the development of the field. As a member of the community, we call for stopping using nuScenes open-loop planning as well.
4143
- [2024/10/14] As kindly pointed out in an [issue](https://github.com/Thinklab-SJTU/Bench2Drive/issues/112), typos were discovered in the ability calculation. We have corrected the typos and updated the multi-ability results. This update will **not affect driving score and success rate**.
@@ -128,10 +130,9 @@ Use the command line: *huggingface-cli download --repo-type dataset --resume-dow
128130
- In our evaluation tools, the launch of CARLA is automatic: https://github.com/Thinklab-SJTU/Bench2Drive/tree/main/leaderboard/leaderboard/leaderboard_evaluator.py#L203. But you could always start CARLA by the one single command line to debug.
129131
- CARLA is not controlled CUDA_VISIBLE_DEVICES! It is controlled by -graphicsadapter in the command line. **Interestingly, in some machines, for some unknown reasons, -graphicsadapter=1 is not available.** For example, with 4 GPUS, it might be: GPU0 -graphicsadapter=0, GPU1 -graphicsadapter=2, GPU2 -graphicsadapter=3, GPU3 -graphicsadapter=4.
130132
- The conflict of PORT is frequently happened. Use lsof-i:YOUR_PORT frequently to avoid conflict. Avoid use small port numbers (<10000 could be unsafe).
131-
- *4.26.2-0+++UE4+Release-4.26 522 0 Disabling core dumps*. Only showing these two lines without termination is good. *WARNING: lavapipe is not a conformant vulkan implementation, testing use only.* is bad.
133+
- *4.26.2-0+++UE4+Release-4.26 522 0 Disabling core dumps*. Showing these two lines without termination is good. *WARNING: lavapipe is not a conformant vulkan implementation, testing use only.* could be good while sometimes it has bugs.
132134
- **If you face issues, always try to start CARLA in one single line to make sure CARLA could run.** If CARLA is finished immediately, it is very possible to be related to Vulkan. *Try /usr/bin/vulkaninfo | head -n 5*
133135
- Re-install vulkan might be helpful *sudo apt install vulkan-tools; sudo apt install vulkan-utils* In the end, you need to make sure your vulkan is correct. We have tested *Vulkan Instance Version: 1.x WARNING: lavapipe is not a conformant vulkan implementation, testing use only.* and version 1.1/1.2/1.3 works fine.
134-
- We find that nvidia driver version 470 is good all the time. 515 has some problems but okay. 550 has lots of bugs.
135136
- *sleep* is important to avoid crash of CARLA. For example, https://github.com/Thinklab-SJTU/Bench2Drive/blob/main/leaderboard/leaderboard/leaderboard_evaluator.py#L207, the sleep time should be extended for slower machines. When it comes to multi-gpu evaluation, https://github.com/Thinklab-SJTU/Bench2Drive/blob/main/leaderboard/scripts/run_evaluation_multi_uniad.sh#L58, the sleep time should also be extended for slower machines.
136137

137138
## Benchmark <a name="benchmark"></a>

0 commit comments

Comments
 (0)