cmd=12 (stop command) label seems wrong

[Eden-Wang1710]

Based on video "TgavkeWrV7U", I randomly select the frames with cmd label 12 (stop). Then I extract the 40 frames after the selected frame to form a 4-second GIF.

It seems that the video clip is not "stop", and they tend to be "overtake other cars".

(first frame 36660)

(first frame 30680)

The above two are two examples. Since Github limits the gif to 10MB, I clip the video to less than 4 seconds. And I can send more to you through email.

[Eden-Wang1710]

I have checked other labels (cmd = 0, 1, 2, 3) and they are consistent with video clip. But cmd 12 is inconsistent with the video.

[Little-Podi]

Hi @Eden-Wang1710, we train a video classifier to annotate driver commands automatically. Therefore, it is possible that some captions do not match the video clips perfectly.

[Eden-Wang1710]

Little-Podi

Thanks for your reply! It's OK that the meaning of command label is correct.

But in this video, all the clips with command 12 (stop/slow down) seems to be speeding up and overtake other cars, like the 2 GIF above. I guess the possible reason is that there's no such speeding up operation in 13 command labels, then the model fails to classify this scene.

[GihhArwtw]

Little-Podi

Thanks for your reply! It's OK that the meaning of command label is correct.

But in this video, all the clips with command 12 (stop/slow down) seems to be speeding up and overtake other cars, like the 2 GIF above. I guess the possible reason is that there's no such speeding up operation in 14 command labels, then the model fails to classify this scene.

Yes, we don't have a speed-up command in the 14 command labels. The design of such labels is subject to existing data with action annotations, i.e. Honda-HDD-Action and Honda-HDD-Cause. Since the two datasets mentioned only contain these 14 labels (without one indicating acceleration), our hands are tied. You may refer to page 19 in our paper for more details.

Still, the two datasets mentioned above do contain labels of deviate (cmd 13), which is similar to overtake. However, the ego vehicle usually cannot complete the whole process of overtaking in 4-second video clips, making it difficult for the model to learn how to classify such video clips as overtaking / deviating.

[Eden-Wang1710]

Little-Podi

Thanks for your reply! It's OK that the meaning of command label is correct.
But in this video, all the clips with command 12 (stop/slow down) seems to be speeding up and overtake other cars, like the 2 GIF above. I guess the possible reason is that there's no such speeding up operation in 14 command labels, then the model fails to classify this scene.

Yes, we don't have a speed-up command in the 14 command labels. The design of such labels is subject to existing data with action annotations, i.e. Honda-HDD-Action and Honda-HDD-Cause. Since the two datasets mentioned only contain these 14 labels (without one indicating acceleration), our hands are tied. You may refer to page 19 in our paper for more details.

Still, the two datasets mentioned above do contain labels of deviate (cmd 13), which is similar to overtake. However, the ego vehicle usually cannot complete the whole process of overtaking in 4-second video clips, making it difficult for the model to learn how to classify such video clips as overtaking / deviating.

OK, thanks a lot for your explanation~