🚀 Beta testing End-to-End Visual Control Exercise Combined Dataset

[codezerro]

We’ve released new combined dataset for the End-to-End Visual Control exercise on Hugging Face. We invite everyone to build and train a model using this dataset. The goal is to develop a model capable of completing at least one full lap of the track, although completing multiple laps is encouraged. Once your model is ready, please share a short video here.

[JitinSaxenaa]

Hi @codezerro @javizqh @jmplaza,

I trained a model using the newly released combined dataset (~4.8 GB) and successfully exported it to ONNX format. However, when testing it in the End-to-End Visual Control exercise, I’m facing an issue where the application does not execute the code properly.

Observations:

• The script starts executing (I can see an initial print("File Running") when placed at the very top).
• However, execution seems to stop right after importing WebGUI.
• Any print statements placed after import WebGUI are not displayed in the terminal.
• The control loop never starts.

This issue is not specific to the new model — even my previously working ONNX model now shows the same behavior.

I have the latest RADI and the latest version of RoboticsAcademy running locally.

[JitinSaxenaa]

On making changes :

From -

host = "ws://127.0.0.1:2303"
gui = WebGUI(host)
start_console()


def showImage(image):
    gui.showImage(image)

To:

host = "ws://127.0.0.1:2303"

if __name__ == "__main__":
    gui = WebGUI(host)
    start_console()


def showImage(image):
    gui.showImage(image)

The code now executes successfully and the car starts moving. However, the WebGUI still does not display any images. These are just a few observations I’ve made, might be relevant.

[javizqh]

From my end I do not have such problem.

[codezerro]

Read the exercise document : https://jderobot.github.io/RoboticsAcademy/exercises/AutonomousCars/end_to_end_visual_control/

we have a model path func

from model import model_path_func

model_path = model_path_func("model.onnx")

[JitinSaxenaa]

The model is loading successfully but there is WebGUI issue on my side

[JitinSaxenaa]

I rebuild the RADI and still stuck here..the code didnt execute the WebGUI..only does when i do if __name__ == "__main__": this change in the vscode..and even then do not able to get the images on the application.

Please guide @codezerro @javizqh

[javizqh]

Can you change the order of the imports? First WebGUI, then HAL

[JitinSaxenaa]

Still same..

[keerthivnair]

Hi @codezerro

Did the testing for the end2end with the combined dataset. Was able to complete more than half the circuit. Maybe shud check the epochs, am working on that. Everything else seems to working fine, eventhough in turns tends..to deviate from the red line correction is happening so dataset is pretty accurate.
Attatching the video of my simulation heree

Screencast.from.2026-04-20.23-40-06.webm

[javizqh]

This dataset is for the new version of the circuit (The one with no walls). Please test it in the new version of Robotics Academy

[keerthivnair]

Oh....okay.... how to update to the new version of the Academy?universe db to be updated then ?

[javizqh]

Pull the latest changes from humble-devel, and update both docker images

[keerthivnair]

okayy thankyou, will test and update

[keerthivnair]

Hi @codezerro @javizqh

updated the academy and the db to latest and retrained the model .... first turn is pretty good but after the turn model just deviates randomly....as is seen in the terminal the velocity value just explodes...am figuring out the issue tho.
I tried reducing the lr and also flipped sm images incase the turns in the dataset was biased towards a particular direction turns ..... still only slight improvement.
Any inputs on could be reasons from your side would be helpful be great!

Screencast.from.2026-04-22.22-23-58.webm