Machine learning

[michaelwareman]

I have tried all three of the new Machine Learning tutorials. Thank for making t:his and Pybricks possible.

A couple of questions:
Is it possible to create other smart sensor setups like line following, getures? What about using the phone's microphone for speech recognition?

Is it possible when using the image classification to get the x, y, width and height values?

When I add more values to be unpacked I get this error:
File "L13_2_classify_minfig.py", line 17, in
ValueError: need more than 2 values to unpack

my block code:

Keep up the Great Work!

[mcortezv]

The error occurs because your model only returns two values ​​(category and confidence), but you're trying to unpack more, which is why it's failing. You need to adjust the code to match what the block actually delivers.

In Pybricks ML, classification only provides label and confidence, not x/y or width/height. Those values ​​are only available with object detection, which isn't included in that system.

[laurensvalk]

Thanks for trying these out! Glad you liked it. There is a third model already that you can try out here:

It can detect objects with position without any training. It can detect every-day objects such as person, cat, keyboard. Using the block you can already use this. There is just no tutorial yet.

For each object that you choose for detection, you will get 5 values. So if you choose 2 objects you will get lists of 10 values.

It is quite similar the simple color tracker, with one extra value per object. You get X/Y/Width/Height/Confidence.

If you make something, share a video here 😄

[michaelwareman]

Hello Lauren:

I was able to create a basic program that has an orange and a car as objects to find. I have attached the Pybricks block program for those interested to see what I did.
L13_2_object.py

Here is the Google Drive link to a short video I created.
https://drive.google.com/file/d/1Lroo9u9dzmccfBP_ICLd6QhKp_SATTa7/view?usp=drive_link

The program begins to rotate the robot in place in 10 degree increaments. Once the orange is spotted the robot changes to 5 degree increaments. If the orange is centered (roughly) the robot moves towards the orange. When the robot is very close it stops getting closer. and starts beeping.

How do I convert the percentage values to coordinates that would make it easier to move towards the object?

I see that I can use the confidence variable, confidence1 = orange, confidence2 = car. So if the confince is greater than 0 an object is spotted. This is what I did in my program. Or is there another way?

My next step is to work out is how to control which object I am looking for in a better way. Maybe I need to create a function that handles obtaining the values x, y, width, height, based on which object and confidence level I am looking for?

Anyone with suggestions to make the program better your suggestions would be most appreciated.

[laurensvalk]

Nice! The video doesn't seem to be public though.

How do I convert the percentage values to coordinates that would make it easier to move towards the object?

They are already coordinates. They're expressed as a percentage of the longest side of your screen.

Anyone with suggestions to make the program better your suggestions would be most appreciated.

Your program looks fine. You could potentially make some helper functions to get the values you want. For example x and y of a particular object:

[michaelwareman]

I am having troubles recreating the above sample program. Is there a tutorial or sample program I can use to create my own?
What I have done so far:
I created a data and index variables - correct?
I created a my task and called it get x and y, I could not put my data and index variables so I renamed them data and index (did need to add a space for the renaming to hold) -- correct?
I then added the return block, added a list with block, then added a 2 list - get block -- correct, and shortened as needed?
This is where I am stuck. I am unable to add the data variable to the list - get block. I am able to add the two math blocks but not able to add the index variables.

[michaelwareman]

I think I figured it out. I happened to click/grab the renamed data variable on the my task block and put it in its place on the list - get block. I repeated that step with the second list - get block.

[michaelwareman]

Now I need help extracting the x and y values so that I can use them control my motors. Again is there a tutorial or sample programs on using lists?

[michaelwareman]

This link should work
https://drive.google.com/file/d/1Lroo9u9dzmccfBP_ICLd6QhKp_SATTa7/view?usp=sharing

[michaelwareman]

I have managed to make use of the x and y values. I used the unpack block to call the get x and y app block and assign them to x and y variables. I thought I could add more variables but that did not work. So, how do I either get all 5 app values or get specific ones like x, y, and confidence? Any help would be greatly appreciated.

[michaelwareman]

Okay??? After a bit of playing around I discover that the get x and y my block is not setting up the first and second oject but rather the first and second values for each object. So in order to get specific values the list get block sets up which value to get. I added a third list get block with an add of 4 for the fifth value -- confidence.

[michaelwareman]

Here are two videos showing my current robot:
https://drive.google.com/file/d/1yuDME48SLjO_zjunC6J_gPHZfMqJ89_v/view?usp=sharing
https://drive.google.com/file/d/1lT1TPdh8TWG0faPLQsULCJhSe85NnMm5/view?usp=sharing
I am reading Daniele Benedettelli's book "The Lego Mindstorms EV3 Laboratory", particulaly chapter 16 about finite state machines. This is were I am heading with all of this. As you can see the robot still needs some fine tuning. I have also included the current pyblocks block program. Any suggestions to improve the program would be appreciated. Also is there a better robot base to work with?
L13_2_object_improved.py
I was thinking of attaching the ultrasonic senor to the front of the robot. This would allow the robot to beter explore the environment

[laurensvalk]

Looks great!