You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+38-14Lines changed: 38 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@ VisionProTeleop
2
2
===========
3
3
4
4
5
-
> **🎉 UPDATE: Now supporting Low-Latency Video Streaming!** You can now stream back your robot's camera feed back to Vision Pro via webRTC protocol, alongside the original hand tracking data stream. No complicated network setting required. Download the app, `pip install avp_stream`, and you're done!
5
+
> **🎉 UPDATE: Now supporting Low-Latency Video Streaming!** You can now stream back your robot's camera feed back to Vision Pro via webRTC protocol, alongside the original hand tracking data stream. No complicated network setting required. Update the app, `pip install --upgrade avp_stream`, and you're done!
6
6
7
7
8
8
<divalign="center">
@@ -40,26 +40,30 @@ If you use this repository in your work, consider citing:
40
40
41
41
### Step 1. Install the app on Vision Pro
42
42
43
-

44
43
45
-
This app is now officially on VisionOS App Store! You can search for **[Tracking Streamer](https://apps.apple.com/us/app/tracking-streamer/id6478969032)** from the App Store and install the app.
44
+
This app is officially on VisionOS App Store! You can search for **[Tracking Streamer](https://apps.apple.com/us/app/tracking-streamer/id6478969032)** from the App Store and install the app.
46
45
47
-
If you want to play around with the app, you can build/install the app yourself too. To learn how to do that, take a look at this [documentation](/how_to_install.md). This requires (a) Apple Developer Account, (b) Vision Pro Developer Strap, and (c) a Mac with Xcode installed.
46
+
47
+
> If you want to play around with the app, you can build/install the app yourself too. To learn how to do that, take a look at this [documentation](/how_to_install.md). This requires (a) Apple Developer Account, (b) Vision Pro Developer Strap, and (c) a Mac with Xcode installed.
48
48
49
49
50
50
### Step 2. Run the app on Vision Pro
51
51
52
-
After installation, click on the app on Vision Pro and click `Start`. That's it! Vision Pro is now streaming the tracking data over your wifi network.
52
+
After installation, click and open the app on Vision Pro. It should show something like this. Click **START with Video Streaming** if you want to stream back videos from your robot over webRTC. Otherwise, click the left button. That's it!
53
+
54
+
Vision Pro is now streaming the tracking data over your wifi network via gRPC, and ready to receive video stream via webRTC.
55
+
56
+
57
+

53
58
54
-
**Tip** Remember the IP address before you click start; you need to specify this IP address to subscribe to the data. Once you click start, the app will immediately enter into pass-through mode. Click on the digital crown to stop streaming.
55
59
56
60
57
61
### Step 3. Receive the hand tracking data from anywhere
58
62
59
63
The following python package allows you to receive the data stream from any device that's connected to the same WiFi network. First, install the package:
60
64
61
65
```
62
-
pip install avp_stream
66
+
pip install --upgrade avp_stream
63
67
```
64
68
65
69
Then, add this code snippet to any of your projects you were developing:
### Step 4. [🎉NEW FEATURE] Stream video feeds back to Vision Pro!
81
+
### Step 4. [🎉V2 Update🎉] Stream video feeds back to Vision Pro!
78
82
79
-
Streaming your robot's video feed back to Vision Pro requires one additional line: `start_video_streaming`.
83
+
Streaming your robot's video feed back to Vision Pro requires one additional line: `start_video_streaming`. This feature is only supported on the latest version of the VisionOS app, and python package. Make sure you upgrade both python library / visionOS app.
80
84
81
85
```python
82
86
from avp_stream import VisionProStreamer
@@ -86,7 +90,7 @@ s = VisionProStreamer(ip = avp_ip)
- work without a physical camera and send over synthetically generated frames (i.e., simulation renderings, or purely synthetic images)
107
120
```python
108
121
s = VisionProStreamer(ip= avp_ip)
109
122
# define your own image generating function, and register
@@ -114,7 +127,14 @@ You can also:
114
127
115
128
which is explained in detail in [examples](examples) folder.
116
129
117
-
> **NOTE**: Finding the right combination of `device`, `format`, `size`, and `fps` can sometime be tricky, since camera models only support certain combination. Also, depending on your operation system and how you plugged the camera, the `device` might have been mounted on a different directory.
130
+
**Note:** Finding the right combination of `device`, `format`, `size`, and `fps` can sometime be tricky, since camera models only support certain combination. Also, depending on your operation system and how you plugged the camera, the camera might have been mounted on a different directory. To find the right combination of these parameters, you can use the following script:
131
+
132
+
```bash
133
+
python test_video_devices.py --live
134
+
```
135
+
When it's the right combination, it'll show up a live feed of the selected camera.
136
+
137
+
118
138
119
139
120
140
@@ -168,15 +188,19 @@ Refer to the image above to see what order the joints are represented in each ha
168
188
169
189
## App Features
170
190
171
-
### Status Window
191
+
### Status / Control Window
172
192
193
+
Status window is a good way to monitor the networking status, and control the life cycle of the app. It is shown in full view when you first start the app, but automatically gets minimized when video frame comes in. You can also minimize / maximize as you need.
194
+
195
+

173
196
174
197
175
198
### Modifying the Video Viewport
176
199
200
+
You can also modify the video viewport -- how far they are from your eyes.
0 commit comments