Skip to content

Commit 393437f

Browse files
TechnicallyComputersTechnicallyComputers
committed
Live Stream Latency Optimizations & Focus
Significant optimizations for livestream video latency for mobile and remote clients. Removing other lobby type options at this time and only supporting livestream. P2P is optional and not supported for input, only relay is an official supported, because P2P input connections depend on user ICE configurations and network setup with clients networks. Updates to session management for reconnects and network transitions for mobile users, not ideal, but there is aggressive session reconnect logic that works well when the user is switching networks. Initial network switch refreshes transports, and for some reason session clean up removes the transports 30 seconds later anyways, but it also prompts and instantaneous refresh and it seems to work pretty smoothly good enough for mobile users switching networks.
1 parent e71892b commit 393437f

16 files changed

Lines changed: 3435 additions & 1131 deletions

‎RELAY_DIAGNOSTIC.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# Relay Video/Audio/Input Diagnostic Guide
2+
3+
Use this checklist to isolate why relay streaming fails. Run tests in order.
4+
5+
---
6+
7+
## 1. Browser Console Checks (Host + Client)
8+
9+
### Host (creator of the room)
10+
11+
Open DevTools → Console. Look for these messages **in order**:
12+
13+
| Step | Expected log | If missing, problem |
14+
|------|--------------|---------------------|
15+
| 1 | `[Netplay] netplaySetupProducers called` | Producer setup never ran |
16+
| 2 | `[Netplay] ✅ Host transports initialized` | SFU transport init failed |
17+
| 3 | `[Netplay] Attempting to capture direct video output...` | Video capture not attempted |
18+
| 4 | `[Netplay] ✅ Video producer created` | Video capture failed or produce failed |
19+
| 5 | `[Netplay] 🔊 Setting up game audio producer...` | Audio setup not reached |
20+
| 6 | `[Netplay] ✅ Game audio producer created` | Audio capture failed |
21+
| 7 | `[Netplay] Data producer created for input` | Input relay data producer failed |
22+
23+
**If you see:**
24+
- `[Netplay] ⚠️ No video track captured` → Canvas/video capture failing (platform-specific)
25+
- `[Netplay] Skipping video capture - not host` → Host role not set correctly
26+
- `[Netplay] Will retry video capture when game starts` → Game not running yet; start a ROM
27+
28+
### Client (joiner)
29+
30+
| Step | Expected log | If missing, problem |
31+
|------|--------------|---------------------|
32+
| 1 | `[Netplay] Setting up consumer transports...` | Consumer setup not started |
33+
| 2 | `[Netplay] ✅ Client receive transport created` | Client transport failed |
34+
| 3 | `[Netplay] Requesting existing producers...` | Never requested producers |
35+
| 4 | `[Netplay] Received existing video/audio producers:` (with array) | SFU returned empty or error |
36+
| 5 | `[Netplay] Creating consumer for existing producer:` | No producers to consume |
37+
| 6 | `[Netplay] ✅ Created video consumer` | createConsumer failed |
38+
| 7 | `[Netplay] ✅ Created audio consumer` | createConsumer failed |
39+
| 8 | `[Netplay] 📡 RECEIVED new-producer event:` | new-producer never emitted (or client joined before host produced) |
40+
41+
**If you see:**
42+
- `[Netplay] Received existing video/audio producers: []` → Host has no producers yet, or room/socket mismatch
43+
- `[Netplay] ⚠️ Video consumer has no track` → Consumer created but track not ready (timing/transport)
44+
- `[Netplay] Failed to create consumer for existing producer` → Producer ID invalid or SFU error
45+
46+
---
47+
48+
## 2. SFU Server Logs (romm-sfu-server)
49+
50+
Run the SFU server with verbose logging. Look for:
51+
52+
| Event | Meaning |
53+
|-------|---------|
54+
| `sfu-produce request from` | Host is producing video/audio |
55+
| `broadcast new-producer to room` | SFU notified clients of new producer |
56+
| `sfu-get-producers: checking producers for sockets:` | Client requested producers |
57+
| `sfu-get-producers: no producers for socket X` | Host socket has no producers |
58+
| `sfu-get-producers: found N producers` | Host has producers; client should get them |
59+
| `sfu-produce-data request from` | Client creating data producer for input |
60+
| `producedata` / `produceData` | Data producer registered |
61+
62+
**Common failure:** `sfu-get-producers: no producers for socket X` — Host's socketId not in room, or host hasn't produced yet.
63+
64+
---
65+
66+
## 3. Quick Tests You Can Run
67+
68+
### Test A: Host-only (desktop)
69+
70+
1. Create a room as host.
71+
2. Start a game (load ROM, run).
72+
3. In console, run:
73+
```js
74+
// Check producer state
75+
window.EJS_emulator?.netplay?.engine?.sfuTransport && {
76+
videoProducer: !!window.EJS_emulator.netplay.engine.sfuTransport.videoProducer,
77+
audioProducer: !!window.EJS_emulator.netplay.engine.sfuTransport.audioProducer,
78+
dataProducer: !!window.EJS_emulator.netplay.engine.sfuTransport.dataProducer,
79+
}
80+
```
81+
Expected: all `true` after a few seconds.
82+
83+
### Test B: Client join (second browser/device)
84+
85+
1. Host creates room and starts game.
86+
2. Client joins.
87+
3. In client console, run:
88+
```js
89+
// Check consumer state
90+
window.EJS_emulator?.netplay?.engine?.sfuTransport?.consumers &&
91+
Array.from(window.EJS_emulator.netplay.engine.sfuTransport.consumers.entries()).map(([id, c]) => ({
92+
id, kind: c.kind, hasTrack: !!c.track
93+
}))
94+
```
95+
Expected: array of consumers with `hasTrack: true`.
96+
97+
### Test C: Relay input mode (default)
98+
99+
1. **Relay** is now the default input mode. In Netplay settings, ensure Input Mode is **Relay**.
100+
2. Join as client with a player slot.
101+
3. Check console for `[DataChannelManager] isReady check` — should show `dataProducer: true` for relay.
102+
103+
### Test D: Socket/room alignment
104+
105+
On the **SFU server**, when a client joins, verify:
106+
- The host's `socketId` is in `room.players` for that room.
107+
- RomM and the SFU server use the same room identifier (sessionid/roomName).
108+
109+
---
110+
111+
## 4. Platform-Specific (Android / Mobile)
112+
113+
- **Video capture:** `captureStream` / `getDisplayMedia` may be restricted. Look for `[Netplay] Direct emulator video output failed` or similar.
114+
- **Data producer delay:** Mobile waits 2s before creating data producer. If you see `[Netplay] Data producer attempt 1/3 failed`, wait for retries.
115+
- **Resolution:** Use 480p in Netplay settings for best latency on LTE. 720p/1080p for faster networks.
116+
117+
---
118+
119+
## 5. One-Line Diagnostic Dump
120+
121+
Paste this in the browser console (host or client) to get a snapshot:
122+
123+
```javascript
124+
(function(){
125+
const e = window.EJS_emulator?.netplay?.engine;
126+
if (!e) return { error: "No netplay engine" };
127+
const sfu = e.sfuTransport;
128+
const sess = e.sessionState;
129+
return {
130+
role: sess?.isHostRole?.() ? "host" : "client",
131+
sfuAvailable: !!sfu?.useSFU,
132+
host: {
133+
videoProducer: !!sfu?.videoProducer,
134+
audioProducer: !!sfu?.audioProducer,
135+
dataProducer: !!sfu?.dataProducer,
136+
},
137+
client: {
138+
consumers: sfu?.consumers ? Array.from(sfu.consumers.entries()).map(([id,c]) => ({ id, kind: c.kind, hasTrack: !!c.track })) : [],
139+
},
140+
dataChannel: {
141+
mode: e.dataChannelManager?.mode,
142+
hasDataProducer: !!e.dataChannelManager?.dataProducer,
143+
p2pChannels: e.dataChannelManager?.p2pChannels?.size ?? 0,
144+
},
145+
};
146+
})();
147+
```
148+
149+
---
150+
151+
## 6. Report Back
152+
153+
When reporting, include:
154+
155+
1. **Role:** Host or Client (or both)?
156+
2. **Platform:** Desktop Chrome, Android Chrome, etc.?
157+
3. **Console snippets:** The exact `[Netplay]` and `[SFUTransport]` lines around the failure.
158+
4. **SFU server logs:** Relevant lines when the failure occurs.
159+
5. **Test results:** Which of Tests A–D passed or failed?
160+
6. **Diagnostic dump:** Output of the one-line diagnostic (Section 5).

0 commit comments

Comments
 (0)