@@ -65,7 +65,11 @@ It's not like LLMs are particularly responsive anyway.
6565It's * impossible* to even retransmit a WebRTC audio packet within a browser; we tried at Discord.
6666The * implementation* is hard-coded for real-time latency ** or else** .
6767
68- Yes, Voice AI agents will eventually get the latency down to the conversational range.
68+ ** UPDATE** : Some WebRTC folks are claiming this is a skill issue.
69+ It might be possible to enable audio NACKs, but we couldn't figure out the correct SDP munging.
70+ Either way, the WebRTC jitter buffer is aggressively small.
71+
72+ And yes, Voice AI agents will eventually get the latency down to the conversational range.
6973But ** reducing latency has trade-offs** .
7074I'm not even sure that purposely degrading audio prompts will ever be worth it.
7175
@@ -235,17 +239,15 @@ But we still have to implement everything for web clients.
235239## But What Instead?
236240If not WebRTC, then what should you use for Voice AI?
237241
238- Honestly, if I was working at OpenAI, I'd just stream audio over WebSockets.
242+ Honestly, if I was working at OpenAI, I'd start by stream audio over WebSockets.
239243You can leverage existing TCP/HTTP infrastructure instead of inventing a custom WebRTC load balancer.
240244It makes for a boring blog post, but it's simple, works with Kubernetes, and SCALES.
241245
242- The day will come when it's desirable to drop audio packets .
243- Or you want to transmit video over the same connection .
244- Then you * should* switch to QUIC/ WebTransport, because...
246+ I think head-of-line blocking is a desirable user experience, not a liability .
247+ But the fated day will come and dropping/prioritizing some packets will be necessary .
248+ Then I think OpenAI * should* copy MoQ and utilize [ WebTransport] ( https://developer.mozilla.org/en-US/docs/Web/API/WebTransport_API ) , because...
245249
246- ## QUIC Fixes This
247- Look I've spent a long time dunking on WebRTC.
248- It's time to be positive and gush about QUIC.
250+ ** QUIC FIXES THIS**
249251
250252Remember the round trip discussion?
251253Good times.
0 commit comments