VoIP call support via webrtc-rs

[DemiMarie]

Implementing VoIP calls in downstream projects is not simple. Web clients can and must use the native browser implementation, but desktop and mobile clients don’t have that option, so e.g. Fractal doesn’t support calls at all. matrix-rust-sdk could provide helpers that make it easy to support calls in desktop and mobile clients.

[Hywan]

Hello @DemiMarie and thank you for your suggestion. What kind of helpers do you imagine? There is this matrix_sdk::widget API that is used to set up VoIP (at least it's used by Element X for Element Call). Do you expect more (maybe a better documentation… 🙄)?

[DemiMarie]

@Hywan I’m imagining an implementation of one-to-one calls that doesn’t rely on an embedded browser engine. Secure end-to-end encryption requires that all servers be treated as untrusted, which means that relying on a webview that loads JavaScript from Element’s servers isn’t an option.

[DemiMarie]

@Hywan To be specific, I would like matrix-rust-sdk to integrate with https://github.com/webrtc-rs/webrtc, which provides a memory-safe WebRTC implementation that doesn’t rely on a web browser.

[Hywan]

@manuroe what do you think about this suggestion?

[DemiMarie]

To elaborate: Windows, macOS, iOS, and Android have a system-provided webview, but desktop Linux and embedded systems do not. Embedded devices, obviously, have no webview unless they bring their own. Desktop Linux has WebKitGTK+ and Qt WebEngine, but these are tied to specific UI frameworks, and not every desktop Linux program uses GTK or Qt. Qt WebEngine also has a rather long patch gap compared to upstream Chromium.

WebRTC is also a large C++ library that has had remote code execution vulnerabilities in the past. Signal’s mobile apps disable some especially risky functionality, but I’m not sure if this can be done in a web environment.

[skyrover7]

matrix-org/matrix-spec-proposals#4143 ?

[Xenofic]

@Hywan is this a planned change?

https://github.com/webrtc-rs/webrtc

cc: @DemiMarie

[manuroe]

@Hywan is this a planned change?

No. It is not something the core team plans to do soon.

On the Element side, we have been discussing for the last weeks about making a native implementation of Element Call. Doing so, we could have used webrtc-rs (or the LiveKit SDK). But it is a significant project that we do not need anymore.
For context, our troubles with the Element Call web app was to properly manage audio routing via the webview. The last blocker for us was iOS but the Safari in the coming iOS 26 release fixes it. So, Element Call works good enough.

Having a natrive Element Call and thus a native webrtc implementation are something we dream of but we will not make it this year for sure. I cannot say for later.

[shaba]

Two years on, the gap is still there. I recently spent some time debugging audio between Fractal and Element Call as a hobby project. Turned out to be an E2EE mismatch: Element Call encrypts the payload with GCM, matrix-rust-sdk doesn't help with distributing the keys, so encrypted RTP ends up at the Opus decoder and we hear noise.

#6564 (community PR) is currently the only concrete native attempt at closing this. Whichever direction upstream picks — merging that PR, a non-Livekit alternative, or committing to widget-only — knowing the intent would help anyone trying to add calls to a native client. More details in my comment on #6564.