spike(#2534): route getDisplayMedia through in-app picker on Wayland#2538
Open
IsmaelMartinez wants to merge 7 commits into
Open
spike(#2534): route getDisplayMedia through in-app picker on Wayland#2538IsmaelMartinez wants to merge 7 commits into
IsmaelMartinez wants to merge 7 commits into
Conversation
) On Wayland with WebRTCPipeWireCapturer enabled, Chromium's native getDisplayMedia handles source selection via xdg-desktop-portal and bypasses Electron's setDisplayMediaRequestHandler entirely. The thumbnail preview window only fires from the in-app StreamSelector path, so Wayland users have never seen it. Spike intercepts getDisplayMedia in the injected renderer script: 1. Calls a new show-tfl-stream-picker IPC to display StreamSelector and pick the source on the main side (which also triggers the preview window creation via the existing handleScreenSourceSelection flow). 2. Synthesises the stream via getUserMedia with chromeMediaSource: 'desktop' + chromeMediaSourceId, the same shape the preview window already uses successfully on Wayland. Out of scope for the spike: config flag, multi-account session handling, automated tests. The original setDisplayMediaRequestHandler path is still registered as fallback for environments where the IPC is unavailable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request implements a spike to route getDisplayMedia through the in-app StreamSelector on Wayland to restore the screen-share preview window. Key changes include adding a new IPC handler for the stream picker, exposing it to the renderer, and updating the injected screen-sharing script to intercept media requests. Review feedback suggests hardening the IPC handler to prevent hanging promises, preserving original video constraints during the getUserMedia fallback, and ensuring the main process is notified to close the preview window if the custom capture path fails.
Contributor
📦 PR Snap Build Artifacts✅ Snap builds successful! Download artifacts: 🐧 Linux Snap Packagesx86_64 (110.67 MB) arm64 (107.50 MB) armv7l (101.56 MB) 📝 Note: Other package formats (.deb, .rpm, .AppImage, .dmg, .exe) are built in the main workflow |
- IPC handler: resolve null when handleScreenSourceSelection fails to find the chosen source (high-priority comment) so the renderer treats it as a cancellation instead of calling getUserMedia with a stale ID. - getDisplayMedia wrapper: drop the post-picker fallback to native getDisplayMedia. The preview window is already open after the picker, and the fallback would open a second native picker on top of it. Surfacing the error directly is the right signal for a spike. Skipped the medium-priority comment about preserving original video constraints; that belongs to the real feature, not the bare-minimum verification. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
📦 PR Build Artifacts✅ Build successful! Download artifacts: 🐧 Linuxx86_64 (447.84 MB) - Contains: .deb, .rpm, .tar.gz, .AppImage arm64 (438.12 MB) - Contains: .deb, .rpm, .tar.gz, .AppImage armv7l (415.96 MB) - Contains: .deb, .rpm, .tar.gz, .AppImage 🍎 macOSx86_64 (129.24 MB) - Contains: .dmg 🪟 Windowsx86_64 (109.59 MB) - Contains: .exe installer 📝 Note: Snap packages (.snap) are built in a separate workflow 🕐 Last updated: 2026-05-20 12:56 UTC |
|
Hello, still no screensharing preview. This time, when selecting screen to share, there appears to be some docked bar under camera preview. Selecting window does nothig, I have to additionally click in that docked bar and select screen second time for screensharing to work. There's also a stack trace in log. 
ELECTRON_ENABLE_LOGGING=true teams-for-linuxLogs: |
Owner
Author
|
Oh, apologies my mistake. I completely messed up the preview window you meant, I assume it was the picker. I will put a message on the issue and rethink it. thanks for testing |
…2534) Previous iteration of this spike opened an in-app StreamSelector on Wayland on top of the OS portal picker, producing the double-picker reported on the PR. The preview thumbnail also failed because its renderer re-captured the source via getUserMedia, which on Wayland needed a fresh PipeWire token that was not reusable. This iteration changes direction: - Drop the in-app picker route from the renderer interceptor. getDisplayMedia goes back to Chromium's native flow: on X11/Win/macOS this lands in our setDisplayMediaRequestHandler -> StreamSelector; on Wayland it lands in xdg-desktop-portal directly. One picker, not two. - Replace the preview window's getUserMedia(chromeMediaSource:'desktop') re-capture with a MessagePort relay. After the Teams renderer captures the share, main posts a port to both renderers via webContents.postMessage; the Teams side pumps VideoFrames through with MediaStreamTrackProcessor, the preview reconstructs the track via MediaStreamTrackGenerator. Single capture, no second portal prompt on Wayland, and roughly halved CPU/GPU cost for screen-share streams. Out of scope for this iteration: removal of the now-unused get-screen-share-stream / get-screen-share-screen IPC handlers in screenSharing/service.js, automated tests around the relay, config flag for the relay path. Left for follow-up once @piotrleszczynski confirms the preview now renders on his Wayland session. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…origin
SonarCloud S2819: the two ipcRenderer.on("screen-share-port") forwarders were
calling window.postMessage(..., "*", ports), which is broad enough to be
flagged as a cross-origin concern. Both posts target the same window's main
world, so window.location.origin always matches the receiver - swap "*" for
window.location.origin to satisfy the rule without behavioural change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… + re-fit preview on source change Two follow-ups from the review pass on #2534: 1. ScreenSharingService used `#selectedScreenShareSource !== null` as the sharing-active signal. The Wayland path now sends `null` (the OS portal handles the picker, no desktopCapturer source ID is produced), so the prior null-guard left the state stale and `getScreenSharingStatus()` returned false while sharing was active. Add a dedicated `#isSharing` boolean set on `screen-sharing-started`, cleared on the two stop paths, and source it from `isScreenSharingActive()` / `#handleGetScreenSharingStatus()`. Source-ID tracking via `#selectedScreenShareSource` stays as-is for the X11 path consumers. 2. The preview window's `loadedmetadata` resize guard (`resizeApplied`) was one-shot per page load. If Teams swaps the shared source mid-session ("Change what's shared"), the new track would render at the prior aspect ratio. Reset `resizeApplied = false` and close the previous writer at the top of `wirePort` so a second port delivery re-fits cleanly. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
To be absolutely sure what's the issue: on mainstream 2.8 version:
on spike version:
|
…tmap snapshots (#2534) Local repro on macOS proved that VideoFrame transfer over a MessagePort does not survive crossing two BrowserWindow renderer processes: every frame fired `messageerror` on the preview side and the thumbnail stayed blank. MediaStreamTrackProcessor + transferable VideoFrame works inside a single renderer; it does not work cross-renderer because the underlying GPU-backed buffer (IOSurface on macOS, DMA-BUF on Linux) is not reconstructable on the receiving side. Switch the relay to periodic ImageBitmap snapshots, which do transfer cross-process cleanly: - Teams renderer attaches a hidden off-screen <video> to the captured stream, then ticks an OffscreenCanvas at 5 fps (drawImage -> transferToImageBitmap -> port.postMessage(bitmap, [bitmap])). The snapshot is scaled so the longest edge is 320px, matching the preview-window size and keeping transfer cost low. - Preview window swaps its <video> + MediaStreamTrackGenerator for a <canvas> with a `bitmaprenderer` 2D context; each incoming ImageBitmap is painted via transferFromImageBitmap, with a one-shot resize-to-fit on the first frame using the bitmap's intrinsic dimensions. Trade-off: 5 fps thumbnail rather than full 30 fps live mirror. For the "what am I sharing" preview use case this is intentional - cheap, single capture path, works cross-platform, works on Wayland without a second portal prompt. Confirmed locally on macOS: preview window now renders the active screen share, zero relay errors across the session. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spike, not for merge
This is a draft PR for @piotrleszczynski (and any other Wayland reporter) to test whether the proposed approach for #2534 holds.
Background
On Wayland with
WebRTCPipeWireCapturerenabled (TFL's default), Chromium's nativegetDisplayMediahandles source selection via xdg-desktop-portal and bypasses Electron'ssetDisplayMediaRequestHandlerentirely. The thumbnail preview window is only created from the in-appStreamSelectorpath, so Wayland users have never seen it.What this spike does
Intercepts
getDisplayMediain the injected renderer script and routes the source-selection step through TFL's existing in-appStreamSelectorinstead:show-tfl-stream-pickeropens the in-appStreamSelector, runs the existinghandleScreenSourceSelectionflow (which creates the preview window), and returns the chosen source ID.getUserMedia({ video: { mandatory: { chromeMediaSource: 'desktop', chromeMediaSourceId } } })to synthesise the stream. This is the same constraint shapeapp/screenSharing/previewWindow.htmlalready uses successfully on Wayland, so Chromium + PipeWire should resolve it identically.The original
setDisplayMediaRequestHandleris unchanged and acts as a fallback when the new IPC is unavailable.Out of scope for the spike
Test plan (for @piotrleszczynski)
Wayland session (KDE Plasma + xdg-desktop-portal-kde, TUXEDO OS):
--ozone-platform=x11flag needed).If steps 4 and 6 pass and step 7 works, the hypothesis is confirmed and we can promote this to a real feature (with config flag, multi-account session handling, X11 verification, tests). If step 4 fails, please attach a fresh log. If step 7 fails, the
getUserMedia+chromeMediaSource: 'desktop'substitution is not equivalent to nativegetDisplayMediaon Wayland for Teams' WebRTC pipeline, and the spike disproves the approach.Refs #2534. Not for merge as-is.