Merge upstream 124.0.4 into master (#16)

* ios: add support for external cameras on iPad

* ci: remove flipper from gumtestapp (react-native-webrtc#1608)

These were causing build errors recently, and were generally unneeded anyways.

* pc: align createDataChannel with standard

- Throw TypeError if no argument passed
- Stringify the label

Fixes: react-native-webrtc#1605

* android: report actual size in camera MediaStreamTrack settings (react-native-webrtc#1598)

* ios: fix exception in iOS 17+ w/ Xcode 15.4

* android: remove no longer used replace rule from manifest (react-native-webrtc#1609)

* sender: fix serializing RTCRtpSendParameters

It's possible for user code to replace encodings entirely. Thus, the
resulting array will not have RTCRtpEncodingParameters object instances,
but plain objects.

Handle it by deep-cloning the objects with JSON.parse(JSON.stringify(x))
since that will take care of appropriately serializing them, no matter
the type.

* misc: make serialization more resilient

Don't directly call toJSON, but rather rely on JSON serialization to do
it when cloning.

* release 124.0.4

89557ca misc: make serialization more resilient  ( Saúl Ibarra Corretgé 2024-08-14 11:53:32 +0200)
6cfedd7 sender: fix serializing RTCRtpSendParameters  ( Saúl Ibarra Corretgé 2024-08-14 11:11:08 +0200)
ac7f578 android: remove no longer used replace rule from manifest (react-native-webrtc#1609)  ( Saúl Ibarra Corretgé 2024-08-07 17:17:03 +0200)
f6667c8 ios: fix exception in iOS 17+ w/ Xcode 15.4  ( mtdxc 2024-08-07 17:22:10 +0800)
4c34ae1 android: report actual size in camera MediaStreamTrack settings (react-native-webrtc#1598)  ( davidliu 2024-08-07 17:56:57 +0900)
fb02a5b pc: align createDataChannel with standard  ( Saúl Ibarra Corretgé 2024-08-06 15:28:24 +0200)
c0ddefd ci: remove flipper from gumtestapp (react-native-webrtc#1608)  ( davidliu 2024-08-07 16:00:01 +0900)
a1bb18a ios: add support for external cameras on iPad  ( mtdxc 2024-07-10 20:25:23 +0800)

* ios: Add RTCAudioSession helper methods needed for CallKit (react-native-webrtc#1614)

* Fix package name references

---------

Co-authored-by: mtdxc <mtdxc@126.com>
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
Co-authored-by: Saúl Ibarra Corretgé <saghul@jitsi.org>

Author: 3 people

Documentation/iOSInstallation.md

@@ -8,7 +8,7 @@ See a sample app in the `examples/GumTestApp` directory.
 
 **IMPORTANT:** Make sure you are using CocoaPods 1.10 or higher.  
 You may have to change the `platform` field in your podfile.  
-`react-native-webrtc` doesn't support iOS < 12 
+`@livekit/react-native-webrtc` doesn't support iOS < 12 
 Set it to '12.0' or above or you'll get an error when running `pod install`.
 
 ```
@@ -26,6 +26,33 @@ Navigate to `<ProjectFolder>/ios/<ProjectName>/` and edit `Info.plist`, add the
 <string>Microphone permission description</string>
 ```
 
+## CallKit
+
+If your app uses a CallKit integration to handle incoming calls, then your
+CXProviderDelegate should call through to `RTCAudioSession.sharedInstance.audioSessionDidActivate/Deactivate` accordingly.
+
+```
+#import <WebRTC/RTCAudioSession.h>
+
+- (void) provider:(CXProvider *) provider didActivateAudioSession:(AVAudioSession *) audioSession {
+    [[RTCAudioSession sharedInstance] audioSessionDidActivate:[AVAudioSession sharedInstance]];
+}
+
+- (void) provider:(CXProvider *) provider didDeactivateAudioSession:(AVAudioSession *) audioSession {
+    [[RTCAudioSession sharedInstance] audioSessionDidDeactivate:[AVAudioSession sharedInstance]];
+}
+```
+
+Javascript methods are also provided to call these methods:
+
+```
+import { RTCAudioSession } from '@livekit/react-native-webrtc'
+
+// Call as needed.
+RTCAudioSession.audioSessionDidActivate();
+RTCAudioSession.audioSessionDidDeactivate();
+```
+
 ## Library not loaded/Code signature invalid
 
 This is an issue with iOS 13.3.1.  

Documentation/tvOSInstallation.md

@@ -13,7 +13,7 @@ Change the following dependency in your projects `package.json` file to get star
 
 **IMPORTANT:** Make sure you are using CocoaPods 1.10 or higher.  
 You may have to change the `platform` field in your podfile.  
-`react-native-webrtc` doesn't support tvOS < 16. Set it to '16.0' or above.
+`@livekit/react-native-webrtc` doesn't support tvOS < 16. Set it to '16.0' or above.
 Older versions of tvOS don't support WebRTC.
 
 ```

android/src/main/AndroidManifest.xml

@@ -8,9 +8,4 @@
                 android:foregroundServiceType="mediaProjection">
         </service>
     </application>
-    <uses-feature
-        android:name="android.hardware.usb.host"
-        android:required="false"
-        tools:node="replace"
-    />
 </manifest>

android/src/main/java/com/oney/WebRTCModule/AbstractVideoCaptureController.java

@@ -3,9 +3,13 @@
 import org.webrtc.VideoCapturer;
 
 public abstract class AbstractVideoCaptureController {
-    protected int width;
-    protected int height;
-    private final int fps;
+    protected final int targetWidth;
+    protected final int targetHeight;
+    protected final int targetFps;
+
+    protected int actualWidth;
+    protected int actualHeight;
+    protected int actualFps;
 
     /**
      * {@link VideoCapturer} which this controller manages.
@@ -15,9 +19,12 @@ public abstract class AbstractVideoCaptureController {
     protected CapturerEventsListener capturerEventsListener;
 
     public AbstractVideoCaptureController(int width, int height, int fps) {
-        this.width = width;
-        this.height = height;
-        this.fps = fps;
+        this.targetWidth = width;
+        this.targetHeight = height;
+        this.targetFps = fps;
+        this.actualWidth = width;
+        this.actualHeight = height;
+        this.actualFps = fps;
     }
 
     public void initializeVideoCapturer() {
@@ -32,15 +39,15 @@ public void dispose() {
     }
 
     public int getHeight() {
-        return height;
+        return actualHeight;
     }
 
     public int getWidth() {
-        return width;
+        return actualWidth;
     }
 
     public int getFrameRate() {
-        return fps;
+        return actualFps;
     }
 
     public VideoCapturer getVideoCapturer() {
@@ -49,7 +56,7 @@ public VideoCapturer getVideoCapturer() {
 
     public void startCapture() {
         try {
-            videoCapturer.startCapture(width, height, fps);
+            videoCapturer.startCapture(targetWidth, targetHeight, targetFps);
         } catch (RuntimeException e) {
             // XXX This can only fail if we initialize the capturer incorrectly,
             // which we don't. Thus, ignore any failures here since we trust

android/src/main/java/com/oney/WebRTCModule/CameraCaptureController.java

@@ -1,11 +1,20 @@
 package com.oney.WebRTCModule;
 
+import android.content.Context;
+import android.hardware.camera2.CameraManager;
 import android.util.Log;
+import android.util.Pair;
+import androidx.annotation.Nullable;
 
 import com.facebook.react.bridge.ReadableMap;
 
+import org.webrtc.Camera1Capturer;
+import org.webrtc.Camera1Helper;
+import org.webrtc.Camera2Capturer;
+import org.webrtc.Camera2Helper;
 import org.webrtc.CameraEnumerator;
 import org.webrtc.CameraVideoCapturer;
+import org.webrtc.Size;
 import org.webrtc.VideoCapturer;
 
 import java.util.ArrayList;
@@ -19,6 +28,7 @@ public class CameraCaptureController extends AbstractVideoCaptureController {
 
     private boolean isFrontFacing;
 
+    private final Context context;
     private final CameraEnumerator cameraEnumerator;
     private final ReadableMap constraints;
 
@@ -30,9 +40,10 @@ public class CameraCaptureController extends AbstractVideoCaptureController {
      */
     private final CameraEventsHandler cameraEventsHandler = new CameraEventsHandler();
 
-    public CameraCaptureController(CameraEnumerator cameraEnumerator, ReadableMap constraints) {
+    public CameraCaptureController(Context context, CameraEnumerator cameraEnumerator, ReadableMap constraints) {
         super(constraints.getInt("width"), constraints.getInt("height"), constraints.getInt("frameRate"));
 
+        this.context = context;
         this.cameraEnumerator = cameraEnumerator;
         this.constraints = constraints;
     }
@@ -75,7 +86,30 @@ protected VideoCapturer createVideoCapturer() {
         String deviceId = ReactBridgeUtil.getMapStrValue(this.constraints, "deviceId");
         String facingMode = ReactBridgeUtil.getMapStrValue(this.constraints, "facingMode");
 
-        return createVideoCapturer(deviceId, facingMode);
+        Pair<String, VideoCapturer> result = createVideoCapturer(deviceId, facingMode);
+        if(result == null) {
+            return null;
+        }
+
+        String cameraName = result.first;
+        VideoCapturer videoCapturer = result.second;
+
+        // Find actual capture format.
+        Size actualSize = null;
+        if (videoCapturer instanceof Camera1Capturer) {
+            int cameraId = Camera1Helper.getCameraId(cameraName);
+            actualSize = Camera1Helper.findClosestCaptureFormat(cameraId, targetWidth, targetHeight);
+        } else if (videoCapturer instanceof Camera2Capturer) {
+            CameraManager cameraManager = (CameraManager) context.getSystemService(Context.CAMERA_SERVICE);
+            actualSize = Camera2Helper.findClosestCaptureFormat(cameraManager, cameraName, targetWidth, targetHeight);
+        }
+
+        if (actualSize != null) {
+            actualWidth = actualSize.width;
+            actualHeight = actualSize.height;
+        }
+
+        return videoCapturer;
     }
 
     /**
@@ -117,10 +151,11 @@ public void onCameraSwitchError(String s) {
      * @param facingMode the facing of the requested video source such as
      * {@code user} and {@code environment}. If {@code null}, "user" is
      * presumed.
-     * @return a {@code VideoCapturer} satisfying the {@code facingMode} or
-     * {@code deviceId} constraint
+     * @return a pair containing the deviceId and {@code VideoCapturer} satisfying the {@code facingMode} or
+     * {@code deviceId} constraint, or null.
      */
-    private VideoCapturer createVideoCapturer(String deviceId, String facingMode) {
+    @Nullable
+    private Pair<String, VideoCapturer> createVideoCapturer(String deviceId, String facingMode) {
         String[] deviceNames = cameraEnumerator.getDeviceNames();
         List<String> failedDevices = new ArrayList<>();
 
@@ -139,7 +174,7 @@ private VideoCapturer createVideoCapturer(String deviceId, String facingMode) {
             if (videoCapturer != null) {
                 Log.d(TAG, message + " succeeded");
                 this.isFrontFacing = cameraEnumerator.isFrontFacing(cameraName);
-                return videoCapturer;
+                return new Pair(cameraName, videoCapturer);
             } else {
                 // fallback to facingMode
                 Log.d(TAG, message + " failed");
@@ -161,7 +196,7 @@ private VideoCapturer createVideoCapturer(String deviceId, String facingMode) {
             if (videoCapturer != null) {
                 Log.d(TAG, message + " succeeded");
                 this.isFrontFacing = cameraEnumerator.isFrontFacing(name);
-                return videoCapturer;
+                return new Pair(name, videoCapturer);
             } else {
                 Log.d(TAG, message + " failed");
                 failedDevices.add(name);
@@ -176,7 +211,7 @@ private VideoCapturer createVideoCapturer(String deviceId, String facingMode) {
                 if (videoCapturer != null) {
                     Log.d(TAG, message + " succeeded");
                     this.isFrontFacing = cameraEnumerator.isFrontFacing(name);
-                    return videoCapturer;
+                    return new Pair(name, videoCapturer);
                 } else {
                     Log.d(TAG, message + " failed");
                     failedDevices.add(name);

android/src/main/java/com/oney/WebRTCModule/GetUserMediaImpl.java

@@ -192,7 +192,7 @@ void getUserMedia(final ReadableMap constraints, final Callback successCallback,
             Log.d(TAG, "getUserMedia(video): " + videoConstraintsMap);
 
             CameraCaptureController cameraCaptureController =
-                    new CameraCaptureController(getCameraEnumerator(), videoConstraintsMap);
+                    new CameraCaptureController(reactContext.getCurrentActivity(), getCameraEnumerator(), videoConstraintsMap);
 
             videoTrack = createVideoTrack(cameraCaptureController);
         }

android/src/main/java/com/oney/WebRTCModule/ScreenCaptureController.java

@@ -41,9 +41,9 @@ public void onOrientationChanged(int orientation) {
                 DisplayMetrics displayMetrics = DisplayUtils.getDisplayMetrics((Activity) context);
                 final int width = displayMetrics.widthPixels;
                 final int height = displayMetrics.heightPixels;
-                if (width != ScreenCaptureController.this.width || height != ScreenCaptureController.this.height) {
-                    ScreenCaptureController.this.width = width;
-                    ScreenCaptureController.this.height = height;
+                if (width != ScreenCaptureController.this.actualWidth || height != ScreenCaptureController.this.actualHeight) {
+                    ScreenCaptureController.this.actualWidth = width;
+                    ScreenCaptureController.this.actualHeight = height;
 
                     // Pivot to the executor thread because videoCapturer.changeCaptureFormat runs in the main
                     // thread and may deadlock.

android/src/main/java/org/webrtc/Camera1Helper.java

@@ -0,0 +1,55 @@
+/*
+ * Copyright 2023-2024 LiveKit, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.webrtc;
+
+import androidx.annotation.Nullable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * A helper to access package-protected methods used in [Camera2Session]
+ * <p>
+ * Note: cameraId as used in the Camera1XXX classes refers to the index within the list of cameras.
+ *
+ * @suppress
+ */
+
+public class Camera1Helper {
+
+    public static int getCameraId(String deviceName) {
+        return Camera1Enumerator.getCameraIndex(deviceName);
+    }
+
+    @Nullable
+    public static List<CameraEnumerationAndroid.CaptureFormat> getSupportedFormats(int cameraId) {
+        return Camera1Enumerator.getSupportedFormats(cameraId);
+    }
+
+    public static Size findClosestCaptureFormat(int cameraId, int width, int height) {
+        List<CameraEnumerationAndroid.CaptureFormat> formats = getSupportedFormats(cameraId);
+
+        List<Size> sizes = new ArrayList<>();
+        if (formats != null) {
+            for (CameraEnumerationAndroid.CaptureFormat format : formats) {
+                sizes.add(new Size(format.width, format.height));
+            }
+        }
+
+        return CameraEnumerationAndroid.getClosestSupportedSize(sizes, width, height);
+    }
+}

android/src/main/java/org/webrtc/Camera2Helper.java

@@ -0,0 +1,51 @@
+/*
+ * Copyright 2023-2024 LiveKit, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.webrtc;
+
+import android.hardware.camera2.CameraManager;
+
+import androidx.annotation.Nullable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * A helper to access package-protected methods used in [Camera2Session]
+ * <p>
+ * Note: cameraId as used in the Camera2XXX classes refers to the id returned
+ * by [CameraManager.getCameraIdList].
+ */
+public class Camera2Helper {
+
+    @Nullable
+    public static List<CameraEnumerationAndroid.CaptureFormat> getSupportedFormats(CameraManager cameraManager, @Nullable String cameraId) {
+        return Camera2Enumerator.getSupportedFormats(cameraManager, cameraId);
+    }
+
+    public static Size findClosestCaptureFormat(CameraManager cameraManager, @Nullable String cameraId, int width, int height) {
+        List<CameraEnumerationAndroid.CaptureFormat> formats = getSupportedFormats(cameraManager, cameraId);
+
+        List<Size> sizes = new ArrayList<>();
+        if (formats != null) {
+            for (CameraEnumerationAndroid.CaptureFormat format : formats) {
+                sizes.add(new Size(format.width, format.height));
+            }
+        }
+
+        return CameraEnumerationAndroid.getClosestSupportedSize(sizes, width, height);
+    }
+}

examples/GumTestApp_macOS/App.js

@@ -17,7 +17,7 @@ import {
   StatusBar,
 } from 'react-native';
 import { Colors } from 'react-native/Libraries/NewAppScreen';
-import { mediaDevices, RTCView } from 'react-native-webrtc';
+import { mediaDevices, RTCView } from '@livekit/react-native-webrtc';
 
 const App: () => React$Node = () => {
   const [stream, setStream] = useState(null);