|
| 1 | +# Code Review: iOS Multichannel Audio Support (#3746) |
| 2 | + |
| 3 | +## 🔍 Executive Summary |
| 4 | + |
| 5 | +**Status:** ⚠️ **Needs safety hardening before merge** |
| 6 | + |
| 7 | +After comparing this implementation against the macOS audio backend and analyzing the design, I've identified **several race conditions and bounds-checking gaps** that could cause crashes or undefined behavior with edge cases—particularly when using actual multichannel USB-C/Lightning interfaces on iPad. |
| 8 | + |
| 9 | +The **core logic is sound**, but defensive protections are missing that exist in the macOS implementation. |
| 10 | + |
| 11 | +--- |
| 12 | + |
| 13 | +## 🔴 **Critical Issues:** |
| 14 | + |
| 15 | +### **1. Race Condition: Unprotected Channel Selection in Audio Callback** |
| 16 | + |
| 17 | +**Location:** `processBufferList()`, lines ~145-150 |
| 18 | + |
| 19 | +The audio callback reads `iNumChan`, `iLeftCh`, and `iRightCh` **without holding the audio mutex**: |
| 20 | + |
| 21 | +```cpp |
| 22 | +const int iNumChan = pSound->buffer.mNumberChannels; // ⚠️ UNPROTECTED |
| 23 | +const int iLeftCh = pSound->iSelInputLeftChannel; // ⚠️ UNPROTECTED |
| 24 | +const int iRightCh = pSound->iSelInputRightChannel; // ⚠️ UNPROTECTED |
| 25 | +``` |
| 26 | + |
| 27 | +Meanwhile, `SwitchDevice()` → `UpdateInputChannelInfo()` can modify these values from another thread (the main UI thread). If this occurs between reading `iLeftCh` and using it in the loop, the code reads: |
| 28 | + |
| 29 | +```cpp |
| 30 | +pData[iNumChan * i + iLeftCh] // Could be out-of-bounds if iLeftCh >= iNumChan |
| 31 | +``` |
| 32 | + |
| 33 | +**macOS Comparison:** The macOS callback (`sound.cpp:1026-1049`) **does** use mutex protection: |
| 34 | +```cpp |
| 35 | +QMutexLocker locker ( &pSound->MutexAudioProcessCallback ); |
| 36 | +``` |
| 37 | +
|
| 38 | +**Impact:** On a fast device switch (e.g., unplugging a 4-channel interface, then plugging in 2-channel), this could read garbage audio or crash. |
| 39 | +
|
| 40 | +**Fix:** Wrap the channel reads with the existing mutex lock. |
| 41 | +
|
| 42 | +--- |
| 43 | +
|
| 44 | +### **2. Missing Bounds Check Before Indexing** |
| 45 | +
|
| 46 | +**Location:** `processBufferList()`, lines ~149-152 |
| 47 | +
|
| 48 | +```cpp |
| 49 | +for ( int i = 0; i < pSound->iCoreAudioBufferSizeMono; i++ ) |
| 50 | +{ |
| 51 | + pSound->vecsTmpAudioSndCrdStereo[2 * i] = (short) ( pData[iNumChan * i + iLeftCh] * _MAXSHORT ); |
| 52 | + pSound->vecsTmpAudioSndCrdStereo[2 * i + 1] = (short) ( pData[iNumChan * i + iRightCh] * _MAXSHORT ); |
| 53 | +} |
| 54 | +``` |
| 55 | + |
| 56 | +**Problem:** No validation that `iLeftCh < iNumChan` and `iRightCh < iNumChan` before using them as indices. |
| 57 | + |
| 58 | +**macOS Comparison:** Macros explicitly bounds-check: |
| 59 | +```cpp |
| 60 | +if ( ( iSelInBufferLeft >= 0 ) && ( iSelInBufferLeft < static_cast<int> ( inInputData->mNumberBuffers ) ) && ... ) |
| 61 | +{ |
| 62 | + // safe to use iSelInBufferLeft |
| 63 | +} |
| 64 | +else |
| 65 | +{ |
| 66 | + // clear buffer and bail out |
| 67 | + pSound->vecsTmpAudioSndCrdStereo.Reset ( 0 ); |
| 68 | +} |
| 69 | +``` |
| 70 | + |
| 71 | +**Impact:** If `iNumChan = 2` but `iRightCh = 5` (due to race condition or device list corruption), the interleaved buffer read will access uninitialized memory. |
| 72 | + |
| 73 | +**Fix:** Add defensive clamping before the loop: |
| 74 | +```cpp |
| 75 | +if ( iLeftCh >= iNumChan ) iLeftCh = 0; |
| 76 | +if ( iRightCh >= iNumChan ) iRightCh = ( iNumChan > 1 ) ? 1 : 0; |
| 77 | +``` |
| 78 | + |
| 79 | +--- |
| 80 | + |
| 81 | +### **3. Silent Failure in Device Port Lookup** |
| 82 | + |
| 83 | +**Location:** `SwitchDevice()`, lines ~406-412 |
| 84 | + |
| 85 | +```cpp |
| 86 | +if ( iPortIdx < availableInputs.count ) |
| 87 | +{ |
| 88 | + [sessionInstance setPreferredInput:availableInputs[iPortIdx] error:&error]; |
| 89 | +} |
| 90 | +// Falls back silently if iPortIdx is out of range—no warning logged |
| 91 | +``` |
| 92 | + |
| 93 | +**Problem:** If the device list changes between `GetAvailableInOutDevices()` and `SwitchDevice()` (e.g., user unplugs interface), the stored index becomes stale. The function silently falls back to the default input with **no diagnostic output**. |
| 94 | + |
| 95 | +**macOS handles this:** The macOS code enumerates devices fresh on each device switch and validates thoroughly. |
| 96 | + |
| 97 | +**Impact:** Silent audio misconfiguration—user thinks they're using the external interface but the app reverted to built-in mic without any indication. |
| 98 | + |
| 99 | +**Fix:** Add explicit warning log: |
| 100 | +```cpp |
| 101 | +else |
| 102 | +{ |
| 103 | + qWarning() << "SwitchDevice: Port index" << iPortIdx << "out of range." |
| 104 | + << "Device list may have changed. Falling back to system default."; |
| 105 | + [sessionInstance setPreferredInput:nil error:&error]; |
| 106 | +} |
| 107 | +``` |
| 108 | + |
| 109 | +--- |
| 110 | + |
| 111 | +### **4. Unchecked Return from `maximumInputNumberOfChannels`** |
| 112 | + |
| 113 | +**Location:** `SwitchDevice()`, lines ~414-418 |
| 114 | + |
| 115 | +```cpp |
| 116 | +const NSInteger iMaxChannels = [sessionInstance maximumInputNumberOfChannels]; |
| 117 | +[sessionInstance setPreferredInputNumberOfChannels:qBound ( 1, iMaxChannels, MAX_NUM_IN_OUT_CHANNELS ) error:&error]; |
| 118 | +``` |
| 119 | + |
| 120 | +**Problem:** If `iMaxChannels` returns 0, negative, or an invalid value (AVFoundation edge cases do happen), `qBound()` will silently clamp it. No warning is logged. |
| 121 | + |
| 122 | +**Impact:** The code proceeds with a clamped channel count that might not match the device's actual capability, leading to silent audio truncation. |
| 123 | + |
| 124 | +**Fix:** Validate and warn: |
| 125 | +```cpp |
| 126 | +if ( iMaxChannels > 0 ) |
| 127 | +{ |
| 128 | + [sessionInstance setPreferredInputNumberOfChannels:qBound ( static_cast<NSInteger> ( 1 ), iMaxChannels, static_cast<NSInteger> ( MAX_NUM_IN_OUT_CHANNELS ) ) |
| 129 | + error:&error]; |
| 130 | +} |
| 131 | +else |
| 132 | +{ |
| 133 | + qWarning() << "SwitchDevice: AVAudioSession.maximumInputNumberOfChannels returned" << iMaxChannels |
| 134 | + << "-- this is unexpected. Proceeding with default 2 channels."; |
| 135 | +} |
| 136 | +``` |
| 137 | + |
| 138 | +--- |
| 139 | + |
| 140 | +### **5. Missing Error Logging in Device Switch** |
| 141 | + |
| 142 | +**Location:** `SwitchDevice()` entire function |
| 143 | + |
| 144 | +The function silently ignores `NSError* error` after all AVAudioSession operations: |
| 145 | + |
| 146 | +```cpp |
| 147 | +[sessionInstance setPreferredInput:... error:&error]; // error ignored |
| 148 | +[sessionInstance setPreferredInputNumberOfChannels:... error:&error]; // error ignored |
| 149 | +``` |
| 150 | + |
| 151 | +**Problem:** If AVAudioSession rejects the device switch (e.g., permission denied, device unavailable), the error is swallowed with no logging. |
| 152 | + |
| 153 | +**macOS Comparison:** macOS logs device notifications and errors via callbacks. |
| 154 | + |
| 155 | +**Impact:** Silent failure—user selects a device that fails to activate, sees no indication, and audio breaks. |
| 156 | + |
| 157 | +**Fix:** Check and log errors: |
| 158 | +```cpp |
| 159 | +if ( error ) |
| 160 | +{ |
| 161 | + qWarning() << "SwitchDevice: AVAudioSession error:" << QString::fromNSString ( error.localizedDescription ); |
| 162 | +} |
| 163 | +``` |
| 164 | + |
| 165 | +--- |
| 166 | + |
| 167 | +## 🟡 **Design Concerns:** |
| 168 | + |
| 169 | +### **6. Changed Device List Indexing Semantics (Fragile but Functional)** |
| 170 | + |
| 171 | +**Old behavior:** Index 0 = System Default, Index 1 = Built-in Mic (hardcoded) |
| 172 | +**New behavior:** Index 0 = System Default, Index 1-N = All enumerated ports |
| 173 | + |
| 174 | +The new approach is better, but the index-1 offset is brittle: |
| 175 | + |
| 176 | +```cpp |
| 177 | +const NSUInteger iPortIdx = static_cast<NSUInteger> ( iDriverIdx - 1 ); |
| 178 | +``` |
| 179 | + |
| 180 | +**Risk:** If UI code or older settings persist with the old indexing scheme, the offset is wrong. Consider adding a comment clarifying the new scheme and incrementing a device-config version number if you have one. |
| 181 | + |
| 182 | +--- |
| 183 | + |
| 184 | +### **7. No Validation of `UpdateInputChannelInfo()` Results** |
| 185 | + |
| 186 | +**Location:** `UpdateInputChannelInfo()`, lines ~417+ |
| 187 | + |
| 188 | +```cpp |
| 189 | +AVAudioSessionPortDescription* inputPort = sessionInstance.currentRoute.inputs.firstObject; |
| 190 | +NSArray<AVAudioSessionChannelDescription*>* channels = inputPort.channels; |
| 191 | + |
| 192 | +for ( int i = 0; i < iNumInChan; i++ ) |
| 193 | +{ |
| 194 | + QString strChanName = QString ( "Channel %1" ).arg ( i + 1 ); |
| 195 | + |
| 196 | + if ( channels && ( i < channels.count ) && channels[i].channelName.length > 0 ) |
| 197 | + { |
| 198 | + strChanName = QString::fromNSString ( channels[i].channelName ); |
| 199 | + } |
| 200 | + |
| 201 | + sChannelNamesInput[i] = QString ( "%1: %2" ).arg ( i + 1 ).arg ( strChanName ); |
| 202 | +} |
| 203 | +``` |
| 204 | + |
| 205 | +**Problem:** If `currentRoute.inputs.firstObject` is null (no input device configured), channel names silently default to "Channel 1", "Channel 2", etc. This could mask serious device configuration failures. |
| 206 | + |
| 207 | +**Suggestion:** Add a debug log to catch this scenario: |
| 208 | +```cpp |
| 209 | +if ( !inputPort ) |
| 210 | +{ |
| 211 | + qDebug() << "UpdateInputChannelInfo: No input port in current audio route. " |
| 212 | + << "Using generic channel names."; |
| 213 | +} |
| 214 | +``` |
| 215 | + |
| 216 | +--- |
| 217 | + |
| 218 | +## ✅ **What Works Well:** |
| 219 | + |
| 220 | +- ✓ Separate input/output `AudioStreamBasicDescription` configs is architecturally correct |
| 221 | +- ✓ Channel name formatting matches macOS conventions |
| 222 | +- ✓ `UpdateInputChannelInfo()` properly re-clamps selections when device changes |
| 223 | +- ✓ The use of `qBound()` for clamping is safe **after validation** |
| 224 | +- ✓ Loop over `availableInputs` is the right approach for enumerating ports |
| 225 | + |
| 226 | +--- |
| 227 | + |
| 228 | +## 📋 **Recommended Actions Before Merge:** |
| 229 | + |
| 230 | +### **Tier 1: Critical (Security/Stability)** |
| 231 | + |
| 232 | +1. **Add mutex protection** in `processBufferList()` when reading channel selection variables |
| 233 | +2. **Add bounds checks** before indexing with `iLeftCh` and `iRightCh` |
| 234 | +3. **Add error logging** in `SwitchDevice()` for all AVAudioSession failures |
| 235 | +4. **Validate `maximumInputNumberOfChannels` result** and warn if unexpected |
| 236 | + |
| 237 | +### **Tier 2: Important (User Experience)** |
| 238 | + |
| 239 | +5. **Add diagnostic logging** if port index goes out of range |
| 240 | +6. **Add debug logging** if `UpdateInputChannelInfo()` finds no input port |
| 241 | +7. **Document the new device list indexing scheme** in code comments |
| 242 | + |
| 243 | +### **Tier 3: Testing (Hardware Validation)** |
| 244 | + |
| 245 | +8. **Build and test on actual iPad with Xcode debugger** |
| 246 | +9. **Test device switching scenarios:** |
| 247 | + - Plug in USB-C audio interface (4 channels) → select channel 3 |
| 248 | + - Unplug interface mid-session → verify graceful fallback |
| 249 | + - Switch between built-in mic, headset, and external interface |
| 250 | + - Test edge case: interface that reports 0 channels (shouldn't happen, but...) |
| 251 | + |
| 252 | +--- |
| 253 | + |
| 254 | +## 🔗 **Comparison to macOS Implementation** |
| 255 | + |
| 256 | +The macOS backend (`src/sound/coreaudio-mac/sound.cpp`) has: |
| 257 | +- Explicit size checks before buffer access (line ~1029) |
| 258 | +- Full device enumeration on each switch (not index-based) |
| 259 | +- Channel mapping logic with buffer-level indirection (`iSelInBufferLeft`, etc.) |
| 260 | +- Comprehensive error handling in device notifications |
| 261 | + |
| 262 | +The iOS code is simpler (fewer devices, no enumeration complexity) but needs the same **defensive rigor**. |
| 263 | + |
| 264 | +--- |
| 265 | + |
| 266 | +## 🎯 **Summary** |
| 267 | + |
| 268 | +This PR brings valuable functionality to iOS Jamulus—multichannel USB/Lightning audio support is important. However, the implementation has **threading safety gaps** and **error handling blind spots** that the macOS backend (rightfully) doesn't have. |
| 269 | + |
| 270 | +With the fixes above (especially Tier 1), this becomes a solid, production-ready feature. **Testing on real hardware is non-negotiable** before merge. |
| 271 | + |
| 272 | +Happy to help implement any of these fixes or discuss alternatives! |
0 commit comments