Remove blocking mic permission request from AudioEngine pre-enable check (m144) - #265
Merged
hiroshihorie merged 2 commits intoJul 9, 2026
Merged
Conversation
EnsureMicrophonePermissionSync blocked the WebRTC worker thread on a semaphore until the user answered the mic permission dialog, which never appears when the app is woken in the background (e.g. a CallKit call), hanging setEngineAvailability indefinitely (#815). Replace it with a passive IsMicrophonePermissionAuthorized check that returns kAudioEngineErrorInsufficientDevicePermission instead of blocking. Requesting permission is moved to the SDK, gated to the foreground. Forward-port of #204 (originally on m137_release) to m144_release.
The old comment said it 'attempts to acquire' permission and had an 'erorr' typo. The check is now a passive authorization read; requesting permission is the SDK's job.
cloudwebrtc
pushed a commit
that referenced
this pull request
Jul 16, 2026
…eck (#265) Forward-port of #204 (originally targeting `m137_release`) to `m144_release`, which is the line currently shipped by the LiveKit Swift SDK (`main` pins `144.7559.10`). The blocking call was never ported forward, so it still lives on `m144_release`. ## Problem `EnsureMicrophonePermissionSync()` blocks the WebRTC worker thread on a semaphore (`dispatch_semaphore_wait(…, DISPATCH_TIME_FOREVER)`) until the user answers the mic permission dialog. When the app is woken in the background (e.g. an incoming CallKit call), no dialog can appear, so `setEngineAvailability` hangs indefinitely. Ref: livekit/client-sdk-swift#815 ## Change Replace the blocking request with a passive `IsMicrophonePermissionAuthorized()` check in the AudioEngine pre-enable path. When the mic is not authorized it returns `kAudioEngineErrorInsufficientDevicePermission` instead of blocking. The (dead) `IsMicrophonePermissionGranted()` is renamed into that check; it had no other callers. Only the `.notDetermined` case changes behavior: previously it prompted and blocked the worker thread; now it returns an error immediately. `authorized` and `denied`/`restricted` are unchanged.
This was referenced Aug 12, 2026
hiroshihorie
added a commit
to livekit/client-sdk-swift
that referenced
this pull request
Aug 31, 2026
webrtc-sdk/webrtc#265 removed the blocking mic permission request from the `AudioEngineDevice` pre-enable path. The ADM now does a passive authorization check and returns `kAudioEngineErrorInsufficientDevicePermission` (-9000) instead of prompting, so requesting permission is the SDK's job. That change shipped in `144.7559.12`, and main is now on **`150.7871.01`** (#1103, merged into this branch). So this is not precautionary: without it, microphone publishing fails on every fresh install, because `.notDetermined` is the state of a newly installed app and nothing prompts. Supersedes #1047. Credit to @silviudeac for the original foreground-gate helper there. ## Where the check goes `LocalAudioTrack.startCapture()` is the one async choke point every microphone track path reaches, and it is what actually opens ADM input via `AudioManager.startLocalRecording`. Publishing arrives through `Track.start()`, pre-connect recording through `LocalAudioTrackRecorder.start()`. ```swift override func startCapture() async throws { let needsMicrophonePermission = await RTC.run { !AudioManager.shared.isManualRenderingMode && AudioManager.shared.engineAvailability.isInputAvailable } if needsMicrophonePermission { try await LiveKitSDK.ensureMicrophoneAccessForRecording() } ... } ``` Manual rendering mode never opens the microphone, and disabled input availability (the CallKit flow) defers opening it entirely, so neither needs permission. Both flags are read on the `@RTC` executor (#1101) since they wait on WebRTC's worker thread. It is the only gated entry point. Everything else either cannot prompt or should not. ## Call path coverage | Entry point | Gated | Behavior when not authorized | | --- | --- | --- | | `LocalParticipant.setMicrophone(enabled: true)` | Yes, via `_publish` and `Track.start()` | Prompts if active, else throws `.deviceAccessDenied` | | `LocalParticipant.set(source: .microphone, enabled: true)` | Yes, same path | Prompts if active, else throws `.deviceAccessDenied` | | `LocalParticipant.publish(audioTrack:)` with an app-created track | Yes, via `Track.start()` | Prompts if active, else throws `.deviceAccessDenied` | | `Track.start()` called directly on a `LocalAudioTrack` | Yes | Prompts if active, else throws `.deviceAccessDenied` | | `Room.withPreConnectAudio { }` | Yes, via `PreConnectAudioBuffer` and `LocalAudioTrackRecorder.start()` | Prompts if active, else throws `.deviceAccessDenied` | | `PreConnectAudioBuffer.startRecording()` | Yes, same path | Prompts if active, else throws `.deviceAccessDenied` | | `LocalAudioTrackRecorder.start()` standalone | Yes | Prompts if active, else throws `.deviceAccessDenied` | | Manual rendering mode (`setManualRenderingMode(true)` then `setMicrophone(enabled: true)`) | No, never touches the mic | Publishes app audio with no permission needed | | `AudioManager.setRecordingAlwaysPreparedMode(true)` | No, prewarming is not user-initiated | Fails fast, ADM returns -9000 mapped to `.deviceAccessDenied` | | `AudioManager.startLocalRecording(_:)` | No, synchronous so it cannot prompt | Fails fast, -9000 mapped | | Publish while `setEngineAvailability(.none)` (CallKit, #815) | Exempt, input never opens | Publishes with input deferred; permission applies when availability is restored | | `AudioManager.setEngineAvailability(_:)` restoring input | No, synchronous so it cannot prompt | Fails fast, -9000 mapped | | `LocalAudioTrack.mute()` and `unmute()` | Not applicable | `_unmute` only restarts video tracks, so mic capture is never reopened | | WebRTC's implicit `InitRecording` on sender attach | Already gated | `_publish` calls `Track.start()` before `addTransceiver` | | `AudioMixRecorder` | Not applicable | No ADM input path | The ungated `AudioManager` calls already surface -9000 as `.deviceAccessDenied` through `checkAdmResult`, and their doc comments now point at `LiveKitSDK.ensureDeviceAccess(for:)` for callers driving the ADM directly. ## Reading the application state The SDK only prompts while the app can actually present the alert. `UIApplication.shared` cannot be referenced here: it does not compile under `APPLICATION_EXTENSION_API_ONLY=YES`, which CI exercises, and consumers build this module into broadcast upload extensions via `LKSampleHandler`. Annotating the reader with `@available(iOSApplicationExtension, unavailable)` compiles, but availability propagates to callers and `startCapture` has to stay extension-available, so it cannot be reached from there. So the state is read through the Objective-C runtime, behind an app-extension check. This is the same shape Google's GoogleUtilities uses in `GULAppDelegateSwizzler.sharedApplication`, which underpins Firebase: extension guard first, then `respondsToSelector:` before a dynamically resolved `sharedApplication`. That precedent ships in a large share of App Store apps, so the dynamic lookup is not an App Review concern. The gate is `== .active`. An inactive app (locked screen, call banner, app switcher, or launch before the scene activates) has the alert deferred rather than presented, and `requestAccess` has no cancellation-aware continuation, so waiting there would suspend the caller for as long as the app stays inactive. Since `startCapture` runs inside `Participant._publishSerialRunner`, that suspension would also block camera and screen share publishing. Failing fast avoids it, and the next attempt prompts normally. The gate applies only to iOS-family devices: Mac Catalyst is excluded (like native macOS, the system presents the mic dialog regardless of frontmost state), matching the `targetEnvironment` exclusion `CameraCapturer` uses. ## Review fixes From @pblazej's review: - Manual rendering mode is no longer gated. `Docs/audio.md` documents it as publishing app audio without touching the microphone, so it must not require permission. This restores a condition the pre-#793 implementation had. - The gate narrowed from `!= .background` to `== .active`, per above. A timeout was considered and rejected, since it would still hold the publish serial runner for the length of the timeout before failing. - `setRecordingAlwaysPreparedMode` is no longer gated. `LocalMedia.observeDevices()` calls it from `init`, so gating it raised a permission dialog on view-model construction and silently lost the prewarm when it failed. - The `republishAllTracks()` concern is resolved by merging main: #1008 rewrote it with a per-track `do/catch` that records the first error and rethrows only after the loop, so camera and screen share are republished even when audio fails. Round 2, from a deeper review pass: - The gate also exempts disabled input availability, restoring the CallKit `setEngineAvailability(.none)` flow where the ADM skips its permission check and defers input (verified against `m150_release`). Both flag reads moved onto the `@RTC` executor per #1101. - Mac Catalyst is excluded from the foreground gate, per above. - Doc corrections: `withPreConnectAudio` no longer recommends calling at app launch (the scene is typically still inactive there), and `setRecordingAlwaysPreparedMode`'s throw is conditioned on the ADM reporting the missing permission (its passive check is skipped in some mute modes). A follow-up branch makes the permission wait cancellation-aware so an unanswered prompt cannot wedge `stop`/`unpublish`/`disconnect`; it will be a separate PR on top of this one. ## Verification - `swift build` and `swift build -Xswiftc -application-extension` pass - `xcodebuild -destination 'generic/platform=iOS Simulator' APPLICATION_EXTENSION_API_ONLY=YES` exits 0, mirroring the CI matrix leg - `AudioManagerAdmResultTests` passes, `swiftlint` clean on the touched files Two caveats for reviewers: - **No new test coverage.** The gate depends on process and application state that is not reachable from a unit test without a seam. Happy to add one if wanted. - **Residual suspension.** If the app is active, the alert is presented, and the user then backgrounds without answering, the request stays suspended until they return and answer. That self-heals, unlike the inactive case this PR removes. ## Follow-up `Docs/audio.md` should mention that microphone permission is now requested by the SDK at capture start, and that apps wanting the prompt earlier should call `LiveKitSDK.ensureDeviceAccess(for:)`. Refs #815, CLT-3243
hiroshihorie
added a commit
to livekit/client-sdk-flutter
that referenced
this pull request
Sep 1, 2026
webrtc-sdk/webrtc#265 (m144.7559.12 and later) removed the blocking mic permission request from the AudioEngine device. It now only checks the status and fails with kAudioEngineErrorInsufficientDevicePermission, so requesting permission is the SDK's job. The native plugin gains ensureMicrophoneAccess: authorized passes, denied or restricted fails, and notDetermined requests access, on iOS only while the app is active. An inactive or backgrounded app has the alert deferred by the system, and waiting on it would suspend getUserMedia and the publish queue behind it, so it fails fast instead and the next foreground attempt prompts normally. LocalTrack.createStream calls it for audio on Apple platforms before getUserMedia, which covers publishing, restartTrack on unmute, and pre-connect audio. Failures surface as TrackCreateException through the existing audio engine error mapping.
hiroshihorie
added a commit
to livekit/client-sdk-flutter
that referenced
this pull request
Sep 1, 2026
Flutter counterpart of client-sdk-swift #1085, matching its final merged behavior. Builds on #1182. ## Why webrtc-sdk/webrtc#265 (first shipped in `m144.7559.12`) removed the blocking mic permission request from the AudioEngine device. The pre-enable check is now passive: it returns `kAudioEngineErrorInsufficientDevicePermission` (-9000) instead of prompting, so requesting permission is the SDK's job. flutter-webrtc still pins `144.7559.10`, so this is not load-bearing yet. It is harmless there, since `getUserMedia` in flutter-webrtc already prompts and the status is resolved before the device's blocking path runs. Once flutter-webrtc bumps past `.12` and the pin here follows, this is what keeps the current behavior. ## What Flutter already had flutter-webrtc's `getUserMedia` calls `AVCaptureDevice requestAccessForMediaType:` and waits for the answer, so every livekit_client mic path (publish, `restartTrack` on unmute, pre-connect audio) already prompted before the audio device saw the track. That part of #1085 needs no port. #1182 already maps -9000 to `TrackCreateException` for the direct ADM entry points (`setEngineAvailability`, `startLocalRecording`). ## What this adds The one behavior from #1085 that was missing: only prompt while the app can show the alert. - Native `ensureMicrophoneAccess` in `LiveKitPlugin.swift`: `authorized` passes, `denied`/`restricted` fail, `notDetermined` requests access. On iOS the request is only made while `UIApplication.shared.applicationState == .active`. An inactive or backgrounded app (locked screen, CallKit wake, app switcher) has the alert deferred by the system, and awaiting it would suspend `getUserMedia` and the `_publishRunner` behind it, blocking camera and screen share publishes for as long as the app stays there. Failing fast lets the next foreground attempt prompt normally. macOS can present the prompt regardless, so it always requests. No app extension concern here, the plugin is app-only. - The gate is skipped while engine input availability is disabled (`setEngineAvailability`, the CallKit flow), mirroring the same late fix in #1085: the audio device module defers opening input entirely and runs no permission check there, so gating would turn a working background connect into a `deviceAccessDenied` failure. The check reads the plugin's tracked availability value, so it also covers gating done natively before the Flutter engine exists. - `LocalTrack.createStream` calls it for `AudioCaptureOptions` on Apple platforms before `getUserMedia`. That is the Flutter choke point: `LocalAudioTrack.create()`, `restartTrack()` and `PreConnectAudioBuffer.startRecording()` all reach it. Since the prompt in Flutter happens at `getUserMedia` rather than at capture start, the gate sits in front of that instead of in `startCapture` as in Swift. - Failures surface as `TrackCreateException` through the `deviceAccessDenied` code introduced in #1182. - Docs for `withPreConnectAudio` and `PreConnectAudioBuffer.startRecording` now say permission is requested at recording start but only while the app is active, so callers running at app launch should request it up front (matching the final #1085 wording). `Native.setEngineAvailability` documents that permission is not requested there and must be granted before input availability is restored. ## Testing - `flutter analyze`, `flutter test`, `dart format --set-exit-if-changed`, `import_sorter --exit-if-changed` clean. - Unit tests cover `Native.ensureMicrophoneAccess` (no-op when unimplemented, propagates `deviceAccessDenied`). The `createStream` gate is behind `lkPlatformIsApple()` and not reachable from unit tests. - The example app builds for iOS (device SDK) and macOS with the change, re-verified after the rebase onto `main`. On-device run against a fresh install (first-launch prompt) and a CallKit background wake still to do. Refs CLT-3243, client-sdk-swift#1085
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Forward-port of #204 (originally targeting
m137_release) tom144_release, which is the line currently shipped by the LiveKit Swift SDK (mainpins144.7559.10). The blocking call was never ported forward, so it still lives onm144_release.Problem
EnsureMicrophonePermissionSync()blocks the WebRTC worker thread on a semaphore (dispatch_semaphore_wait(…, DISPATCH_TIME_FOREVER)) until the user answers the mic permission dialog. When the app is woken in the background (e.g. an incoming CallKit call), no dialog can appear, sosetEngineAvailabilityhangs indefinitely.Ref: livekit/client-sdk-swift#815
Change
Replace the blocking request with a passive
IsMicrophonePermissionAuthorized()check in the AudioEngine pre-enable path. When the mic is not authorized it returnskAudioEngineErrorInsufficientDevicePermissioninstead of blocking. The (dead)IsMicrophonePermissionGranted()is renamed into that check; it had no other callers.Only the
.notDeterminedcase changes behavior: previously it prompted and blocked the worker thread; now it returns an error immediately.authorizedanddenied/restrictedare unchanged.