Skip to content

Commit f014f69

Browse files
authored
Request microphone permission before audio capture starts (#1183)
Flutter counterpart of client-sdk-swift #1085, matching its final merged behavior. Builds on #1182. ## Why webrtc-sdk/webrtc#265 (first shipped in `m144.7559.12`) removed the blocking mic permission request from the AudioEngine device. The pre-enable check is now passive: it returns `kAudioEngineErrorInsufficientDevicePermission` (-9000) instead of prompting, so requesting permission is the SDK's job. flutter-webrtc still pins `144.7559.10`, so this is not load-bearing yet. It is harmless there, since `getUserMedia` in flutter-webrtc already prompts and the status is resolved before the device's blocking path runs. Once flutter-webrtc bumps past `.12` and the pin here follows, this is what keeps the current behavior. ## What Flutter already had flutter-webrtc's `getUserMedia` calls `AVCaptureDevice requestAccessForMediaType:` and waits for the answer, so every livekit_client mic path (publish, `restartTrack` on unmute, pre-connect audio) already prompted before the audio device saw the track. That part of #1085 needs no port. #1182 already maps -9000 to `TrackCreateException` for the direct ADM entry points (`setEngineAvailability`, `startLocalRecording`). ## What this adds The one behavior from #1085 that was missing: only prompt while the app can show the alert. - Native `ensureMicrophoneAccess` in `LiveKitPlugin.swift`: `authorized` passes, `denied`/`restricted` fail, `notDetermined` requests access. On iOS the request is only made while `UIApplication.shared.applicationState == .active`. An inactive or backgrounded app (locked screen, CallKit wake, app switcher) has the alert deferred by the system, and awaiting it would suspend `getUserMedia` and the `_publishRunner` behind it, blocking camera and screen share publishes for as long as the app stays there. Failing fast lets the next foreground attempt prompt normally. macOS can present the prompt regardless, so it always requests. No app extension concern here, the plugin is app-only. - The gate is skipped while engine input availability is disabled (`setEngineAvailability`, the CallKit flow), mirroring the same late fix in #1085: the audio device module defers opening input entirely and runs no permission check there, so gating would turn a working background connect into a `deviceAccessDenied` failure. The check reads the plugin's tracked availability value, so it also covers gating done natively before the Flutter engine exists. - `LocalTrack.createStream` calls it for `AudioCaptureOptions` on Apple platforms before `getUserMedia`. That is the Flutter choke point: `LocalAudioTrack.create()`, `restartTrack()` and `PreConnectAudioBuffer.startRecording()` all reach it. Since the prompt in Flutter happens at `getUserMedia` rather than at capture start, the gate sits in front of that instead of in `startCapture` as in Swift. - Failures surface as `TrackCreateException` through the `deviceAccessDenied` code introduced in #1182. - Docs for `withPreConnectAudio` and `PreConnectAudioBuffer.startRecording` now say permission is requested at recording start but only while the app is active, so callers running at app launch should request it up front (matching the final #1085 wording). `Native.setEngineAvailability` documents that permission is not requested there and must be granted before input availability is restored. ## Testing - `flutter analyze`, `flutter test`, `dart format --set-exit-if-changed`, `import_sorter --exit-if-changed` clean. - Unit tests cover `Native.ensureMicrophoneAccess` (no-op when unimplemented, propagates `deviceAccessDenied`). The `createStream` gate is behind `lkPlatformIsApple()` and not reachable from unit tests. - The example app builds for iOS (device SDK) and macOS with the change, re-verified after the rebase onto `main`. On-device run against a fresh install (first-launch prompt) and a CallKit background wake still to do. Refs CLT-3243, client-sdk-swift#1085
1 parent 8827c32 commit f014f69

11 files changed

Lines changed: 285 additions & 9 deletions
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
patch type="fixed" "iOS/macOS: request microphone permission before audio capture starts, failing fast with TrackCreateException while the app is not in the foreground"

.changes/preconnect-buffer-cleanup

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
patch type="fixed" "Pre-connect audio buffer returns to a reusable state when recording fails to start, instead of ignoring retries and leaking the agent timeout"

lib/src/core/room_preconnect.dart

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,12 @@ extension RoomPreConnect on Room {
4141
/// );
4242
/// ```
4343
///
44-
/// - Note: Ensure microphone permissions are granted early in your app
45-
/// lifecycle so pre-connect can start without additional prompts.
44+
/// - Note: Requires microphone permission. On iOS/macOS the SDK requests it
45+
/// when recording starts, but only while the app is active, so call this
46+
/// once the app has become active and the system prompt can appear. At app
47+
/// launch the app may not be active yet, so when calling this early,
48+
/// request permission up front (for example with the permission_handler
49+
/// package). Otherwise it throws a [TrackCreateException].
4650
/// - SeeAlso: [PreConnectAudioBuffer]
4751
Future<T> withPreConnectAudio<T>(
4852
Future<T> Function() operation, {

lib/src/preconnect/pre_connect_audio_buffer.dart

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -105,8 +105,9 @@ class PreConnectAudioBuffer {
105105
/// [agentReadyFuture] completes with an error and callers should [reset] the
106106
/// buffer.
107107
///
108-
/// Ensure microphone permissions are granted before calling this.
109-
/// Audio capture may fail without permissions.
108+
/// Requires microphone permission. On iOS/macOS it is requested here while
109+
/// the app is in the foreground. Throws a [TrackCreateException] when it is
110+
/// denied or cannot be requested (app not in the foreground).
110111
Future<void> startRecording({
111112
Duration timeout = const Duration(seconds: 20),
112113
}) async {
@@ -119,7 +120,14 @@ class PreConnectAudioBuffer {
119120
// Set up timeout for agent readiness
120121
_agentReadyManager.setTimer(timeout, timeoutReason: 'Agent did not become ready within timeout');
121122

122-
_localTrack = await LocalAudioTrack.create();
123+
try {
124+
_localTrack = await LocalAudioTrack.create();
125+
} catch (error) {
126+
logger.severe('[Preconnect audio] failed to create local track: $error');
127+
_notifyError(error);
128+
await stopRecording(withError: error);
129+
rethrow;
130+
}
123131
logger.fine('[Preconnect audio] created local track ${_localTrack!.mediaStreamTrack.id}');
124132

125133
final rendererId = Uuid().v4();
@@ -137,7 +145,7 @@ class PreConnectAudioBuffer {
137145
if (!result) {
138146
final error = StateError('Failed to start audio renderer ($result)');
139147
logger.severe('[Preconnect audio] $error');
140-
_onError?.call(error);
148+
_notifyError(error);
141149
await stopRecording(withError: error);
142150
await _localTrack?.stop();
143151
_localTrack = null;
@@ -149,7 +157,7 @@ class PreConnectAudioBuffer {
149157
_nativeRecordingStarted = lkPlatformSupportsExplicitAudioRecordingStart();
150158
} catch (error) {
151159
logger.severe('[Preconnect audio] failed to start local recording: $error');
152-
_onError?.call(error);
160+
_notifyError(error);
153161
await stopRecording(withError: error);
154162
await _localTrack?.stop();
155163
_localTrack = null;
@@ -183,7 +191,7 @@ class PreConnectAudioBuffer {
183191
_agentReadyManager.complete();
184192
} catch (error) {
185193
_agentReadyManager.completeError(error);
186-
_onError?.call(error);
194+
_notifyError(error);
187195
}
188196
},
189197
);
@@ -345,4 +353,15 @@ class PreConnectAudioBuffer {
345353
void setErrorHandler(PreConnectOnError? onError) {
346354
_onError = onError;
347355
}
356+
357+
/// Invokes the app-provided error callback without letting a throwing
358+
/// callback derail the failure path it is called from: cleanup must still
359+
/// run and the original error must stay the one callers see.
360+
void _notifyError(Object error) {
361+
try {
362+
_onError?.call(error);
363+
} catch (callbackError) {
364+
logger.warning('[Preconnect audio] onError callback threw: $callbackError');
365+
}
366+
}
348367
}

lib/src/support/native.dart

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,12 +266,39 @@ class Native {
266266
}
267267
}
268268

269+
/// Requests microphone permission before audio capture starts (iOS/macOS).
270+
///
271+
/// The WebRTC audio device only checks the permission and fails when it is
272+
/// missing, so the SDK requests it here. On iOS the prompt is only shown while
273+
/// the app is active. Throws a [PlatformException] with code
274+
/// `deviceAccessDenied` when permission is denied, restricted, or could not be
275+
/// requested. A no-op where the platform does not implement it, or while the
276+
/// engine's input availability is disabled via [setEngineAvailability] (the
277+
/// audio device defers opening input there, so no permission is needed yet).
278+
@internal
279+
static Future<void> ensureMicrophoneAccess() async {
280+
try {
281+
await channel.invokeMethod<void>('ensureMicrophoneAccess', <String, dynamic>{});
282+
} on PlatformException catch (error) {
283+
if (error.code == 'Unimplemented') return;
284+
rethrow;
285+
} on MissingPluginException {
286+
return;
287+
}
288+
}
289+
269290
/// Sets whether the WebRTC audio engine is allowed to run (iOS/macOS).
270291
///
271292
/// Unlike most methods in this class this deliberately does not swallow
272293
/// platform errors: a failed availability change means the engine may run
273294
/// outside the window the caller intended (e.g. CallKit's
274295
/// didActivate/didDeactivate), so the error must reach the caller.
296+
///
297+
/// Microphone permission is not requested here. A recording requested while
298+
/// input was unavailable is honored on re-enable, but the audio device only
299+
/// passively checks permission at that point, so it must be granted before
300+
/// input availability is restored. Otherwise this throws a
301+
/// [PlatformException] with code `deviceAccessDenied`.
275302
@internal
276303
static Future<void> setEngineAvailability({
277304
required bool isInputAvailable,

lib/src/support/reusable_completer.dart

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,14 @@ class ReusableCompleter<T> {
7272
return false;
7373
}
7474

75+
if (!_hasPendingListener) {
76+
// No one can observe this error: [future] creates a fresh completer once
77+
// completed, so delivering it would only surface an unhandled async
78+
// error. Mark completed silently, like reset() and dispose() do.
79+
_markCompletedWithoutNotify();
80+
return true;
81+
}
82+
7583
_completeCurrent((completer) => completer.completeError(error, stackTrace));
7684
return true;
7785
}

lib/src/track/local/local.dart

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,17 +16,20 @@ import 'dart:async';
1616

1717
import 'package:flutter/foundation.dart' show kIsWeb;
1818
import 'package:flutter/material.dart';
19+
import 'package:flutter/services.dart' show PlatformException;
1920

2021
import 'package:flutter_webrtc/flutter_webrtc.dart' as rtc;
2122
import 'package:meta/meta.dart';
2223

24+
import '../../audio/audio_engine_error.dart';
2325
import '../../audio/audio_frame_capture.dart';
2426
import '../../events.dart';
2527
import '../../exceptions.dart';
2628
import '../../extensions.dart';
2729
import '../../internal/events.dart';
2830
import '../../logger.dart';
2931
import '../../participant/remote.dart';
32+
import '../../support/native.dart';
3033
import '../../support/platform.dart';
3134
import '../../types/other.dart';
3235
import '../options.dart';
@@ -254,6 +257,20 @@ abstract class LocalTrack extends Track {
254257
'video': options is VideoCaptureOptions ? options.toMediaConstraintsMap() : false,
255258
};
256259

260+
if (options is AudioCaptureOptions && lkPlatformIsApple()) {
261+
// The WebRTC audio device only checks microphone permission and fails
262+
// when it is missing, so the SDK requests it before opening the mic. On
263+
// iOS this fails fast while the app is not in the foreground instead of
264+
// suspending getUserMedia (and the publish queue behind it) on a prompt
265+
// the system cannot show yet.
266+
try {
267+
await Native.ensureMicrophoneAccess();
268+
} on PlatformException catch (error) {
269+
throw audioEngineExceptionFrom(error) ??
270+
TrackCreateException(error.message ?? 'Microphone permission is not granted');
271+
}
272+
}
273+
257274
final rtc.MediaStream stream;
258275
if (options is ScreenShareCaptureOptions) {
259276
if (kIsWeb) {

shared_swift/LiveKitPlugin.swift

Lines changed: 64 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -542,6 +542,63 @@ public class LiveKitPlugin: NSObject, FlutterPlugin {
542542
}
543543
}
544544

545+
// MARK: - Microphone permission
546+
547+
/// Ensures microphone access is granted before audio capture starts.
548+
///
549+
/// The WebRTC audio device does not request microphone permission itself. It
550+
/// only checks it and fails with kAudioEngineErrorInsufficientDevicePermission,
551+
/// so requesting is the SDK's job. On iOS the request is only made while the
552+
/// app is active: while inactive or in the background the system defers the
553+
/// alert, and waiting on it would suspend the caller, and the publish queue
554+
/// behind it, for as long as the app stays there. Failing fast lets the next
555+
/// attempt prompt normally. macOS can present the prompt regardless.
556+
///
557+
/// Method channel handlers run on the main thread, which UIApplication needs.
558+
public func handleEnsureMicrophoneAccess(result: @escaping FlutterResult) {
559+
// With engine input availability disabled (the CallKit flow, see
560+
// setEngineAvailability) the audio device module defers opening input
561+
// entirely and runs no permission check, so gating here would turn a
562+
// working background connect into a deviceAccessDenied failure. The
563+
// last-set value is tracked by both the channel and native static
564+
// paths, so it covers gating done before the Flutter engine exists.
565+
LiveKitPlugin.engineAvailabilityLock.lock()
566+
let pendingAvailability = LiveKitPlugin.pendingEngineAvailability
567+
LiveKitPlugin.engineAvailabilityLock.unlock()
568+
if let pendingAvailability, !pendingAvailability.isInputAvailable.boolValue {
569+
result(nil)
570+
return
571+
}
572+
573+
let denied = { (message: String) in
574+
result(FlutterError(code: LiveKitPlugin.deviceAccessDeniedErrorCode, message: message, details: nil))
575+
}
576+
switch AVCaptureDevice.authorizationStatus(for: .audio) {
577+
case .authorized:
578+
result(nil)
579+
case .notDetermined:
580+
#if !os(macOS)
581+
guard UIApplication.shared.applicationState == .active else {
582+
denied("Microphone permission could not be requested because the app is not in the foreground. Request it while the app is active before enabling recording.")
583+
return
584+
}
585+
#endif
586+
AVCaptureDevice.requestAccess(for: .audio) { granted in
587+
DispatchQueue.main.async {
588+
if granted {
589+
result(nil)
590+
} else {
591+
denied("Microphone permission was denied.")
592+
}
593+
}
594+
}
595+
case .denied, .restricted:
596+
denied("Microphone permission is not granted.")
597+
@unknown default:
598+
denied("Microphone permission is not granted.")
599+
}
600+
}
601+
545602
// MARK: - Microphone mute mode
546603

547604
static func muteModeString(_ mode: RTCAudioEngineMuteMode) -> String {
@@ -768,6 +825,8 @@ public class LiveKitPlugin: NSObject, FlutterPlugin {
768825
handleStopLocalRecording(result: result)
769826
case "setEngineAvailability":
770827
handleSetEngineAvailability(args: args, result: result)
828+
case "ensureMicrophoneAccess":
829+
handleEnsureMicrophoneAccess(result: result)
771830
case "setAudioProcessingOptions":
772831
handleSetAudioProcessingOptions(args: args, result: result)
773832
case "getAudioProcessingState":
@@ -800,14 +859,18 @@ extension LiveKitPlugin {
800859
static let kAudioEngineErrorInsufficientDevicePermission = -9000
801860
static let kAudioEngineErrorAudioSessionInvalidCategory = -9001
802861

862+
/// FlutterError code for missing microphone permission. Dart maps it to
863+
/// TrackCreateException (see audio_engine_error.dart).
864+
static let deviceAccessDeniedErrorCode = "deviceAccessDenied"
865+
803866
/// Maps a non-zero audio device module result to a `FlutterError` whose code
804867
/// the Dart side can act on. Codes with a known cause get their own error
805868
/// code, mirroring client-sdk-swift's `checkAdmResult`. Anything else falls
806869
/// back to `fallbackCode` with the raw value in the message.
807870
static func flutterError(forAudioEngineResult result: Int, fallbackCode: String) -> FlutterError {
808871
switch result {
809872
case kAudioEngineErrorInsufficientDevicePermission:
810-
return FlutterError(code: "deviceAccessDenied",
873+
return FlutterError(code: deviceAccessDeniedErrorCode,
811874
message: "Microphone permission is not granted (audio engine error \(result))",
812875
details: result)
813876
case kAudioEngineErrorAudioSessionInvalidCategory:

test/audio/audio_session_test.dart

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -876,6 +876,37 @@ void main() {
876876
});
877877
});
878878

879+
group('Native.ensureMicrophoneAccess', () {
880+
test('is a no-op when the platform does not implement it', () async {
881+
TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger.setMockMethodCallHandler(
882+
Native.channel,
883+
(call) async => throw PlatformException(code: 'Unimplemented'),
884+
);
885+
await expectLater(Native.ensureMicrophoneAccess(), completes);
886+
887+
TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger.setMockMethodCallHandler(
888+
Native.channel,
889+
null,
890+
);
891+
await expectLater(Native.ensureMicrophoneAccess(), completes);
892+
});
893+
894+
test('propagates a denied permission so callers can map it', () async {
895+
TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger.setMockMethodCallHandler(
896+
Native.channel,
897+
(call) async {
898+
expect(call.method, 'ensureMicrophoneAccess');
899+
throw PlatformException(code: audioEngineErrorCodeDeviceAccessDenied, message: 'denied');
900+
},
901+
);
902+
903+
await expectLater(
904+
Native.ensureMicrophoneAccess(),
905+
throwsA(isA<PlatformException>().having((error) => error.code, 'code', audioEngineErrorCodeDeviceAccessDenied)),
906+
);
907+
});
908+
});
909+
879910
group('audioEngineExceptionFrom', () {
880911
test('maps missing microphone permission to TrackCreateException', () {
881912
final error = audioEngineExceptionFrom(

0 commit comments

Comments
 (0)