Skip to content

Commit 5c43bd3

Browse files
authored
fix(hooks): allow VAD streamInsert/streamStop during streaming (#1196)
## Description `useVAD` exposed `stream`, `streamInsert`, and `streamStop` all through `useModuleFactory.runForward`, which rejects calls while `isGenerating === true`. Once `stream()` was in flight, every `streamInsert(chunk)` rejected with *"model is currently generating"* — making the streaming API unusable in practice. Add a `runSideChannel` primitive to `useModuleFactory` that only checks `isReady` (never `isGenerating` and never flips it). Streaming hooks now have two regimes: - Gated (`runForward`): `forward`, `stream` — one at a time. - Side-channel (`runSideChannel`): `streamInsert`, `streamStop` — allowed *during* a running stream. `useVAD.streamInsert` / `streamStop` now use `runSideChannel`. `useTextToSpeech` and `useSpeechToText` aren't affected (they avoid `useModuleFactory` entirely for their side-channel methods); they're candidates for a follow-up migration onto the new primitive. ### Introduces a breaking change? - [ ] Yes - [x] No ### Type of change - [x] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [ ] iOS - [x] Android ### Testing instructions End-to-end on the `apps/speech` Voice Activity Detection screen: 1. Build and install the speech app on an Android device (mic required; the screen disables Start on simulators). 2. Tap **Start VAD Stream** multiple times very quick, speak and pause. With this PR: `SPEAKING` / `SILENT` events appear in the log as expected. Without it (`git checkout HEAD~1 -- packages/react-native-executorch/src/hooks/{useModuleFactory.ts,natural_language_processing/useVAD.ts}` and reload Metro): `streamInsert` rejects on every 100 ms audio chunk and no SPEAKING transition fires and error occurs. ### Related issues Fixes #1173
1 parent 778c507 commit 5c43bd3

2 files changed

Lines changed: 28 additions & 18 deletions

File tree

‎packages/react-native-executorch/src/hooks/natural_language_processing/useVAD.ts‎

Lines changed: 16 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,20 @@ import { useModuleFactory } from '../useModuleFactory';
99
* @returns Ready to use VAD model.
1010
*/
1111
export const useVAD = ({ model, preventLoad = false }: VADProps): VADType => {
12-
const { error, isReady, isGenerating, downloadProgress, runForward } =
13-
useModuleFactory({
14-
factory: (config, onProgress) =>
15-
VADModule.fromModelName(config, onProgress),
16-
config: model,
17-
deps: [model.modelName, model.modelSource],
18-
preventLoad,
19-
});
12+
const {
13+
error,
14+
isReady,
15+
isGenerating,
16+
downloadProgress,
17+
runForward,
18+
runSideChannel,
19+
} = useModuleFactory({
20+
factory: (config, onProgress) =>
21+
VADModule.fromModelName(config, onProgress),
22+
config: model,
23+
deps: [model.modelName, model.modelSource],
24+
preventLoad,
25+
});
2026

2127
const forward = (waveform: Float32Array) =>
2228
runForward((inst) => inst.forward(waveform));
@@ -25,16 +31,9 @@ export const useVAD = ({ model, preventLoad = false }: VADProps): VADType => {
2531
runForward((inst) => inst.stream(input));
2632

2733
const streamInsert = (waveform: Float32Array) =>
28-
runForward((inst) => {
29-
inst.streamInsert(waveform);
30-
return Promise.resolve();
31-
});
34+
runSideChannel((inst) => inst.streamInsert(waveform));
3235

33-
const streamStop = () =>
34-
runForward((inst) => {
35-
inst.streamStop();
36-
return Promise.resolve();
37-
});
36+
const streamStop = () => runSideChannel((inst) => inst.streamStop());
3837

3938
return {
4039
error,

‎packages/react-native-executorch/src/hooks/useModuleFactory.ts‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ type RunOnFrame<M> = M extends { runOnFrame: infer R } ? R : never;
1414
* not-loaded / already-generating guards so individual hooks only need to
1515
* define their typed `forward` wrapper.
1616
* @param props - Options object containing the factory function, config, deps array, and optional preventLoad flag.
17-
* @returns An object with error, isReady, isGenerating, downloadProgress, runForward, instance, and runOnFrame.
17+
* @returns An object with error, isReady, isGenerating, downloadProgress, runForward, runSideChannel, instance, and runOnFrame.
1818
* @internal
1919
*/
2020
export function useModuleFactory<M extends Deletable, Config>({
@@ -89,6 +89,16 @@ export function useModuleFactory<M extends Deletable, Config>({
8989
}
9090
};
9191

92+
// Non-gating call path for streaming modules: only checks `isReady`, never
93+
// `isGenerating`. Lets side-channel methods (e.g. `streamInsert` buffer push,
94+
// `streamStop` interrupt signal) run while `stream`/`forward` is in flight.
95+
const runSideChannel = <R>(fn: (instance: M) => R): R => {
96+
if (!isReady || !instance) {
97+
throw new RnExecutorchError(RnExecutorchErrorCode.ModuleNotLoaded);
98+
}
99+
return fn(instance);
100+
};
101+
92102
const runOnFrame = useMemo(
93103
() =>
94104
instance && 'runOnFrame' in instance
@@ -103,6 +113,7 @@ export function useModuleFactory<M extends Deletable, Config>({
103113
isGenerating,
104114
downloadProgress,
105115
runForward,
116+
runSideChannel,
106117
instance,
107118
runOnFrame,
108119
};

0 commit comments

Comments
 (0)