Repository navigation
Conversation
MultiSpeakerAdapter signals a suppressed background-speaker segment with a FINAL_TRANSCRIPT whose text is empty. Diarizing providers report interims without a speaker id, so a background interim bypasses suppression and is buffered by AudioRecognition, while the attributed final is suppressed. AudioRecognition returned on the empty final before clearing _audio_interim_transcript. The stale background words stayed in _current_transcript, where they counted toward the min_words interruption gate, and commit_user_turn appended them to the user turn. Drop the buffered interim and preflight text when the final is empty. The event is not forwarded: the transcript transport cannot carry a wire-level clear (an empty text stream has zero data chunks, which clients ignore), so a client-visible clear needs protocol support and is out of scope here.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| self._audio_interim_transcript = "" | ||
| self._audio_preflight_transcript = "" |
There was a problem hiding this comment.
π΄ Background final erases primary speaker words
When a background final follows a primary speaker interim, _audio_interim_transcript loses the primary speaker's words. The adapter sends the empty final without identifying which interim it clears. A manual commit then omits those words.
Learn more
The adapter forwards unattributed interim results but replaces a suppressed background final with an empty final that has no speaker or segment identifier. Recognition holds only one interim and one preflight buffer. If the latest interim belongs to the primary speaker while a background final arrives, clearing those buffers drops legitimate unfinalized speech. The same loss occurs for a primary speaker preflight transcript. The adapter cannot tell recognition which transcript was suppressed.
Example: The primary speaker says βbook a ticket,β producing an interim. A background speaker's attributed final arrives next and is suppressed. The empty final deletes βbook a ticket,β so an immediate manual commit returns no ticket request instead of preserving those words.
Recommended fix: Carry a segment identity or ordering information through the adapter and clear only interim/preflight state belonging to the suppressed segment. Validate interleaved speaker events as well as the sequential events already tested.
Was this helpful? React with π or π to provide feedback.
| self._audio_interim_transcript = "" | ||
| self._audio_preflight_transcript = "" | ||
| return |
There was a problem hiding this comment.
π‘ Manual commit waits after empty final
When the previous nonempty final is over 0.5 seconds old, _last_final_transcript_time remains stale after an empty final. Manual commit then discards its completion signal and waits until timeout without a later final.
Learn more
Manual commit uses the last-final timestamp to decide whether it must wait for a new STT result. A suppressed segment produces a real final event, but the empty-final path does not update that timestamp. After a quiet interval, the commit clears the completion event that the empty final just set and waits for an additional final or for the timeout. The regression test overwrites the timestamp and skips this path.
Example: A primary final arrives at t=0; a background interim and its empty final arrive at t=1. A manual commit at t=1.1 clears _final_transcript_received and waits for the configured transcript timeout, although the suppressed segment has already finalized.
Recommended fix: Decide whether an empty final can satisfy the manual-commit freshness check, and update the relevant timestamp when it does. Exercise commit without overwriting the timestamp in the test.
Was this helpful? React with π or π to provide feedback.
This branch has not been deployed
1 participant
fix(voice): clear the stale interim on an empty final transcript
What breaks
MultiSpeakerAdaptersignals "clear the interim" by emitting an emptyFINAL_TRANSCRIPT(documented inmulti_speaker_adapter.py) when a background speaker's attributed final is suppressed.AudioRecognition._process_stt_eventsets_final_transcript_receivedand returns on an empty final β before clearing_audio_interim_transcriptand before anything reaches the hooks.With a diarizing STT that sends speaker-less interims (Deepgram sets
speaker=Noneon every interim), the background speaker's interim is stored before its attributed final is suppressed, and the clearing event is dropped:_audio_interim_transcriptuntil the next non-empty transcript,_current_transcript'smin_wordsinterruption gate,commit_user_turncommits the background words into the user turn β defeatingsuppress_background_speaker=True.Fix
On an empty
FINAL_TRANSCRIPT, clear_audio_interim_transcriptand_audio_preflight_transcript, then return._final_transcript_receivedis still set; every non-empty path (accumulation, interrupt handling, turn limits) is untouched.Scope note: the client-visible interim display is not fixed by this change. The transcript transport carries text as chunked streams, and an empty final is a zero-chunk stream that all reference clients ignore (no wire-level "clear" exists today); a client clear needs protocol/client support and is tracked as a follow-up. This fix restores the internal state (interruption gate +
commit_user_turn), which is wheresuppress_background_speakerwas actually defeated.Tests
tests/test_multi_speaker_interim_clear.pyfeeds the adapter's documented sequence (finalhelloS0 β speaker-less interimbackground chatterβ empty final):assert ar._audio_interim_transcript == ''fails ('background chatter' != '') andcommit_user_turnreturns'hello background chatter'instead of'hello'._current_transcript == 'hello', commit returns'hello'; a gate-free variant covers an empty final as the first event.3 passed; multi-speaker+delay group 42, audio-recognition 504 (identical at base),
test_agent_session110,test_room_io60; ruff clean. The fix went through two independent review rounds β the first rejected a client-clear attempt (a stream sink change that would have finalized the suppressed background text, and then a zero-chunk clear that reference clients ignore), which is why this revision intentionally stops at the internal state and makes no client-side claim.