Skip to content

Commit c26ce3c

Browse files
authored
feat(livekit): trace end-of-utterance waits (#2806)
Add `eou_wait` spans to show the time between user speech ending and LiveKit accepting the turn as complete. Capture separate waits when speech resumes, including pauses spent awaiting transcription. ```text user_turn 1.0–4.0s ├─ user_speech 1.0–2.0s ├─ eou_wait outcome=user_resumed 2.0–2.3s ├─ user_speech 2.3–3.5s └─ eou_wait outcome=committed 3.5–4.0s ```
1 parent bfe5f37 commit c26ce3c

38 files changed

Lines changed: 1672 additions & 124 deletions

‎.changeset/livekit-eou-wait.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"braintrust": minor
3+
---
4+
5+
feat(livekit): trace end-of-utterance waits

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-audio.span-tree.json‎

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-audio.span-tree.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ span_tree:
1111
}
1212
}
1313
metadata: {
14-
"audio.schema.version": "3.0.0",
14+
"audio.schema.version": "3.1.0",
1515
"audio.segments": {
1616
"part-0": {
1717
"attachment": {
@@ -197,6 +197,7 @@ span_tree:
197197
}
198198
└── livekit.eou_wait [task]
199199
metadata: {
200+
"contrib.livekit.eou.canonical_omitted": "unobserved_speech_end",
200201
"contrib.livekit.eou.endpointing_delay": 0,
201202
"contrib.livekit.eou.not_committed_count": 0,
202203
"contrib.livekit.eou.outcome": "committed",

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-diarized.span-tree.json‎

Lines changed: 8 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-diarized.span-tree.txt‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ span_tree:
1111
}
1212
}
1313
metadata: {
14-
"audio.schema.version": "3.0.0",
14+
"audio.schema.version": "3.1.0",
1515
"audio.segments": {
1616
"part-0": {
1717
"attachment": {
@@ -405,14 +405,19 @@ span_tree:
405405
│ │ "testRunId": "<run:1>",
406406
│ │ "turn.id": "<span:1>"
407407
│ │ }
408-
│ ├── livekit.eou_wait [task]
408+
│ ├── eou_wait [task]
409409
│ │ metadata: {
410+
│ │ "audio.source_id": "session-input",
410411
│ │ "contrib.livekit.eou.endpointing_delay": 0,
412+
│ │ "contrib.livekit.eou.native_end_unix_ms": 0,
413+
│ │ "contrib.livekit.eou.native_start_unix_ms": 0,
411414
│ │ "contrib.livekit.eou.not_committed_count": 0,
412415
│ │ "contrib.livekit.eou.outcome": "committed",
413416
│ │ "contrib.livekit.eou.rearm_count": 0,
414417
│ │ "contrib.livekit.eou.source": "manual",
415418
│ │ "contrib.livekit.eou.wait_duration": 0,
419+
│ │ "eou.outcome": "committed",
420+
│ │ "eou.speech_end_source": "vad",
416421
│ │ "scenario": "livekit-instrumentation",
417422
│ │ "testRunId": "<run:1>",
418423
│ │ "turn.id": "<span:1>"
@@ -439,6 +444,7 @@ span_tree:
439444
│ }
440445
│ └── livekit.eou_wait [task]
441446
│ metadata: {
447+
│ "contrib.livekit.eou.canonical_omitted": "unobserved_speech_end",
442448
│ "contrib.livekit.eou.endpointing_delay": 0,
443449
│ "contrib.livekit.eou.not_committed_count": 0,
444450
│ "contrib.livekit.eou.outcome": "committed",

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-disabled.span-tree.json‎

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-disabled.span-tree.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
span_tree:
22
└── session [task]
33
metadata: {
4-
"audio.schema.version": "3.0.0",
4+
"audio.schema.version": "3.1.0",
55
"audio.sources": {
66
"agent-output": {
77
"label": "Agent audio"
@@ -152,6 +152,7 @@ span_tree:
152152
}
153153
└── livekit.eou_wait [task]
154154
metadata: {
155+
"contrib.livekit.eou.canonical_omitted": "unobserved_speech_end",
155156
"contrib.livekit.eou.endpointing_delay": 0,
156157
"contrib.livekit.eou.not_committed_count": 0,
157158
"contrib.livekit.eou.outcome": "committed",

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-interrupted.span-tree.json‎

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-interrupted.span-tree.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ span_tree:
1111
}
1212
}
1313
metadata: {
14-
"audio.schema.version": "3.0.0",
14+
"audio.schema.version": "3.1.0",
1515
"audio.segments": {
1616
"part-0": {
1717
"attachment": {
@@ -538,6 +538,7 @@ span_tree:
538538
}
539539
└── livekit.eou_wait [task]
540540
metadata: {
541+
"contrib.livekit.eou.canonical_omitted": "unobserved_speech_end",
541542
"contrib.livekit.eou.endpointing_delay": 0,
542543
"contrib.livekit.eou.not_committed_count": 0,
543544
"contrib.livekit.eou.outcome": "committed",

‎e2e/scenarios/livekit-instrumentation/__snapshots__/livekit-v1-latest-audio.span-tree.json‎

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)