You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1295d73
Browse filesBrowse the repository at this point in the historyBrowse files
TEST-273 docs: document transcriber and endpointing simulation latency limits and the unmeasured flag
Add the transcriber and endpointing metrics to simulation latency limits, and
explain the per-turn unmeasured list that decides which turns count. Merge
after VapiAI/vapi#25252, which adds the metrics.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: fern/assistants/model-intelligence/understanding-latency.mdx
+8-2Lines changed: 8 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,8 +37,14 @@ Keep these points in mind when you read the numbers:
37
37
-**Model and voice latency include more than the provider.**`modelLatency` includes knowledge base retrieval and request routing before the provider request. `voiceLatency` includes collecting enough of the reply to send the first voice request, as set by `voice.chunkPlan` (`minCharacters` and punctuation boundaries), and any sample-rate conversion after the provider returns audio.
38
38
-**Vapi measures on its servers.** None of the fields include network time to or from the caller, or playback on the caller's device.
39
39
-**Only replies to the caller are recorded.** Turns where the assistant didn't speak, for example because the caller interrupted first, have no entry. When the assistant speaks first, its first message has no entry either. When it waits for the caller (`firstMessageMode` set to `assistant-waits-for-user`), the first message is recorded as a reply, with `endpointingLatency` as `0`.
40
-
-**`0` can mean "not measured".** A stage that didn't run or wasn't measured on a turn is recorded as `0` rather than left out. For example, when a turn's first audio is a static tool message rather than model output, `modelLatency` and `voiceLatency` are `0`. When `transcriberLatency` is `0` because it wasn't measured, `turnLatency` leaves that interval out too.
41
-
-**Averages count every turn.** The call's `modelLatencyAverage`, `voiceLatencyAverage`, and other averages divide by every entry, including those zeros. Simulation latency limits ignore a `0` for `modelLatency` and `voiceLatency`, so a simulation's model or voice result can be higher than the call's average for the same call.
40
+
-**`unmeasured` tells a missing value from a real `0`.** A stage that didn't run or wasn't measured on a turn is recorded as `0`, and the entry's `unmeasured` array names it. A `0` for a component that isn't listed is a real 0 ms. The array is left out when every component was measured. Entries recorded before `unmeasured` existed don't have it, so a `0` there can mean either. A component is listed when:
41
+
-`transcriber`: voice activity detection saw no speech for the turn, the sample was too stale to use, or the transcript committed before voice activity detection saw the end of speech, so the value was measured from the start of speech.
42
+
-`endpointing`: no caller transcript was sent to the model that turn, for example the first message or a reply to a tool result.
43
+
-`model`: the model sent no output token that turn, for example when the first audio is a static tool message or a guardrail held the reply.
44
+
-`voice`: there was no model output to time the voice from.
45
+
46
+
When `transcriberLatency` is unmeasured, `turnLatency` leaves that interval out too.
47
+
-**Averages count every turn.** The call's `modelLatencyAverage`, `voiceLatencyAverage`, and other averages divide by every entry, including those zeros. Simulation latency limits leave out turns that list the metric in `unmeasured`, and a `0` for `modelLatency` or `voiceLatency` on older entries, so a simulation's result can be higher than the call's average for the same call.
@@ -330,8 +331,10 @@ Each metric reads one field of the per-turn latency that every call records, mea
330
331
|`turn`|`turnLatency`| Time from the end of the AI tester's speech to the assistant starting its reply |
331
332
|`model`|`modelLatency`| Time from the start of the turn's model stage, including knowledge base retrieval, to the model's first output token |
332
333
|`voice`|`voiceLatency`| Time from the model's first output token to the first reply audio being ready, including the voice provider and any sample-rate conversion |
334
+
|`transcriber`|`transcriberLatency`| Time from the AI tester's last voice activity, normally the end of its speech, to the transcriber returning the words |
335
+
|`endpointing`|`endpointingLatency`| Time from the transcriber returning the words to Vapi deciding the AI tester's turn is over |
333
336
334
-
`model` and `voice` happen inside `turn`, so they don't add up to it. Turns that recorded `0` for `model` or `voice`didn't measure it and are left out of those metrics. The AI tester speaks with clean synthetic audio, so transcription, endpointing, and turn latency can run faster than with real callers.
337
+
`turn` is `transcriber` plus `endpointing` plus the time until the reply starts, and `model` and `voice` happen inside that last part, so the metrics don't add up to `turn`. A turn counts toward a metric unless its per-turn `unmeasured` list names that metric: a real 0 ms counts, and a turn that didn't measure the component is left out. `turn` counts every turn. See [`unmeasured`](/assistants/model-intelligence/understanding-latency#per-turn-latency-on-a-call) for when each component is listed. The AI tester speaks with clean synthetic audio, so transcription, endpointing, and turn latency can run faster than with real callers.
335
338
336
339
The aggregation reduces the call's turns to one value: `mean`, `median`, `p95`, or `max`. `p95` uses the nearest-rank method, so on a call with fewer than 20 turns it equals `max`. Measured values are rounded to the nearest millisecond before they are compared with the limit.
337
340
@@ -401,7 +404,7 @@ For a transient scenario, set the same `targetOverrides` on the inline `scenario
401
404
| The assistant never asks whether the AI tester is still there | Simulations remove `customer.speech.timeout` hooks. To test them, set `targetOverrides.hooks` on the scenario. See [**Hooks and call endings**](#hooks-and-call-endings). |
402
405
| A voice simulation's assistant call ends as `customer-ended-call`| The AI tester hung up first, which matches a real caller hanging up. |
403
406
| A tool looks up the wrong caller | Set `customer.number` in the scenario's variable values. See [**Simulate a call from a specific phone number**](#simulate-a-call-from-a-specific-phone-number). |
404
-
| A latency limit fails without a measured value | No turn in the call measured that metric. Confirm the run used voice mode and that the conversation completed at least one exchange. |
407
+
| A latency limit fails without a measured value | No turn in the call measured that metric. Confirm the run used voice mode and that the conversation completed at least one exchange. For `transcriber` and `endpointing`, check each turn's `unmeasured` list in the target call's `artifact.performanceMetrics.turnLatencies`. |
405
408
| Latency limits show as skipped | Check whether the run used chat mode or the assistant under test uses GPT-Live. Both skip latency limits. |
0 commit comments