You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 656ce74
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: fern/assistants/background-speech-denoising.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -393,6 +393,8 @@ Fourier Denoising should never be used alone. It's designed to complement Smart
393
393
4. Monitor user feedback
394
394
5. A/B test different configurations
395
395
396
+
Denoising also applies when a simulation's AI tester plays [background sound](/observability/simulations-configure-ai-tester#background-sound), so the assistant hears the call the way it would in production.
Copy file name to clipboardExpand all lines: fern/observability/simulations-configure-ai-tester.mdx
+54-2Lines changed: 54 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,7 +25,7 @@ The available settings depend on whether you configure the AI tester in the Dash
25
25
| Area | Dashboard settings | Additional API settings |
26
26
| -- | -- | -- |
27
27
|**Scenario**| Name and intent | Overrides for the assistant or squad under test, webhook destinations, transcript and message inclusion, and recording inclusion |
28
-
|**Behavior**| Behavior and who starts first| First message, start-speaking plan, stop-speaking plan, duration limit, background sound, and background-speech denoising |
28
+
|**Behavior**| Behavior, who starts first, and [background sound](#background-sound)| First message, start-speaking plan, stop-speaking plan, duration limit, and background-speech denoising |
29
29
|**Model**| Provider and model | Temperature, maximum tokens, fallback models, tools, prompt caching, knowledge base, and model-specific reasoning |
30
30
|**Transcriber**| Provider, model, and language | Language detection and hints, endpoint delay, vocabulary, contextual hints, and fallback transcribers |
31
31
|**Voice**| Provider and voice ID | Speed, version, language, pronunciation dictionaries, chunking, formatting, and caching |
@@ -186,6 +186,58 @@ The available fields vary by provider. See [**Update Personality**](/api-referen
186
186
Editing a built-in **(Default)** personality in the Dashboard saves it as a new personality you own. The built-in personality remains unchanged.
187
187
</Note>
188
188
189
+
## Background sound
190
+
191
+
Add background sound to a voice simulation to test how the assistant or squad you're testing handles a caller in a noisy place. The AI tester plays the sound under its own speech. Chat simulations ignore background sound.
192
+
193
+
In the Dashboard, open the **Personality** tab, expand **Advanced settings**, and turn on **Background sound**. Then select a preset or **Custom URL**.
194
+
195
+
| Preset | Sound | URL |
196
+
| -- | -- | -- |
197
+
|**Office**| Office chatter and typing |`https://vapi-public-media.s3.us-west-2.amazonaws.com/assets/audio/simulation-background-sounds/v1/office.wav`|
|**City**| Street traffic |`https://vapi-public-media.s3.us-west-2.amazonaws.com/assets/audio/simulation-background-sounds/v1/city-traffic.wav`|
200
+
|**Crowd**| Many people talking |`https://vapi-public-media.s3.us-west-2.amazonaws.com/assets/audio/simulation-background-sounds/v1/crowd-conversation.wav`|
201
+
|**TV**| A television playing |`https://vapi-public-media.s3.us-west-2.amazonaws.com/assets/audio/simulation-background-sounds/v1/tv.wav`|
202
+
|**Farm**| Chickens and roosters |`https://vapi-public-media.s3.us-west-2.amazonaws.com/assets/audio/simulation-background-sounds/v1/chickens-roosters.wav`|
203
+
204
+
With **Custom URL**, enter a public HTTPS URL to an audio file of 25 MB or less. The file loops for the whole call. Use WAV when you can, because an MP3 file can leave a short gap at the loop point.
205
+
206
+
Through the API, set `backgroundSound` in the personality's `assistant` to a preset URL or your own URL:
Include the rest of the personality's assistant configuration in the same request, as described in [Configure the AI tester](#configure-the-ai-tester).
217
+
218
+
<Note>
219
+
The assistant or squad you're testing denoises incoming audio before it transcribes and records the call. Smart denoising is on by default, so it removes most of the background sound and the recording can sound clean. This is expected, because the simulation tests your production setup.
220
+
221
+
To hear the raw noise, turn off denoising in the scenario's `targetOverrides`. Set `backgroundDenoisingEnabled` as well, because this legacy flag takes precedence when the target sets it:
222
+
223
+
```json
224
+
{
225
+
"targetOverrides": {
226
+
"backgroundDenoisingEnabled": false,
227
+
"backgroundSpeechDenoisingPlan": {
228
+
"smartDenoisingPlan": { "enabled": false }
229
+
}
230
+
}
231
+
}
232
+
```
233
+
234
+
If the target enables [Fourier denoising](/documentation/assistants/conversation-behavior/background-speech-denoising#fourier-denoising-experimental), also set `backgroundSpeechDenoisingPlan.fourierDenoisingPlan.enabled` to `false`. Updating a scenario replaces its `targetOverrides`, so include any existing overrides you want to keep.
235
+
</Note>
236
+
237
+
<Warning>
238
+
GPT Live (realtime) AI testers add background sound only while they're speaking, not during silence. They also don't play the built-in `office` value, so use the hosted **Office** preset URL instead.
Copy file name to clipboardExpand all lines: fern/test/simulations-best-practices.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -108,6 +108,8 @@ This creates useful variation without changing the business requirement.
108
108
109
109
Build and debug a scenario in chat mode, then run important paths in voice mode before launch. Also use voice mode after changing the transcriber, voice, or turn-taking configuration.
110
110
111
+
- Add a noisy-caller scenario for important paths. Turn on [background sound](/observability/simulations-configure-ai-tester#background-sound) in the AI tester's personality to check that the agent still understands the caller and completes the task.
112
+
111
113
<Note>
112
114
Synthetic callers do not fully represent real callers, devices, or environments.
113
115
Supplement voice Simulations with manual calls and production-call review for
0 commit comments