You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 86eb869
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: fern/assistants/background-speech-denoising.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -393,6 +393,8 @@ Fourier Denoising should never be used alone. It's designed to complement Smart
393
393
4. Monitor user feedback
394
394
5. A/B test different configurations
395
395
396
+
Denoising also applies when a simulation's AI tester plays [background sound](/observability/simulations-configure-ai-tester#background-sound), so the assistant hears the call the way it would in production.
Copy file name to clipboardExpand all lines: fern/observability/simulations-configure-ai-tester.mdx
+56-1Lines changed: 56 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,7 +25,7 @@ The available settings depend on whether you configure the AI tester in the Dash
25
25
| Area | Dashboard settings | Additional API settings |
26
26
| -- | -- | -- |
27
27
|**Scenario**| Name and intent | Overrides for the assistant or squad under test, webhook destinations, transcript and message inclusion, and recording inclusion |
28
-
|**Behavior**| Behavior and who starts first| First message, start-speaking plan, stop-speaking plan, duration limit, background sound, and background-speech denoising |
28
+
|**Behavior**| Behavior, who starts first, and [background sound](#background-sound)| First message, start-speaking plan, stop-speaking plan, duration limit, and background-speech denoising |
29
29
|**Model**| Provider and model | Temperature, maximum tokens, fallback models, tools, prompt caching, knowledge base, and model-specific reasoning |
30
30
|**Transcriber**| Provider, model, and language | Language detection and hints, endpoint delay, vocabulary, contextual hints, and fallback transcribers |
31
31
|**Voice**| Provider and voice ID | Speed, version, language, pronunciation dictionaries, chunking, formatting, and caching |
@@ -186,6 +186,61 @@ The available fields vary by provider. See [**Update Personality**](/api-referen
186
186
Editing a built-in **(Default)** personality in the Dashboard saves it as a new personality you own. The built-in personality remains unchanged.
187
187
</Note>
188
188
189
+
## Background sound
190
+
191
+
Background sound is an audio file that the AI tester plays throughout a voice simulation, mixed with its speech. Use it to test how the assistant or squad you're testing handles a caller in a noisy place. Chat simulations ignore background sound.
192
+
193
+
<Warning>
194
+
[GPT-Live](/gpt-live/overview) AI testers add background sound only while they're speaking, not during silence. They also don't play the **Office** preset, so choose another one.
195
+
</Warning>
196
+
197
+
In the Dashboard, open the **Personality** tab, expand **Advanced settings**, and turn on **Background sound**. Turning it on selects **White noise**. To change it, select another preset or **Custom URL**.
198
+
199
+
| Preset | Sound | API value |
200
+
| -- | -- | -- |
201
+
|**Office**| Office chatter and typing |`office`|
202
+
|**White noise**| Steady static |`white-noise`|
203
+
|**City**| Street traffic |`city-traffic`|
204
+
|**Crowd**| Many people talking |`crowd-conversation`|
205
+
|**Farm**| Chickens and roosters |`chickens-roosters`|
206
+
207
+
Office is the quietest preset, at about 30 dB below the AI tester's speech. The others play about 15 dB below it.
208
+
209
+
With **Custom URL**, enter a public HTTPS URL of up to 1,000 characters that points to an audio file of 25 MB or less. The file loops for the whole call. Use WAV when you can, because an MP3 file can leave a short gap at the loop point.
210
+
211
+
Through the API, set `backgroundSound` in the personality's `assistant` to a preset's API value or your own URL:
212
+
213
+
```json
214
+
{
215
+
"assistant": {
216
+
"backgroundSound": "white-noise"
217
+
}
218
+
}
219
+
```
220
+
221
+
Include the rest of the personality's assistant configuration in the same request, as described in [Configure the AI tester](#configure-the-ai-tester).
222
+
223
+
<Note>
224
+
The assistant or squad you're testing denoises incoming audio before it transcribes and records the call. Smart Denoising is on by default, so it removes most of the background sound, and the recording may sound clean. A clean recording is expected, because the simulation tests your production setup.
225
+
</Note>
226
+
227
+
### Hear the raw noise
228
+
229
+
To hear the raw noise, turn off denoising in the scenario's `targetOverrides`. Also set `backgroundDenoisingEnabled` to `false`, because this legacy flag takes precedence when the assistant or squad you're testing sets it:
230
+
231
+
```json
232
+
{
233
+
"targetOverrides": {
234
+
"backgroundDenoisingEnabled": false,
235
+
"backgroundSpeechDenoisingPlan": {
236
+
"smartDenoisingPlan": { "enabled": false }
237
+
}
238
+
}
239
+
}
240
+
```
241
+
242
+
If the assistant or squad you're testing enables [Fourier denoising](/documentation/assistants/conversation-behavior/background-speech-denoising#fourier-denoising-experimental), also set `backgroundSpeechDenoisingPlan.fourierDenoisingPlan.enabled` to `false`. Updating a scenario replaces its `targetOverrides`, so include any existing overrides you want to keep.
Copy file name to clipboardExpand all lines: fern/test/simulations-best-practices.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -108,6 +108,8 @@ This creates useful variation without changing the business requirement.
108
108
109
109
Build and debug a scenario in chat mode, then run important paths in voice mode before launch. Also use voice mode after changing the transcriber, voice, or turn-taking configuration.
110
110
111
+
- Add a noisy-caller scenario for important paths. Turn on [background sound](/observability/simulations-configure-ai-tester#background-sound) in the AI tester's personality to check that the agent still understands the caller and completes the task.
112
+
111
113
<Note>
112
114
Synthetic callers do not fully represent real callers, devices, or environments.
113
115
Supplement voice Simulations with manual calls and production-call review for
0 commit comments