Skip to content

Commit 86eb869

Browse files
TEST-61 Document background sound for simulations
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent d72df74 commit 86eb869

3 files changed

Lines changed: 60 additions & 1 deletion

File tree

‎fern/assistants/background-speech-denoising.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -393,6 +393,8 @@ Fourier Denoising should never be used alone. It's designed to complement Smart
393393
4. Monitor user feedback
394394
5. A/B test different configurations
395395

396+
Denoising also applies when a simulation's AI tester plays [background sound](/observability/simulations-configure-ai-tester#background-sound), so the assistant hears the call the way it would in production.
397+
396398
## Troubleshooting fourier denoising
397399

398400
<AccordionGroup>

‎fern/observability/simulations-configure-ai-tester.mdx‎

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ The available settings depend on whether you configure the AI tester in the Dash
2525
| Area | Dashboard settings | Additional API settings |
2626
| -- | -- | -- |
2727
| **Scenario** | Name and intent | Overrides for the assistant or squad under test, webhook destinations, transcript and message inclusion, and recording inclusion |
28-
| **Behavior** | Behavior and who starts first | First message, start-speaking plan, stop-speaking plan, duration limit, background sound, and background-speech denoising |
28+
| **Behavior** | Behavior, who starts first, and [background sound](#background-sound) | First message, start-speaking plan, stop-speaking plan, duration limit, and background-speech denoising |
2929
| **Model** | Provider and model | Temperature, maximum tokens, fallback models, tools, prompt caching, knowledge base, and model-specific reasoning |
3030
| **Transcriber** | Provider, model, and language | Language detection and hints, endpoint delay, vocabulary, contextual hints, and fallback transcribers |
3131
| **Voice** | Provider and voice ID | Speed, version, language, pronunciation dictionaries, chunking, formatting, and caching |
@@ -186,6 +186,61 @@ The available fields vary by provider. See [**Update Personality**](/api-referen
186186
Editing a built-in **(Default)** personality in the Dashboard saves it as a new personality you own. The built-in personality remains unchanged.
187187
</Note>
188188

189+
## Background sound
190+
191+
Background sound is an audio file that the AI tester plays throughout a voice simulation, mixed with its speech. Use it to test how the assistant or squad you're testing handles a caller in a noisy place. Chat simulations ignore background sound.
192+
193+
<Warning>
194+
[GPT-Live](/gpt-live/overview) AI testers add background sound only while they're speaking, not during silence. They also don't play the **Office** preset, so choose another one.
195+
</Warning>
196+
197+
In the Dashboard, open the **Personality** tab, expand **Advanced settings**, and turn on **Background sound**. Turning it on selects **White noise**. To change it, select another preset or **Custom URL**.
198+
199+
| Preset | Sound | API value |
200+
| -- | -- | -- |
201+
| **Office** | Office chatter and typing | `office` |
202+
| **White noise** | Steady static | `white-noise` |
203+
| **City** | Street traffic | `city-traffic` |
204+
| **Crowd** | Many people talking | `crowd-conversation` |
205+
| **Farm** | Chickens and roosters | `chickens-roosters` |
206+
207+
Office is the quietest preset, at about 30 dB below the AI tester's speech. The others play about 15 dB below it.
208+
209+
With **Custom URL**, enter a public HTTPS URL of up to 1,000 characters that points to an audio file of 25 MB or less. The file loops for the whole call. Use WAV when you can, because an MP3 file can leave a short gap at the loop point.
210+
211+
Through the API, set `backgroundSound` in the personality's `assistant` to a preset's API value or your own URL:
212+
213+
```json
214+
{
215+
"assistant": {
216+
"backgroundSound": "white-noise"
217+
}
218+
}
219+
```
220+
221+
Include the rest of the personality's assistant configuration in the same request, as described in [Configure the AI tester](#configure-the-ai-tester).
222+
223+
<Note>
224+
The assistant or squad you're testing denoises incoming audio before it transcribes and records the call. Smart Denoising is on by default, so it removes most of the background sound, and the recording may sound clean. A clean recording is expected, because the simulation tests your production setup.
225+
</Note>
226+
227+
### Hear the raw noise
228+
229+
To hear the raw noise, turn off denoising in the scenario's `targetOverrides`. Also set `backgroundDenoisingEnabled` to `false`, because this legacy flag takes precedence when the assistant or squad you're testing sets it:
230+
231+
```json
232+
{
233+
"targetOverrides": {
234+
"backgroundDenoisingEnabled": false,
235+
"backgroundSpeechDenoisingPlan": {
236+
"smartDenoisingPlan": { "enabled": false }
237+
}
238+
}
239+
}
240+
```
241+
242+
If the assistant or squad you're testing enables [Fourier denoising](/documentation/assistants/conversation-behavior/background-speech-denoising#fourier-denoising-experimental), also set `backgroundSpeechDenoisingPlan.fourierDenoisingPlan.enabled` to `false`. Updating a scenario replaces its `targetOverrides`, so include any existing overrides you want to keep.
243+
189244
## Next steps
190245

191246
<CardGroup cols={2}>

‎fern/test/simulations-best-practices.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,8 @@ This creates useful variation without changing the business requirement.
108108

109109
Build and debug a scenario in chat mode, then run important paths in voice mode before launch. Also use voice mode after changing the transcriber, voice, or turn-taking configuration.
110110

111+
- Add a noisy-caller scenario for important paths. Turn on [background sound](/observability/simulations-configure-ai-tester#background-sound) in the AI tester's personality to check that the agent still understands the caller and completes the task.
112+
111113
<Note>
112114
Synthetic callers do not fully represent real callers, devices, or environments.
113115
Supplement voice Simulations with manual calls and production-call review for

0 commit comments

Comments
 (0)