diff --git a/fern/assistants/pronunciation-dictionaries.mdx b/fern/assistants/pronunciation-dictionaries.mdx index 2f3a1e6c6..626cc4556 100644 --- a/fern/assistants/pronunciation-dictionaries.mdx +++ b/fern/assistants/pronunciation-dictionaries.mdx @@ -252,7 +252,7 @@ Cartesia pronunciation dictionaries use a `text` and `alias` format. Each entry - For Vapi v1 built-in voices, create the pronunciation dictionary through the ElevenLabs endpoint shown above. The response includes both a dictionary ID and a `versionId`; keep both values. + For Vapi v1 built-in voices, create the pronunciation dictionary through the ElevenLabs endpoint shown above. The response includes both a dictionary ID and a `versionId`. Keep the dictionary ID, and keep the `versionId` if you want to pin that version. Vapi built-in voices use a unified `pronunciationDictionary` field, but most v1 built-in voices route through ElevenLabs under the hood. Do not assume a Cartesia dictionary ID works with every Vapi built-in voice. Cartesia dictionary IDs only apply to Vapi voices that are routed to a Cartesia Sonic voice path. @@ -277,7 +277,7 @@ Cartesia pronunciation dictionaries use a `text` and `alias` format. Each entry ``` - For ElevenLabs-backed dictionaries, `versionId` is required. It pins the exact dictionary version that the Vapi voice should use. + For ElevenLabs-backed dictionaries, `versionId` is optional. Set it to pin the exact dictionary version that the Vapi voice should use. To use the dictionary's latest version, leave out `versionId` and set `provider` to `11labs`. diff --git a/fern/calls/assistant-based-warm-transfer.mdx b/fern/calls/assistant-based-warm-transfer.mdx index f8193d26b..51eb1dc37 100644 --- a/fern/calls/assistant-based-warm-transfer.mdx +++ b/fern/calls/assistant-based-warm-transfer.mdx @@ -195,6 +195,8 @@ Use these fields to control the transfer assistant and the two call legs. See th | `transferAssistant.maxDurationSeconds` | Maximum length of the transfer-assistant conversation | 120 seconds; allowed range is 10–43,200 seconds | | `transferAssistant.silenceTimeoutSeconds` | Silence allowed before the transfer is canceled | 30 seconds; allowed range is 5–3,600 seconds | | `transferAssistant.model` | Model and instructions used for the operator conversation | Required when you configure `transferAssistant` | +| `transferAssistant.hooks` | Hooks the transfer assistant runs during the operator conversation. The operator counts as the customer, so `customer.speech.timeout` fires on operator silence | `call.ending` and `call.timeElapsed` hooks aren't allowed. A `tool` action needs an inline `tool`, not a `toolId` | +| `transferAssistant.assistantHooksInheritEnabled` | Whether the transfer assistant also runs the original assistant's hooks during the operator conversation | `true`. When `false`, only `transferAssistant.hooks` and the original assistant's `call.ending` hooks run | | `holdAudioUrl` | MP3 or WAV audio played to the customer while on hold | Default hold audio | | `transferCompleteAudioUrl` | MP3 or WAV audio played to the destination after the introduction | No custom completion audio | | `fallbackPlan` | Customer message and end-call behavior after a failed warm transfer | If omitted, the original assistant says "Could not transfer the call." and continues the conversation | @@ -255,6 +257,7 @@ Open the affected call in [**Calls**](/observability/logs/call-logs) under **Log | The customer hears the wrong audio | Hold audio was configured as a tool message | Put customer hold audio in `transferPlan.holdAudioUrl`. Use `transferCompleteAudioUrl` for audio played to the destination after the introduction. | | The customer cannot continue after a failed transfer | `fallbackPlan.endCallEnabled` is `true` | Set it to `false` so the original assistant remains on the call. | | The transfer ends during a long operator interaction | A duration or silence timeout is too short | Increase `maxDurationSeconds` or `silenceTimeoutSeconds` within the supported ranges. | +| The whole call ends when the operator goes quiet | The transfer assistant inherited a `customer.speech.timeout` hook with an `endCall` action from the original assistant | Set `transferAssistant.assistantHooksInheritEnabled` to `false`, and add any hooks the operator conversation needs to `transferAssistant.hooks`. | ## API reference diff --git a/fern/calls/call-ended-reason.mdx b/fern/calls/call-ended-reason.mdx index 6ac710991..68f3cb982 100644 --- a/fern/calls/call-ended-reason.mdx +++ b/fern/calls/call-ended-reason.mdx @@ -137,7 +137,7 @@ Each voice provider has specific error codes. Common patterns: - `*-quota-exceeded` / `*-out-of-credits` — Voice provider credits exhausted. - `*-unauthorized-access` / `*-invalid-api-key` — Voice provider credential issue. - `*-unsupported-model` — The voice provider rejected the configured model. -- `*-socket-hang-up` / `*-500-server-error` / `*-503-server-error` — Voice provider infrastructure issue. +- `*-socket-hang-up` / `*-500-server-error` / `*-502-server-error` / `*-503-server-error` — Voice provider infrastructure issue. **Supported providers:** ElevenLabs, Cartesia, Deepgram, Azure, OpenAI, Rime AI, Smallest AI, Neuphonic, Hume, Sesame, Inworld, Minimax, WellSaid, Custom Voice. @@ -151,7 +151,14 @@ Common transcriber error patterns: - `*-returning-403-*` — Model access denied on the transcriber. - `*-returning-500-*` / `*-returning-502-*` — Transcriber provider server error. -**Supported providers:** Deepgram, AssemblyAI, Gladia, Speechmatics, Azure Speech, Google, OpenAI, Soniox, ElevenLabs, Custom Transcriber. +**Supported providers:** Deepgram, AssemblyAI, Gladia, Speechmatics, Azure Speech, Google, OpenAI, Soniox, ElevenLabs, Reson8, Custom Transcriber. + +Soniox, xAI, and Reson8 report these more specific reasons instead of a generic failure: + +- `*-transcriber-auth-failed` — The transcriber provider rejected the credentials. +- `*-transcriber-invalid-config` — The transcriber provider rejected the transcriber settings. +- `*-transcriber-rate-limited` — The transcriber provider rate-limited the request. +- `reson8-transcriber-closed` — The Reson8 connection closed unexpectedly. ## Transfer errors diff --git a/fern/docs.yml b/fern/docs.yml index 3e4dfba78..6d114b129 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -884,6 +884,8 @@ navigation: path: providers/transcriber/google.mdx - page: OpenAI path: providers/transcriber/openai.mdx + - page: Reson8 + path: providers/transcriber/reson8.mdx - page: Soniox path: providers/transcriber/soniox.mdx - page: Speechmatics diff --git a/fern/gpt-live/configuration.mdx b/fern/gpt-live/configuration.mdx index ce9e8106b..196018193 100644 --- a/fern/gpt-live/configuration.mdx +++ b/fern/gpt-live/configuration.mdx @@ -101,7 +101,7 @@ These fields belong to the assistant object. See [Create Assistant](/api-referen | `model.tools` | Inline base tool definitions, available whether or not a skill is loaded | | `model.toolIds` | IDs of saved base tools | | `voice.provider` | `openai` | -| `voice.voiceId` | One of the [22 supported voices](#voices) | +| `voice.voiceId` | One of the [31 supported voices](#voices) | ## Prompt defaults and overrides diff --git a/fern/gpt-live/overview.mdx b/fern/gpt-live/overview.mdx index dd4985d89..39b21fa77 100644 --- a/fern/gpt-live/overview.mdx +++ b/fern/gpt-live/overview.mdx @@ -66,7 +66,7 @@ Vapi connects GPT-Live to your phone numbers and tools, and provides call record - Browser, Twilio, Vapi SIP, and WebSocket calls. - Function, API request, and MCP tools, Google Calendar, Google Sheets, and Slack tools, plus built-in Hang Up, transfer, and DTMF tools. -- Speaker and reasoner prompts, reasoner skills, personality packs, and 22 voices. +- Speaker and reasoner prompts, reasoner skills, personality packs, and 31 voices. - Idle-message hooks and HTTP live call control. - Transcripts, recordings, structured outputs, scorecards, Boards, Monitoring, and Voice Simulations. diff --git a/fern/providers/model/anthropic-bedrock.mdx b/fern/providers/model/anthropic-bedrock.mdx index dbe6da05f..ade46bc78 100644 --- a/fern/providers/model/anthropic-bedrock.mdx +++ b/fern/providers/model/anthropic-bedrock.mdx @@ -17,7 +17,15 @@ Vapi's Anthropic Bedrock integration allows you to connect your own AWS Bedrock For the current Anthropic Claude models available through Amazon Bedrock, see [Anthropic models in the Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards-anthropic.html). Model availability varies by AWS Region. -For additional API configuration options, review the [`AnthropicBedrockModel` fields](/api-reference/assistants/create#request.body.model.AnthropicBedrock) in the Create Assistant API reference. +Some Claude models are available through Bedrock's global cross-Region inference profiles. Their model IDs start with `global.`. A global profile can route requests to supported commercial AWS Regions worldwide. + +To use a global profile, grant access to the profile, its foundation model in the source Region, and its global foundation model ARN. See the [`AnthropicBedrockModel` fields](/api-reference/assistants/create#request.body.model.AnthropicBedrock) for the model IDs Vapi accepts. + +For more information, see the following Amazon Bedrock documentation: + +- [Cross-Region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html) +- [Global cross-Region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/global-cross-region-inference.html), including its IAM policy requirements +- [Supported Regions and models for inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html) ### Prerequisites @@ -46,11 +54,15 @@ Before configuring Anthropic Bedrock with Vapi, ensure you have: Configure the trust policy to allow Vapi's AWS account to assume this role. + An External ID is optional. If you use one, enter the same unique value in the trust policy and Vapi credential. Both examples trust Vapi's production AWS account. + + For more information about these policies, see AWS's guides to [creating a role with a custom trust policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html) and [using External IDs for third-party access](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_third-party.html). + - - Use this simpler policy if you don't need the additional security of an External ID: + + Replace `YOUR_UNIQUE_EXTERNAL_ID` with a unique value before use. AWS recommends an External ID for third-party cross-account access. - ```json title="Trust Policy (Basic)" + ```json title="Trust policy with External ID" { "Version": "2012-10-17", "Statement": [ @@ -59,16 +71,21 @@ Before configuring Anthropic Bedrock with Vapi, ensure you have: "Principal": { "AWS": "arn:aws:iam::533267069243:root" }, - "Action": "sts:AssumeRole" + "Action": "sts:AssumeRole", + "Condition": { + "StringEquals": { + "sts:ExternalId": "YOUR_UNIQUE_EXTERNAL_ID" + } + } } ] } ``` - - Use this policy if you want the additional security of an External ID: + + Use this policy only if you do not require an External ID for the role. It does not check which Vapi customer the request is for. - ```json title="Trust Policy (With External ID)" + ```json title="Trust policy without External ID" { "Version": "2012-10-17", "Statement": [ @@ -77,85 +94,61 @@ Before configuring Anthropic Bedrock with Vapi, ensure you have: "Principal": { "AWS": "arn:aws:iam::533267069243:root" }, - "Action": "sts:AssumeRole", - "Condition": { - "StringEquals": { - "sts:ExternalId": "YOUR_EXTERNAL_ID" - } - } + "Action": "sts:AssumeRole" } ] } ``` - - Replace `YOUR_EXTERNAL_ID` with your chosen value. You'll use the same value when creating the Vapi credential. - - - **About External ID**: The External ID is an optional security feature that provides an additional layer of protection for cross-account access. If you choose to use one, specify the same value in both your IAM trust policy and the Vapi credential configuration. - - Create and attach a permissions policy that grants access to Bedrock model invocation. + Use this policy when your Vapi credential Region is `us-east-1`. It allows Claude Haiku 5.5 through the US inference profile, which can route to `us-east-1`, `us-east-2`, or `us-west-2`. Replace `YOUR_AWS_ACCOUNT_ID` with your AWS account ID before creating the policy. - - - Use this policy to grant access to all Anthropic models across all regions: - - ```json title="Permissions Policy (All Models)" + ```json title="Permissions policy: us-east-1" + { + "Version": "2012-10-17", + "Statement": [ { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "bedrock:InvokeModel", - "bedrock:InvokeModelWithResponseStream" - ], - "Resource": [ - "arn:aws:bedrock:*::foundation-model/anthropic.*", - "arn:aws:bedrock:*:*:inference-profile/us.anthropic.*" - ] - } - ] - } - ``` - - - Use this policy to limit access to specific models and regions: - - ```json title="Permissions Policy (Specific Models)" + "Effect": "Allow", + "Action": [ + "bedrock:InvokeModel", + "bedrock:InvokeModelWithResponseStream" + ], + "Resource": "arn:aws:bedrock:us-east-1:YOUR_AWS_ACCOUNT_ID:inference-profile/us.anthropic.claude-haiku-5-5" + }, { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "bedrock:InvokeModel", - "bedrock:InvokeModelWithResponseStream" - ], - "Resource": [ - "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-*", - "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-7-sonnet-*", - "arn:aws:bedrock:us-east-1:*:inference-profile/us.anthropic.claude-3-5-sonnet-*", - "arn:aws:bedrock:us-east-1:*:inference-profile/us.anthropic.claude-3-7-sonnet-*" - ] + "Effect": "Allow", + "Action": [ + "bedrock:InvokeModel", + "bedrock:InvokeModelWithResponseStream" + ], + "Resource": [ + "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-haiku-5-5", + "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-haiku-5-5", + "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-haiku-5-5" + ], + "Condition": { + "StringEquals": { + "bedrock:InferenceProfileArn": "arn:aws:bedrock:us-east-1:YOUR_AWS_ACCOUNT_ID:inference-profile/us.anthropic.claude-haiku-5-5" } - ] + } } - ``` + ] + } + ``` - - Replace `us-east-1` with your preferred AWS region. You can also add or remove specific model versions based on your needs. - - - + To adapt the policy, check the exact profile ID, supported source Regions, and destination Regions in the [AWS Claude Haiku 5.5 model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-haiku-5-5.html). You can also call [GetInferenceProfile](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-view.html) from your source Region with an AWS identity that has `bedrock:GetInferenceProfile` permission. Update the profile ARN, foundation model ARNs, and `bedrock:InferenceProfileArn` condition together. Check the [`AnthropicBedrockModel` fields](/api-reference/assistants/create#request.body.model.AnthropicBedrock) for the model IDs Vapi accepts. + + For more information about the required resources, see AWS's [inference profile prerequisites](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html). Global profiles require a different policy shape; see AWS's [global cross-Region inference guidance](https://docs.aws.amazon.com/bedrock/latest/userguide/global-cross-region-inference.html). + + Before attaching the policy, use [IAM Access Analyzer policy validation](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-analyzer-policy-validation.html) to review its syntax and permissions. Other policies and service control policies can also affect the role's effective access. To attach the policy: + 1. In the IAM Console, go to **Policies** and click **Create policy** - 2. Select **JSON** and paste your chosen policy + 2. Select **JSON** and paste the policy 3. Name the policy (e.g., `VapiBedrockInvokePolicy`) 4. Attach the policy to your IAM role @@ -173,7 +166,7 @@ Before configuring Anthropic Bedrock with Vapi, ensure you have: "authenticationPlan": { "type": "aws-sts", "roleArn": "arn:aws:iam::YOUR_AWS_ACCOUNT_ID:role/VapiBedrockRole", - "externalId": "your-optional-external-id" + "externalId": "YOUR_UNIQUE_EXTERNAL_ID" } }' ``` @@ -184,11 +177,9 @@ Before configuring Anthropic Bedrock with Vapi, ensure you have: | `region` | AWS region where Bedrock is enabled (e.g., `us-east-1`) | | `authenticationPlan.type` | Must be `aws-sts` for role assumption | | `authenticationPlan.roleArn` | The ARN of the IAM role you created | - | `authenticationPlan.externalId` | Optional: Your chosen External ID for additional security | + | `authenticationPlan.externalId` | Optional; include only with the matching External ID trust policy | - - If you don't need an External ID, you can remove the `Condition` block from your trust policy entirely. - + If you choose the trust policy without an External ID, omit the `externalId` field from the request. diff --git a/fern/providers/transcriber/assembly-ai.mdx b/fern/providers/transcriber/assembly-ai.mdx index 61f6f726d..cf00d438b 100644 --- a/fern/providers/transcriber/assembly-ai.mdx +++ b/fern/providers/transcriber/assembly-ai.mdx @@ -44,6 +44,7 @@ Both Universal Pro models support additional configuration: - `agentContextAutoUpdateEnabled`: Automatically send what the assistant just said as agent context after every assistant turn. Defaults to `false`. - `languageCodes`: Steer automatic language detection with language codes such as `["en", "es"]`. - `keytermsPrompt`: Boost recognition of domain-specific terms. Included at no extra cost on the Universal Pro models. +- `confidenceThreshold`: Discard caller transcripts with a confidence below this value, from `0` to `1`. Universal Pro models don't filter by confidence unless you set it. In the dashboard, turn on **Low-confidence filter** in the transcriber settings. ### Automatic agent context diff --git a/fern/providers/transcriber/deepgram.mdx b/fern/providers/transcriber/deepgram.mdx index b95e0239c..262b8376d 100644 --- a/fern/providers/transcriber/deepgram.mdx +++ b/fern/providers/transcriber/deepgram.mdx @@ -112,7 +112,7 @@ The Nova 3 column applies to Nova 3 and Nova 3 General, and the Nova 2 column ap | Macedonian | `mk` | X | | | | | | | Malay | `ms` | X | | X | | | | | Marathi | `mr` | X | | | | | | -| Multilingual | `multi` | X | | X | | | | +| Multilingual | `multi` | X | X | X | | | | | Norwegian | `no` | X | | X | | | | | Persian | `fa` | X | | | | | | | Polish | `pl` | X | | X | | | | @@ -139,7 +139,7 @@ The Nova 3 column applies to Nova 3 and Nova 3 General, and the Nova 2 column ap ### Multilingual transcription -For Nova 3 or Nova 2, set `transcriber.language` to `multi`. For Flux, select `flux-general-multi` and omit `transcriber.languages` to use automatic language detection. You can also provide a `languages` array to give Flux language hints for multilingual or code-switching conversations. See [Deepgram's Flux language prompting documentation](https://developers.deepgram.com/docs/flux/language-prompting). +For Nova 3, Nova 3 Medical, or Nova 2, set `transcriber.language` to `multi`. For Flux, select `flux-general-multi` and omit `transcriber.languages` to use automatic language detection. You can also provide a `languages` array to give Flux language hints for multilingual or code-switching conversations. See [Deepgram's Flux language prompting documentation](https://developers.deepgram.com/docs/flux/language-prompting). ## Related diff --git a/fern/providers/transcriber/overview.mdx b/fern/providers/transcriber/overview.mdx index 393abdccf..fed3d275f 100644 --- a/fern/providers/transcriber/overview.mdx +++ b/fern/providers/transcriber/overview.mdx @@ -25,6 +25,7 @@ The provider string is the exact value you pass in `transcriber.provider`. | [Gladia](/providers/transcriber/gladia#speech-to-text) | `gladia` | | [Google](/providers/transcriber/google#speech-to-text) | `google` | | [OpenAI](/providers/transcriber/openai#speech-to-text) | `openai` | +| [Reson8](/providers/transcriber/reson8#speech-to-text) | `reson8` | | [Soniox](/providers/transcriber/soniox#speech-to-text) | `soniox` | | [Speechmatics](/providers/transcriber/speechmatics#speech-to-text) | `speechmatics` | | [xAI](/providers/transcriber/xai#speech-to-text) | `xai` | diff --git a/fern/providers/transcriber/reson8.mdx b/fern/providers/transcriber/reson8.mdx new file mode 100644 index 000000000..e59359611 --- /dev/null +++ b/fern/providers/transcriber/reson8.mdx @@ -0,0 +1,74 @@ +--- +title: Reson8 +subtitle: Use Reson8 for speech-to-text in Vapi +description: Configure Reson8 as your Vapi assistant's transcriber, including supported languages, keyterms, custom models, and turn detection settings. +slug: providers/transcriber/reson8 +--- + +Reson8 provides speech-to-text with built-in turn detection for Vapi voice agents. + +You can use Reson8 through Vapi's default integration, or connect your own account in [Integrations](/providers/integrations). + +## Supported capability + +| Capability | Provider value | +| --- | --- | +| Speech-to-text | `reson8` | + +## Speech-to-text + +Set `transcriber.provider` to `reson8`. The optional `model` field accepts only `turns`, Reson8's turn detection endpoint. Set `language` to one language code, or to a comma-separated list such as `nl,en` to limit automatic language detection to those languages. Leave `language` out to detect the language on each utterance. + +```bash +curl -X PATCH "https://api.vapi.ai/assistant/ASSISTANT_ID" \ + -H "Authorization: Bearer $VAPI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "transcriber": { + "provider": "reson8", + "model": "turns", + "language": "nl,en", + "keyterms": ["Vapi", "order number"] + } + }' +``` + +For additional API configuration options, review the [`Reson8Transcriber` fields](/api-reference/assistants/create#request.body.transcriber.Reson8Transcriber) in the Create Assistant API reference. + +### Transcriber options + +| Field | Required | Vapi behavior | Description | +| --- | --- | --- | --- | +| `model` | No | Uses `turns` | The only accepted value is `turns`. | +| `language` | No | Detects the language on each utterance | One language code, or a comma-separated list of candidate languages. | +| `keyterms` | No | No keyterms | Up to 250 terms, such as brand names or product codes, to improve recognition. | +| `customModelId` | No | No custom model | The ID of a phrase collection you manage in Reson8. | +| `eagerTurnProbability` | No | Reson8's default, `0.5` | The probability, from `0` to `1`, at which Reson8 flags a possible end of turn. | +| `finalTurnProbability` | No | Reson8's default, `0.92` | The probability, from `0` to `1`, at which Reson8 confirms the end of a turn. | +| `fillerMode` | No | Reson8's default, `natural` | How filler words appear in transcripts: `clean`, `natural`, or `verbatim`. | + +### Supported languages + +| Language | Language code | +| --- | --- | +| Dutch | `nl` | +| English | `en` | +| French | `fr` | +| Frisian | `fy` | +| German | `de` | +| Italian | `it` | +| Polish | `pl` | +| Portuguese | `pt` | +| Spanish | `es` | +| Swedish | `sv` | + +## Related + + + + Compare every transcriber provider Vapi supports. + + + Configure backup transcribers for your assistant. + + diff --git a/fern/providers/transcriber/soniox.mdx b/fern/providers/transcriber/soniox.mdx index 04bdf4cb8..ee6088932 100644 --- a/fern/providers/transcriber/soniox.mdx +++ b/fern/providers/transcriber/soniox.mdx @@ -1,19 +1,19 @@ --- title: Soniox subtitle: Configure real-time, multilingual speech-to-text for Vapi assistants -description: Connect Soniox to a Vapi assistant and configure the stt-rt-v5 real-time speech-to-text model for low-latency, multilingual transcription and endpointing. +description: Connect Soniox to a Vapi assistant and configure Soniox real-time speech-to-text models for low-latency, multilingual transcription and endpointing. slug: providers/transcriber/soniox --- -Soniox is a real-time speech-to-text (STT) provider that transcribes caller audio for Vapi assistants. Use the `stt-rt-v5` model for new configurations. +Soniox is a real-time speech-to-text (STT) provider that transcribes caller audio for Vapi assistants. See [Supported models](#supported-models) for the models you can use. ## What is Soniox? -Soniox provides low-latency, multilingual transcription and semantic endpointing for voice assistants. The `stt-rt-v5` model supports real-time audio and is the current model for new Soniox configurations. +Soniox provides low-latency, multilingual transcription and semantic endpointing for voice assistants. ## Speech-to-text -Set `transcriber.provider` to `soniox` and `transcriber.model` to `stt-rt-v5`. Set `transcriber.languages` to an empty array for automatic language detection. +Set `transcriber.provider` to `soniox` and `transcriber.model` to one of the [supported models](#supported-models). Set `transcriber.languages` to an empty array for automatic language detection. ```bash curl -X PATCH "https://api.vapi.ai/assistant/ASSISTANT_ID" \ @@ -34,6 +34,7 @@ For additional API configuration options, review the [`SonioxTranscriber` fields | Model | Model ID | | --- | --- | +| Soniox STT Real-Time v6 | `stt-rt-v6` | | Soniox STT Real-Time v5 | `stt-rt-v5` | ### Automatic language detection @@ -126,6 +127,23 @@ curl -X PATCH "https://api.vapi.ai/assistant/ASSISTANT_ID" \ Confidence is the mean of the per-token scores for the transcript. Any final transcript below the threshold is discarded. When `confidenceThreshold` is unset, nothing is discarded by this setting. See the [`SonioxTranscriber` fields](/api-reference/assistants/create#request.body.transcriber.SonioxTranscriber) for the full configuration. +## Echo filtering + +With `stt-rt-v6`, Vapi sends the assistant's speech to Soniox as context. Soniox uses it to filter the assistant's own voice out of the caller's transcript, such as echo from a speakerphone, and to improve caller transcription. This is on by default. To turn it off, set `transcriber.agentTextInputEnabled` to `false`. Other Soniox models don't support this setting. + +```bash +curl -X PATCH "https://api.vapi.ai/assistant/ASSISTANT_ID" \ + -H "Authorization: Bearer $VAPI_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "transcriber": { + "provider": "soniox", + "model": "stt-rt-v6", + "agentTextInputEnabled": false + } + }' +``` + ## Related - Configure a [transcriber fallback plan](/customization/transcriber-fallback-plan). diff --git a/fern/voice-fallback-plan.mdx b/fern/voice-fallback-plan.mdx index 90e2928d1..b96bd36ae 100644 --- a/fern/voice-fallback-plan.mdx +++ b/fern/voice-fallback-plan.mdx @@ -7,7 +7,7 @@ description: Configure an ordered voice fallback plan in the Dashboard or API, t ## Overview -Voice fallback lets you configure an ordered list of backup voices. When a voice error triggers fallback, the assistant tries the next voice in your plan so the call can continue. +Voice fallback lets you configure an ordered list of backup voices. When the current voice fails or goes silent, the assistant tries the next voice in your plan so the call can continue. Built-in fallback behavior without a configured plan depends on the selected voice. Configure `voice.fallbackPlan.voices` to choose the backup voices and their order. @@ -17,11 +17,17 @@ Built-in fallback behavior without a configured plan depends on the selected voi ## How it works -When a voice error triggers fallback, the assistant: +Fallback starts when the current voice: + +- Returns an error. +- Sends no audio within 3 seconds after the model finishes a reply. +- Finishes an ElevenLabs reply with no audio. Vapi retries that reply once before it switches voices. + +When fallback starts, the assistant: 1. Tries the first voice in `voice.fallbackPlan.voices`. -2. Tries the next voice if a backup also fails and triggers fallback. -3. May end the call if no voice succeeds. +2. Tries the next voice if a backup also fails or goes silent. +3. If no voice succeeds, may end the call after an error. After silence, the call continues with the current voice. ## Configure via Dashboard