Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions fern/assistants/concepts/transient-vs-permanent-configurations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,39 @@ The following examples are `assistant.model` fragments:
cannot be managed through the dashboard or reused across calls.
</Warning>

### Model URLs on public-key requests

A request authenticated with a public API key or a public JWT can't supply a model URL (`model.url`, the **Custom LLM URL** field in the Dashboard). This applies to `POST /call/web`, `POST /chat/web`, and `POST /chat/web/responses`, which are the endpoints the Vapi Web SDK and the chat widget use. A JWT with no `token` claim counts as a public JWT. For more information, see [JWT authentication](/customization/jwt-authentication).

The request is rejected when `model.url` appears anywhere in the request body, including:

- a transient `assistant`
- `assistantOverrides`, including a `tools:append` entry
- a transient `squad` member, its `assistantOverrides`, `squad.membersOverrides`, or `squadOverrides`
- the destinations of a handoff tool, at any depth
- a transient `workflow` or one of its nodes

A model URL on a stored assistant isn't affected. Neither are Dashboard sessions, or requests to `POST /call` or `POST /chat` authenticated with a private API key or a private JWT.

<Note>
Vapi is turning on this check for organizations in phases, and emails affected organizations before the check applies to them.
</Note>

A rejected request returns a `403` response with this body:

```json
{
"message": "Supplying `model.url` in the request body is not permitted on public-key authenticated calls. Configure the model URL on a stored assistant instead.",
"error": "Forbidden",
"statusCode": 403
}
```

To avoid the rejection, use one of these approaches:

- **Store the assistant.** Set `model.url` on a stored assistant, then pass its `assistantId` from the browser. Keep per-call changes in `assistantOverrides` fields that don't set a model URL, such as `variableValues`. On `POST /chat/web` and `POST /chat/web/responses`, `variableValues` is the only `assistantOverrides` field accepted.
- **Start the call or chat from your server.** Send the request to `POST /call` or `POST /chat` with your private API key or a private JWT, which can still supply a model URL. Never send your private API key to a browser or mobile app.

## Permanent configurations

Use **permanent configurations** for reusable setups that multiple teams can access and manage through the dashboard.
Expand Down
4 changes: 4 additions & 0 deletions fern/customization/custom-llm/using-your-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@ On the **Assistant** tab, select **MODEL**. In **Model Settings**, open **Model*
**3. Enter Ngrok URL:**
Enter `gpt-4.1-mini` in the second **Model** field. Paste the public base URL for your OpenAI-compatible endpoint into **Custom LLM URL** (for example, the URL generated by Ngrok for your server). The server handles requests at `/chat/completions`.

<Note>
Set the custom LLM URL on a stored assistant, as in this step. A web call or chat authenticated with a public API key or a public JWT that supplies `model.url` is rejected with a `403` response. For more information, including how to set the URL per request from your server, see [Model URLs on public-key requests](/assistants/concepts/transient-vs-permanent-configurations#model-urls-on-public-key-requests).
</Note>

**4. Test the Connection:**
Select **Publish**, review the changes, then select **Next**. Review the version and traffic settings, then select **Publish**. Select **Talk** to test the assistant in the Dashboard. Check that your server receives the request and the assistant responds.

Expand Down
Loading