You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Document blank replies caused by off-schema streamed tool calls
The Gemini warnings added earlier only help someone who already suspects their provider. The people hitting this see random empty assistant messages with nothing in the logs, and search for that.
Connection Errors gains a section keyed on the symptom: what it looks like, why a missing `index` on a streamed tool call makes the call disappear, a curl that tells you in one command whether your own endpoint does it, and the four ways out. The troubleshooting index gets a symptom row, and the Memory page warns at the point where users turn the feature on, since Memory is what triggers this for people who never configured a tool themselves.
Written provider-neutrally because any OpenAI-compatible endpoint that omits `index` is affected; Google's compatibility layer is named as the known case.
Copy file name to clipboardExpand all lines: docs/alternatives/gemini.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -85,7 +85,7 @@ Gemini models are available through Open WebUI via the Google AI API. You can us
85
85
4. Gemini models will appear in your model selector
86
86
87
87
:::warning Tools need extra setup on this endpoint
88
-
Google's OpenAI compatibility layer does not follow the OpenAI schema for tool calls, so a turn where the model decides to call a tool comes back as an empty assistant message. This covers the builtin tools such as Memory, so it shows up as the occasional blank reply rather than an obvious failure. Set **Function Calling** to **Legacy** in the model's Advanced Params, or route Gemini through a gateway such as LiteLLM or OpenRouter. The [OpenAI-compatible provider guide](/getting-started/quick-start/connect-a-provider/starting-with-openai-compatible) has the details.
88
+
Google's OpenAI compatibility layer does not follow the OpenAI schema for tool calls, so a turn where the model decides to call a tool comes back as an empty assistant message. This covers the builtin tools such as Memory, so it shows up as the occasional blank reply rather than an obvious failure. Set **Function Calling** to **Legacy** in the model's Advanced Params, or route Gemini through a gateway such as LiteLLM or OpenRouter. See [Blank Replies When the Model Uses a Tool](/troubleshooting/connection-error#-blank-replies-when-the-model-uses-a-tool) and the [OpenAI-compatible provider guide](/getting-started/quick-start/connect-a-provider/starting-with-openai-compatible).
Copy file name to clipboardExpand all lines: docs/features/chat-conversations/memory.mdx
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -79,6 +79,10 @@ These are set automatically by the memory tools; you do not need to assign them
79
79
3.**Minimum Model Tier**: Use a current model with solid tool calling β at minimum GPT-5.6 Luna, Gemini 3.5 Flash-Lite, MiniMax M3, or Muse Glimmer 30B locally. Very small models may not manage memories reliably.
80
80
4.**Per-Model Category Toggle**: Ensure the **Memory** category is enabled for the model in **Workspace > Models > Edit > Builtin Tools** (enabled by default).
81
81
82
+
:::warning Blank replies instead of memory operations
83
+
Memory runs as a native tool call, so it only works on a connection whose streaming output follows the OpenAI schema. On an endpoint that omits the `index` field on streamed tool calls, notably Google's OpenAI compatibility layer, the call is dropped and the turn returns an empty assistant message with nothing in the logs. Because the model decides on its own when to call `list_memories` or `add_memory`, this shows up as occasional blank replies rather than an obvious memory failure. See [Blank Replies When the Model Uses a Tool](/troubleshooting/connection-error#-blank-replies-when-the-model-uses-a-tool).
84
+
:::
85
+
82
86
:::info Central Tool Documentation
83
87
For complete details on all built-in agentic tools (including memory, web search, and knowledge bases) and how to configure them, see the [**Native/Agentic Mode Tools Guide**](/features/extensibility/plugin/tools#built-in-system-tools-nativeagentic-mode).
Copy file name to clipboardExpand all lines: docs/getting-started/quick-start/connect-a-provider/starting-with-openai-compatible.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -115,7 +115,7 @@ Each connection has a **toggle switch** that lets you enable or disable it witho
115
115
- Turn the **builtin tools** capability off for the model if you only want plain chat.
116
116
- For working native tool calling, put a gateway in front that translates to Google's own API, such as LiteLLM or OpenRouter, or use a pipe that calls `generateContent` directly.
117
117
118
-
This is a deviation on Google's side, so Open WebUI will not work around it. See[#28492](https://github.com/open-webui/open-webui/issues/28492).
118
+
This is a deviation on Google's side, so Open WebUI will not work around it. [Blank Replies When the Model Uses a Tool](/troubleshooting/connection-error#-blank-replies-when-the-model-uses-a-tool) covers the symptom in full, including how to test any other provider for it. Reported in[#28492](https://github.com/open-webui/open-webui/issues/28492).
-**Caddy**: Generally handles SSE correctly by default, but check for any buffering plugins
165
165
166
+
## π Blank Replies When the Model Uses a Tool
167
+
168
+
### Common Symptoms
169
+
170
+
- The assistant message comes back **completely empty**, with no error in the UI and nothing in the logs at any log level.
171
+
- It looks **random**: most turns are fine, and the same prompt sometimes works and sometimes does not.
172
+
- The provider still bills you for output tokens on the empty turn.
173
+
- The stored message has `content: ''` and `output: []` while `usage.completion_tokens` is non-zero.
174
+
- It gets much worse once tools are enabled, and worst of all with the builtin tools, since the model calls those on its own.
175
+
176
+
### Cause: the provider omits `index` on streamed tool calls
177
+
178
+
The OpenAI streaming schema puts an `index` field on every tool call delta, which is what lets a client stitch the fragments of a call back together across chunks. Open WebUI keys its accumulator on that field. A provider that sends each tool call complete in a single chunk and leaves `index` out is off-schema, and the call is dropped. The turn then ends with no text and no tool run, which is the empty message you see.
179
+
180
+
This is not tied to one vendor. It affects any OpenAI-compatible endpoint that streams tool calls without `index`. **Google's OpenAI compatibility layer at `https://generativelanguage.googleapis.com/v1beta/openai` is the known case**, and Gemini 3 models add a second problem on top: a proprietary `thought_signature` that Google requires to be sent back on the follow-up request, which no OpenAI-schema client will do.
181
+
182
+
### Confirm it in one command
183
+
184
+
Ask the endpoint directly and look at the tool call chunk. Substitute your own URL, key and model:
185
+
186
+
```bash
187
+
curl -s -N -X POST "https://your-provider.example/v1/chat/completions" -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" -d '{"model":"your-model","stream":true,"messages":[{"role":"user","content":"What is the weather in Paris?"}],"tools":[{"type":"function","function":{"name":"get_weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]}'
188
+
```
189
+
190
+
In the `delta.tool_calls` entry, look for `"index"`. If it is absent, you have this problem. The `index` on the surrounding `choices` entry is a different field and does not count.
191
+
192
+
### Solutions
193
+
194
+
1.**Put a gateway in front.** LiteLLM, OpenRouter or any proxy that talks the provider's native API and emits schema-correct OpenAI output. This is the only option that keeps native tool calling and the builtin tools working.
195
+
2.**Switch the model to Legacy function calling.** In the model's **Advanced Params**, set **Function Calling** to **Legacy**. Tools are then selected by a separate non-streaming request and never go through the streaming accumulator. Note that **Default** is not the same as Legacy here: an unset value means native. The trade-off is that the builtin tools (Memory and the rest) are unavailable in Legacy mode, since they are injected only for native function calling.
196
+
3.**Turn tools off for that model.** If you only want plain chat, disable the **builtin tools** capability on the model so it never tries to call anything. Chat completions themselves are unaffected by this bug.
197
+
4.**Use a pipe.** A [pipe function](/features/extensibility/plugin/functions/pipe) that calls the provider's own API directly bypasses the OpenAI-compatible path entirely.
198
+
199
+
Open WebUI follows the OpenAI Chat Completions schema and does not add per-provider workarounds for endpoints that diverge from it, which is the same reasoning described under [Protocol-Oriented Design](/getting-started/quick-start/connect-a-provider/starting-with-openai-compatible#protocol-oriented-design).
200
+
166
201
## π Frontend vs. Backend Connections (localhost Confusion)
167
202
168
203
Several Open WebUI features offer **two ways** to configure connections: a **user/direct** method (from the browser) and an **admin/global** method (from the backend). These work at completely different network levels, and the same URL can succeed in one and fail in the other.
| Assistant replies are randomly blank, worse with tools enabled |[Connection Errors β Blank Replies on Tool Calls](./connection-error#-blank-replies-when-the-model-uses-a-tool)|
28
29
| Can't connect to Ollama from Open WebUI |[Connection Errors β Ollama](./connection-error#connection-to-ollama-server)|
29
30
| Model list takes forever to load / `500` on `/api/models`|[Connection Errors β Model List Loading](./connection-error#%EF%B8%8F-model-list-loading-issues-slow-ui--unreachable-endpoints)|
0 commit comments