Tool calling support for Ollama-compatible API

[SimonePal]

Tool calling / function calling not working

Running rkllama on a Rock 5B (RK3588 NPU) with qwen2.5-coder-3b, which supports tool calling natively.

When sending a request with a tools field, rkllama ignores it completely and just returns a plain text reply:

curl http://localhost:8080/api/chat -H "Content-Type: application/json" -d '{
  "model": "code-3b",
  "messages": [{"role": "user", "content": "hello"}],
  "tools": [{"type": "function", "function": {"name": "test", "description": "test"}}],
  "stream": false
}'

Response has no tool_calls, just "content": "Hello! How can I assist you today?"

This breaks agent/plan mode in Continue (VS Code) since it relies on tool calling to work. Would be great to have this supported since the model itself is capable of it!

[danielferr85]

Hi! Tool calls works....the tag tool_call in the response is optional and only apears when the model whants to execute a tool.

In your example there is no sense that the model wants to execute the function test.

Official ollama docs:

https://github.com/ollama/ollama/blob/main/docs/api.md

Please send me a valid example to check. Also models with more parameter perform better tool calls.