Custom body parameters

[Thyraz]

Hi skye,

as a lot cloud providers use parameters in the base level of the request instead under chat template arguments,
I wanted to ask about your position to add a similar option entry here.

For example reasoning and similar options can't be set otherwise.

I currently built this as fork for myself here and I'm not sure if this is something worth to discuss to create a PR for ...
It already blocks keys reserved by the integration or that already have an UI setting in the options dialog.

https://github.com/Thyraz/hass_local_openai_llm/tree/request-cody-parameters

[Thyraz]

Another usecase:

I selected a few providers to the whitelist in OpenRouter, so only they are allowed.
They are the fastest (time to first token and high tokens per second) for the model I use.

But the latency isn't something that is staying the same the whole day.
Sometimes some providers are slower due to higher load.

So I set this rule for the request to OpenRouter, which always selects the provider with the lowest latency currently available from my whitelist:

[skye-harris]

its never not funny to me that my integration, targeting the local space, gets so many requests centered around cloud services 😅

someones made an issue requesting it as well I think but Ive been noping out at the wall-of-llm-generated-text each time I go to open it so far. not against the idea... I just hate reading AI walls-of-text

[Thyraz]

Well, you simply made the best integration for OpenAI compatible endpoints.
Who wouldn't want to use it, regardless of cloud or local. 😉

I also tend to ignore issues or PRs that are written like lengthy AI slop advertisements. 😵‍💫
If you need an LLM to summarize the text, someone else generated with an LLM as well, things are going wrong ...

I have to update my branch to your latest code before sending an PR, to check if there is anything that needs to be adopted.
Then let's see if the additions would be ok for you ...

[skye-harris]

haha yeah sorry about that. Ive been breaking up the main handler method to be sane and testable, along with abstracting sections as new server types call for it

[skye-harris]

This is now released in https://github.com/skye-harris/hass_local_openai_llm/releases/tag/1.11.0