Repository navigation
unsloth desktop provider support #11430
texastoland
started this conversation in
Ideas
Replies: 1 comment 1 reply
|
can't you update your model.yml file? providers:
rtx3090Unsloth:
baseUrl: http://127.0.0.1:8889/v1
apiKey: sk-unsloth-key
api: openai-completions
models:
- id: unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF
name: Qwen3-Coder-30B-A3B-Instruct (3090)
contextWindow: 262144
maxTokens: 65536 |
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
currently works as
/login llama.cppw/ env:export LLAMA_CPP_BASE_URL=http://127.0.0.1:8888/v1All reactions