Repository navigation
Commit 086c678
feat(server): add legacy /v1/completions endpoint for the inbuilt inference server
The built-in inference server only exposed the chat protocol at
/v1/chat/completions, so OpenAI-legacy clients (text completions) could
not use it. Extract the request_config parsing, approach/model resolution,
client building and dispatch of proxy() into a shared _execute_completion()
pipeline, and add POST /v1/completions on top of it:
- prompt (str or list[str]) maps to user messages; the pipeline runs once
per prompt and n still expands attempts into multiple choices
- returns the OpenAI legacy shape: id cmpl-*, object text_completion,
choices[].text/logprobs/finish_reason and usage; passthrough completion
dicts (none approach / full-response plugins / batch mode) are mapped
choice-by-choice
- stream=true returns 400 in this first version (chat endpoint streams
exactly as before); optillm_approach, max_tokens and other params pass
through like on the chat endpoint
- /v1/chat/completions responses are unchanged; conversation logging,
batch mode and error mapping (500 JSON) behave as before
Fixes #1701 parent 4088efc commit 086c678
3 files changed
Lines changed: 467 additions & 83 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
375 | 375 | | |
376 | 376 | | |
377 | 377 | | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
378 | 393 | | |
379 | 394 | | |
380 | 395 | | |
| |||
0 commit comments