Cookbook llama.cpp prebuilt lookup can never match: /releases/latest skips prereleases, and the Linux asset patterns target nonexistent CUDA/.zip artifacts

[jaehho]

Prerequisites

• I searched open issues and discussions and did not find an existing report of this bug. (Cookbook prebuilt llama-server install fails — zipfile.ZipFile() cannot open a .tar.gz asset #5636 is adjacent — it reports the .tar.gz-into-zipfile extraction failure. This issue is about the three earlier failures that mean the URL is never found in the first place, so Cookbook prebuilt llama-server install fails — zipfile.ZipFile() cannot open a .tar.gz asset #5636's extractor is unreachable on Linux today.)
• This is not a security vulnerability.
• Reproduces on dev.

Odysseus Revision

upstream/dev @ affaee1

Install Method

Manual (uvicorn)

Operating System

Linux (Arch, kernel 7.2.2)

Steps to Reproduce

1. Download a GGUF via Cookbook → Download (unsloth/gpt-oss-20b-GGUF, Q4_K_M).
2. Cookbook → Serve, Engine: llama.cpp, on a host with no llama-server on PATH.
3. Launch.

Expected Behaviour

The prebuilt lookup in _append_llamacpp_runner_lines finds a matching llama.cpp release asset, downloads it, and llama-server starts in seconds.

Actual Behaviour

_odysseus_prebuilt_url is always empty, so every launch falls through to git clone + a from-source build. Mine printed:

Native llama-server not found — building from source (one-time, may take a few minutes)...
Cloning into 'llama.cpp'...

and then wedged: 3h20m elapsed, 17.7 MB fetched, byte-identical size 8 seconds apart. Chat showed [Error: Cannot reach http://localhost:8000] the whole time, which is accurate — nothing ever bound the port.

There are three independent reasons the URL is never found. Each alone is sufficient.

1. /releases/latest excludes prereleases; llama.cpp ships binaries only on prereleases.

routes/cookbook_helpers.py:875 queries:

https://api.github.com/repos/ggml-org/llama.cpp/releases/latest

GitHub defines that as the latest non-prerelease release. Current state of the repo:

v0.4.0    prerelease=false   assets=1    -> nightly-tag.txt
b10816    prerelease=true    assets=27   -> llama-b10816-bin-ubuntu-x64.tar.gz, ...
b10814    prerelease=true    assets=27
b10813    prerelease=true    assets=27

So the response contains exactly one browser_download_url, pointing at a text file. No pattern can match it. (v0.4.0 was published 2026-09-04T19:56:47Z. Before it existed the endpoint had no non-prerelease to return at all, so curl -f failed and the URL was empty for a different reason — the endpoint has no working state here either way.)

2. ubuntu.*cuda matches nothing — llama.cpp publishes no Linux CUDA binary.

Line 869 sets _odysseus_pat="ubuntu.*cuda" whenever nvidia-smi reports a GPU. Every CUDA asset is Windows-only:

llama-b10816-bin-win-cuda-12.4-x64.zip
llama-b10816-bin-win-cuda-13.3-x64.zip
llama-b10816-bin-win-cuda-13.4-arm64.zip

Checked against the full 27-asset list of b10816:

ubuntu.*cuda        -> NO MATCH
ubuntu.*vulkan      -> llama-b10816-bin-ubuntu-vulkan-x64.tar.gz

So every NVIDIA Linux host takes the branch that cannot match, and never reaches the vulkan branch that would have worked. This is the branch most likely to matter: NVIDIA + Linux is the common self-host configuration.

3. The CPU fallback pattern ends in .zip; Linux assets are .tar.gz.

Line 873 sets _odysseus_pat="ubuntu-x64\\.zip". The asset is llama-b10816-bin-ubuntu-x64.tar.gz:

ubuntu-x64\.zip     -> NO MATCH
ubuntu-x64\.tar\.gz -> llama-b10816-bin-ubuntu-x64.tar.gz

Only the Windows assets are .zip. This is the same .zip-vs-.tar.gz assumption behind #5636, one step earlier in the pipeline — here it stops the match, there it stops the extract.

Logs / Screenshots

$ curl -s https://api.github.com/repos/ggml-org/llama.cpp/releases/latest | jq -r '.tag_name, (.assets[].name)'
v0.4.0
nightly-tag.txt

$ curl -s "https://api.github.com/repos/ggml-org/llama.cpp/releases?per_page=5" \
    | jq -r '.[] | "\(.tag_name) prerelease=\(.prerelease) assets=\(.assets|length)"'
v0.4.0 prerelease=false assets=1
b10816 prerelease=true assets=27
b10814 prerelease=true assets=27
b10813 prerelease=true assets=27
b10809 prerelease=true assets=27

Model / Backend (if relevant)

llama.cpp / unsloth/gpt-oss-20b-GGUF Q4_K_M. Host: RTX 4050 Laptop 6 GB, 31 GB RAM, x86_64.

Are you willing to submit a fix?

Yes — happy to open a PR against dev if the approach below looks right.

Additional Information

Suggested fix, all in the same block at routes/cookbook_helpers.py:869-875:

• Query /releases?per_page=10 and take the first entry that actually carries a matching asset, instead of /releases/latest. That is prerelease-tolerant and self-corrects when upstream changes its release model again.
• Make the asset patterns .tar.gz-aware on Linux, and drop cuda from the Linux pattern set entirely since no such asset is published. Ordering that reflects what exists: ubuntu.*vulkan when a Vulkan loader and a render node are present, else ubuntu-x64\.tar\.gz.
• Since the Linux artifacts are tarballs, the extractor needs the tar path (Cookbook prebuilt llama-server install fails — zipfile.ZipFile() cannot open a .tar.gz asset #5636 covers this); bsdtar already handles both, plain unzip/zipfile handles neither.

Worth noting the practical consequence of #2: on NVIDIA Linux the best available prebuilt is the Vulkan build, not a CUDA one. That is still a large win over a from-source build — and a from-source CUDA build has its own trap, since nvcc is often installed but not on PATH (mine is at /opt/cuda/bin/nvcc), so the fallback silently produces a CPU-only binary. Related: #5606, #831.