Real measured numbers for local LLM inference on a Strix Halo box: decode pace, time to first answer token, prompt-cache effect, sustained-load drift, quant and backend comparisons. Every value below links to a permanent claim page with its scope, limitations and sealed raw runs — nothing here is hand-typed, the tables are generated from the claim registry.
The box: Beelink GTR9 Pro — AMD Ryzen AI Max+ 395, Radeon 8060S (gfx1151), 128 GB LPDDR5X-8000 unified memory. Two commercially identical units. Runtime: llama.cpp server (Vulkan and ROCm backends), build pinned by image digest. Models: pinned by artifact hash.
Quick answers, from the tables below:
- How fast is a 35B-class MoE on Strix Halo? Sub-second first answer token and comfortable single-reader decode on Qwen3.6-35B-A3B Q4_K_M (llama.cpp Vulkan) — exact figures in the interactive table.
- Q8_0 or Q4_K_M? Same box, same build: quality gates saw no difference, Q8_0 decodes slower. Full write-up.
- Vulkan or ROCm? Vulkan decodes faster in this configuration; details and limits in the comparison.
- Does it throttle all day? Three continuous hours at concurrency 4: drift between the first and last five minutes is in the endurance table. Write-up.
- Can a team share one box? The concurrency ladder (1 → 4 → 8) is measured in the capacity report.
| Metric | Model | Backend | Concurrency | Value | Evidence level | Claim page |
|---|---|---|---|---|---|---|
| answerless (empty) responses in default mode | gemma-4-26B-A4B Q4_0 | llama.cpp Vulkan | 1 | 8.3 % of requests | lab_repeated | strix.gemma4.interactive2.c1.answerless-default |
| time to first answer token in default mode | gemma-4-26B-A4B Q4_0 | llama.cpp Vulkan | 1 | 10768 ms | lab_repeated | strix.gemma4.interactive2.c1.ttfa-default |
| time to the first token of any output | gemma-4-26B-A4B Q4_0 | llama.cpp Vulkan | 1 | 282 ms | lab_repeated | strix.gemma4.interactive2.c1.ttft-any-token |
| answerless (empty) responses at a 1k budget | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 58.3 % of requests | lab_unit_replicated | strix.qwen36.interactive2.c1.answerless-1k |
| request completion | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 100.0 % of requests | lab_unit_replicated | strix.qwen36.interactive2.c1.completion-nothink |
| time to first answer token | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 210 ms | lab_unit_replicated | strix.qwen36.interactive2.c1.ttfa-nothink |
| time to first answer token with reasoning on | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 20191 ms | lab_unit_replicated | strix.qwen36.interactive2.c1.ttfa-thinking |
| time to the first token of any output | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 212 ms | lab_unit_replicated | strix.qwen36.interactive2.c1.ttft-any-token |
| request completion | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 4 | 100.0 % of requests | lab_repeated | strix.qwen36.interactive2.c4.completion-nothink |
| time to first answer token | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 4 | 338 ms | lab_repeated | strix.qwen36.interactive2.c4.ttfa-nothink |
| request completion | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 8 | 100.0 % of requests | lab_repeated | strix.qwen36.interactive2.c8.completion-nothink |
| time to first answer token | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 8 | 865 ms | lab_repeated | strix.qwen36.interactive2.c8.ttfa-nothink |
| inter-token latency | Qwen3.6-35B-A3B Q4_K_M | llama.cpp ROCm | 1 | 18.7 ms/token | lab_repeated | strix.qwen36q4.rocm.c1.itl-nothink |
| time to first answer token | Qwen3.6-35B-A3B Q4_K_M | llama.cpp ROCm | 1 | 215 ms | lab_repeated | strix.qwen36q4.rocm.c1.ttfa-nothink |
| inter-token latency | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 15.9 ms/token | lab_repeated | strix.qwen36q4.vulkan.c1.itl-nothink |
| inter-token latency | Qwen3.6-35B-A3B Q8_0 | llama.cpp Vulkan | 1 | 18.7 ms/token | lab_repeated | strix.qwen36q8.vulkan.c1.itl-nothink |
| Metric | Model | Backend | Concurrency | Value | Evidence level | Claim page |
|---|---|---|---|---|---|---|
| time to first token, first question over a 32k document | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 33940 ms | lab_repeated | strix.qwen36.docsession.c1.ttft-q1-32k |
| time to first token, second question with prompt cache on | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 860 ms | lab_repeated | strix.qwen36.docsession.c1.ttft-q2-32k-cache |
| time to first token, second question with prompt cache off | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 33728 ms | lab_repeated | strix.qwen36.docsession.c1.ttft-q2-32k-nocache |
| time to first token, second question over an 8k document, cache on | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 660 ms | lab_repeated | strix.qwen36.docsession.c1.ttft-q2-8k-cache |
| Metric | Model | Backend | Concurrency | Value | Evidence level | Claim page |
|---|---|---|---|---|---|---|
| request completion over three hours | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 4 | 100.0 % of requests | lab_single_run | strix.qwen36.endurance.c4.completion-180m |
| decode-pace drift over three hours | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 4 | 1.6 % | lab_single_run | strix.qwen36.endurance.c4.itl-drift-180m |
| inter-token latency | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 4 | 30.0 ms/token | lab_single_run | strix.qwen36.endurance.c4.itl-median |
| Metric | Model | Backend | Concurrency | Value | Evidence level | Claim page |
|---|---|---|---|---|---|---|
| unanswerable-control honesty | gemma-4-26B-A4B Q4_0 | llama.cpp Vulkan | 1 | 75.0 % of requests | lab_repeated | strix.gemma4.longctx.c1.control-success |
| needle retrieval success | gemma-4-26B-A4B Q4_0 | llama.cpp Vulkan | 1 | 95.8 % of requests | lab_repeated | strix.gemma4.longctx.c1.needle-success |
| unanswerable-control honesty | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 100.0 % of requests | lab_unit_replicated | strix.qwen36.longctx.c1.control-success |
| needle retrieval success | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 100.0 % of requests | lab_repeated | strix.qwen36.longctx.c1.needle-success |
| time to first token at a 2k-token document | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 1907 ms | lab_repeated | strix.qwen36.longctx.c1.ttft-2k-en |
| time to first token at a 32k-token document | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 33965 ms | lab_repeated | strix.qwen36.longctx.c1.ttft-32k-en |
| Metric | Model | Backend | Concurrency | Value | Evidence level | Claim page |
|---|---|---|---|---|---|---|
| strict-JSON task success | gemma-4-26B-A4B Q4_0 | llama.cpp Vulkan | 1 | 93.8 % of requests | lab_repeated | strix.gemma4.structured.c1.task-success |
| end-to-end time to a complete answer | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 844 ms | lab_repeated | strix.qwen36.structured.c1.e2e-nothink |
| end-to-end time with reasoning on | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 14677 ms | lab_repeated | strix.qwen36.structured.c1.e2e-think4k |
| strict-JSON task success | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 100.0 % of requests | lab_repeated | strix.qwen36.structured.c1.task-success-nothink |
| strict-JSON task success with reasoning on | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 100.0 % of requests | lab_repeated | strix.qwen36.structured.c1.task-success-think4k |
| strict-JSON task success | Qwen3.6-35B-A3B Q4_K_M | llama.cpp ROCm | 1 | 100.0 % of requests | lab_repeated | strix.qwen36q4.rocm.c1.task-success |
| strict-JSON task success | Qwen3.6-35B-A3B Q8_0 | llama.cpp Vulkan | 1 | 100.0 % of requests | lab_repeated | strix.qwen36q8.vulkan.c1.task-success |
| Metric | Model | Backend | Concurrency | Value | Evidence level | Claim page |
|---|---|---|---|---|---|---|
| answerless (empty) responses at a 1k budget | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 75.0 % of requests | lab_repeated | strix.qwen36.interactive.c1.answerless-1k |
| time to first answer token | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 220 ms | lab_repeated | strix.qwen36.interactive.c1.ttfa-nothink |
| time to first answer token with reasoning on | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 23453 ms | lab_repeated | strix.qwen36.interactive.c1.ttfa-thinking |
| time to the first token of any output | Qwen3.6-35B-A3B Q4_K_M | llama.cpp Vulkan | 1 | 221 ms | lab_repeated | strix.qwen36.interactive.c1.ttft-any-token |
- Frozen workloads with published corpora; failed requests stay in the denominator.
- Three repeated runs per cell (evidence level
lab_repeated), or the level the table states;lab_unit_replicatedmeans the pattern reproduced on both identical units. - Quality gates grade the answer text (format, language, repetition, groundedness), not the HTTP status.
- Harness, corpora and gate tests are open: agmind-bench. Sealed run bundles: agmind-lab.
- Reproduce a claim on your own Strix Halo hardware — confirmed external reproductions raise the claim's evidence level, failed ones get published too.
- Got a number that disagrees? Submit it — the config delta behind a disagreement is usually more informative than either figure.
- Prompt cache: why the second question over a document is instant
- Model, quant and backend swaps on one box
- What reasoning mode costs before the first visible token
- Cyrillic token cost: the same answer is more expensive in Russian
- Empty answers under HTTP 200: the answerless failure mode
Data: CC BY 4.0 — take the tables, keep the claim links. Maintained by AGmind Systems Lab · Telegram