Qwen3.8-Flash-Next on one laptop CPU: native C, automatic 8 GB path

[shyringo]

I built qwen3.8-flash-next-in-c, a purpose-built native C runtime for the Unsloth Qwen3.8-Flash-Next-UD-IQ1_S split GGUF.

It runs Qwen3.8-Flash-Next on a single laptop CPU without a GPU, Python, PyTorch, model conversion, or another inference runtime. The engine implements the Qwen4Exp graph directly, including the 51B PLE table, four-stream Hyper-Connections, Gated DeltaNet, QSA and 512-expert MoE. It provides terminal chat and a resident OpenAI-compatible local API with SSE and function tools.

Measured on an Intel Core i5-1340P / 32 GB Windows 11 + WSL2 laptop:

• resident single-conversation chat: 5.03 token/s, TPOT 0.199 s/token
• exact batch-4 verification: 9.89 positions/s
• automatic 8 GB path: 1.64 token/s, 596 MiB peak RSS for a 19-token prompt plus 16-token output at context 2,048

The near-10 result is aggregate exact-batch throughput, not single-chat TPOT. The batch benchmark first computes a token-major reference and reports speed only when all four greedy IDs and the complete final-logit buffer match byte for byte. The 8 GB path also preserves the same 248,320-logit SHA.

The Quick Start downloads the pinned Unsloth shards from ModelScope with resume and SHA-256 verification, then opens chat:

https://github.com/shyringo/qwen3.8-flash-next-in-c

Thank you for publishing the quantized shards that made this single-CPU path practical. Results from other laptop CPUs, native Linux and ARM64/macOS are welcome in the repository's benchmark thread.