← Back to README · Architecture Guide · Model Hub & Engines · Flash-Stream Research
Welcome to the benchmarking and empirical verification center of intel-arc-models-runner. Metrics in this documentation are labelled as measured, derived, or projected; measured values come from real Intel silicon (Intel® Core™ Ultra 5 125H with Xe-LPG iGPU and AI Boost NPU 3720) or derived from foundational mathematical principles.
The table below is generated by scripts/render_bench_docs.py from the schema-v1 files in docs/benchmarks/results/. Method, metric definitions and claim thresholds are in METHOD.md.
Which statistic each cell is. The value before +/- is the median of the n timed (non-warm-up) samples. The number after +/- is not a confidence interval of that median: it is the half-width of the 95% Student-t interval of the arithmetic mean (student_t_ci95, computed on the same samples). So a cell reads "median +/- (Student-t 95% half-width of the mean)"; the column headers below that say "median +/- 95% CI" are shortened labels from the generator and should be read that way. No bootstrap interval, IQR or non-normality flag is printed here (they can be computed from the result files). Decode tok/s is derived from OpenVINO counters, not read from a separate meter, even where the Source column says measured.
| Model | Backend (labels) | Result file | n | TTFT ms (median +/- 95% CI) | Decode tok/s (median +/- 95% CI) | Source |
|---|---|---|---|---|---|---|
| qwen-2.5-1.5b/hard | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 73.7 +/- 1.74 | 40.1 +/- 0.295 | measured |
| qwen-2.5-1.5b/hard | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 170 +/- 34.5 | 13.7 +/- 1.1 | measured |
| qwen-2.5-1.5b/hard | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 2233 +/- 59.9 | 14.4 +/- 0.734 | measured |
| qwen-2.5-1.5b/low | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 20.1 +/- 0.631 | 52.2 +/- 1.92 | measured |
| qwen-2.5-1.5b/low | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 355 +/- 112 | 13.4 +/- 2.35 | measured |
| qwen-2.5-1.5b/low | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 2250 +/- 57.1 | 15.4 +/- 1.12 | measured |
| qwen-2.5-1.5b/medium | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 25 +/- 0.729 | 41.4 +/- 0.864 | measured |
| qwen-2.5-1.5b/medium | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 72 +/- 6.16 | 13.1 +/- 1.12 | measured |
| qwen-2.5-1.5b/medium | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261029T111424Z.v1.json | 10 | 2271 +/- 232 | 14.7 +/- 0.386 | measured |
| qwen-2.5-1.5b/hard | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 47.2 +/- 3.56 | 50.3 +/- 3.06 | measured |
| qwen-2.5-1.5b/hard | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 188 +/- 53.7 | 12.2 +/- 1.76 | measured |
| qwen-2.5-1.5b/hard | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 2298 +/- 66.1 | 13.8 +/- 1.16 | measured |
| qwen-2.5-1.5b/low | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 20.4 +/- 0.437 | 50.5 +/- 0.999 | measured |
| qwen-2.5-1.5b/low | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 313 +/- 60 | 13.6 +/- 3.19 | measured |
| qwen-2.5-1.5b/low | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 2274 +/- 28.5 | 15.1 +/- 0.571 | measured |
| qwen-2.5-1.5b/medium | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 19.5 +/- 0.649 | 47.2 +/- 1.84 | measured |
| qwen-2.5-1.5b/medium | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 83.1 +/- 17.3 | 12.1 +/- 0.933 | measured |
| qwen-2.5-1.5b/medium | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261029T162643Z.v1.json | 10 | 2252 +/- 40.1 | 14.5 +/- 0.354 | measured |
| qwen-2.5-1.5b/hard | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 71.4 +/- 37.2 | 36.3 +/- 1.67 | measured |
| qwen-2.5-1.5b/hard | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 276 +/- 54.4 | 13.1 +/- 1.14 | measured |
| qwen-2.5-1.5b/hard | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 2377 +/- 200 | 13.5 +/- 0.77 | measured |
| qwen-2.5-1.5b/low | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 23.8 +/- 4.37 | 46.2 +/- 2.87 | measured |
| qwen-2.5-1.5b/low | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 261 +/- 20.9 | 18.2 +/- 1.73 | measured |
| qwen-2.5-1.5b/low | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 2339 +/- 18.8 | 15 +/- 0.341 | measured |
| qwen-2.5-1.5b/medium | arc-runner (OpenVINO on GPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 25 +/- 12.8 | 41.4 +/- 2.47 | measured |
| qwen-2.5-1.5b/medium | cpu-baseline (OpenVINO on CPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 106 +/- 25 | 13.3 +/- 1.16 | measured |
| qwen-2.5-1.5b/medium | npu (OpenVINO on NPU) | llm_qwen-2.5-1.5b_20261030T070037Z.v1.json | 30 | 2321 +/- 35.2 | 14.7 +/- 0.44 | measured |
| Workload | Model | Backend (labels) | Result file | n | Metric | Value (median +/- 95% CI) | Source |
|---|---|---|---|---|---|---|---|
| audio_stt | whisper-tiny/hard | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | audio_duration_s | 18 +/- 0 | measured |
| audio_stt | whisper-tiny/hard | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | latency_s | 0.0882 +/- 0.00512 | measured |
| audio_stt | whisper-tiny/hard | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | rtf | 0.0049 +/- 0.000284 | measured |
| audio_stt | whisper-tiny/hard | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | throughput_audio_s_per_s | 204 +/- 11.4 | measured |
| audio_stt | whisper-tiny/hard | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | audio_duration_s | 18 +/- 0 | measured |
| audio_stt | whisper-tiny/hard | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | latency_s | 0.331 +/- 0.0108 | measured |
| audio_stt | whisper-tiny/hard | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | rtf | 0.0184 +/- 0.0006 | measured |
| audio_stt | whisper-tiny/hard | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | throughput_audio_s_per_s | 54.4 +/- 1.66 | measured |
| audio_stt | whisper-tiny/low | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | audio_duration_s | 3 +/- 0 | measured |
| audio_stt | whisper-tiny/low | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | latency_s | 0.062 +/- 0.00243 | measured |
| audio_stt | whisper-tiny/low | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | rtf | 0.0207 +/- 0.000809 | measured |
| audio_stt | whisper-tiny/low | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | throughput_audio_s_per_s | 48.4 +/- 1.98 | measured |
| audio_stt | whisper-tiny/low | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | audio_duration_s | 3 +/- 0 | measured |
| audio_stt | whisper-tiny/low | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | latency_s | 0.29 +/- 0.0103 | measured |
| audio_stt | whisper-tiny/low | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | rtf | 0.0966 +/- 0.00343 | measured |
| audio_stt | whisper-tiny/low | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | throughput_audio_s_per_s | 10.4 +/- 0.368 | measured |
| audio_stt | whisper-tiny/medium | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | audio_duration_s | 8.5 +/- 0 | measured |
| audio_stt | whisper-tiny/medium | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | latency_s | 0.08 +/- 0.00584 | measured |
| audio_stt | whisper-tiny/medium | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | rtf | 0.00941 +/- 0.000687 | measured |
| audio_stt | whisper-tiny/medium | arc-runner (OpenVINO on GPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | throughput_audio_s_per_s | 106 +/- 7.32 | measured |
| audio_stt | whisper-tiny/medium | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | audio_duration_s | 8.5 +/- 0 | measured |
| audio_stt | whisper-tiny/medium | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | latency_s | 0.308 +/- 0.0129 | measured |
| audio_stt | whisper-tiny/medium | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | rtf | 0.0363 +/- 0.00152 | measured |
| audio_stt | whisper-tiny/medium | cpu-baseline (OpenVINO on CPU) | audio_stt_whisper-tiny_20261029T123205Z.v1.json | 10 | throughput_audio_s_per_s | 27.6 +/- 1.06 | measured |
| image | stable-diffusion-v1-5/low | arc-runner (OpenVINO on GPU (FP16 IR)) | image_stable-diffusion-v1-5_20261030T082445Z.v1.json | 3 | iteration_rate_it_s | 10.6 +/- 0.972 | measured |
| image | stable-diffusion-v1-5/low | arc-runner (OpenVINO on GPU (FP16 IR)) | image_stable-diffusion-v1-5_20261030T082445Z.v1.json | 3 | latency_s | 11.3 +/- 1.67 | measured |
| image | stable-diffusion-v1-5/low | arc-runner (OpenVINO on GPU (FP16 IR)) | image_stable-diffusion-v1-5_20261030T082445Z.v1.json | 3 | megapixels_per_s | 0.0233 +/- 0.00344 | measured |
| image | stable-diffusion-v1-5/low | arc-runner (OpenVINO on GPU (FP16 IR)) | image_stable-diffusion-v1-5_20261030T082445Z.v1.json | 3 | step_latency_ms | 94 +/- 8.12 | measured |
| vlm | qwen2-vl-7b/hard | arc-runner (OpenVINO on GPU (INT4)) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | latency_s | 15.6 +/- 2.17 | measured |
| vlm | qwen2-vl-7b/hard | arc-runner (OpenVINO on GPU (INT4)) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | output_chunk_count | 104 +/- 16.9 | measured |
| vlm | qwen2-vl-7b/hard | cpu-baseline (OpenVINO on CPU) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | latency_s | 85.1 +/- 11.7 | measured |
| vlm | qwen2-vl-7b/hard | cpu-baseline (OpenVINO on CPU) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | output_chunk_count | 96 +/- 0 | measured |
| vlm | qwen2-vl-7b/low | arc-runner (OpenVINO on GPU (INT4)) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | latency_s | 1.23 +/- 0.026 | measured |
| vlm | qwen2-vl-7b/low | arc-runner (OpenVINO on GPU (INT4)) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | output_chunk_count | 12 +/- 0 | measured |
| vlm | qwen2-vl-7b/low | cpu-baseline (OpenVINO on CPU) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | latency_s | 8.75 +/- 0.731 | measured |
| vlm | qwen2-vl-7b/low | cpu-baseline (OpenVINO on CPU) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | output_chunk_count | 12 +/- 0 | measured |
| vlm | qwen2-vl-7b/medium | arc-runner (OpenVINO on GPU (INT4)) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | latency_s | 2.12 +/- 0.0374 | measured |
| vlm | qwen2-vl-7b/medium | arc-runner (OpenVINO on GPU (INT4)) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | output_chunk_count | 15 +/- 0 | measured |
| vlm | qwen2-vl-7b/medium | cpu-baseline (OpenVINO on CPU) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | latency_s | 13.9 +/- 1.67 | measured |
| vlm | qwen2-vl-7b/medium | cpu-baseline (OpenVINO on CPU) | vlm_qwen2-vl-7b_20261029T122931Z.v1.json | 10 | output_chunk_count | 15 +/- 0 | measured |
| Document | Focus & Scope | Key Content |
|---|---|---|
| 1. Comparative Benchmark Matrix | Legacy (UNVERIFIED) Model Matrix & Heavy-Model Tables | Tables 1–6 covering LLMs, 5W NPU Embeddings, Multimodal VLM, Audio AI, Image Generation, Video AI; Table 7 (Flash-Stream 32B/72B), which came from a synthetic 8-layer container, not real 32B/72B weights, and a Leviathan speculative-efficiency derivation (a formula; speed-up was never measured). |
| 2. Mathematical Proofs (M59–M134) | Formal Mathematical Theorems | 70 formal proofs covering Audio AI (M59–M72), Text-to-Image (M73–M93), Video AI (M94–M114), and Flash-Stream Zero-RAM Heterogeneous Execution (M121–M134). |
| 3. Empirical Silicon Proofs | Physical Hardware Telemetry | Testbed specs, Head-to-head Arc GPU vs Host CPU (legacy 30.8× TTFT figure, not reproduced: measured ratios are 2.3×–17.7×), live sequential reclaim benchmark, hardware integration tests, Standalone GPU vs NPU breakdown (M50–M55), Section 6: Flash-Stream memory trace from a synthetic container (mapped bytes, not resident memory), and Section 7: Live Ollama CPU vs Arc GPU multi-tier showdown. |
| 4. Ollama Showdown & 30-Case Audit | Empirical Showdown vs. Ollama | Ollama "Silent CPU Fallback" anatomy (Proof M56), head-to-head telemetry, Shannon Token Entropy (M57–M58), 30-case Triad Audit, Section 5: Heavy Models Ollama OOM Collapse vs Flash-Stream Zero-RAM, and Section 6: Side-by-side prompt responses. |
- Host CPU: Intel® Core™ Ultra 5 125H (Meteor Lake; 18 logical CPUs per
probe.json; the core count is not captured by the probe) - Host GPU: Intel® Arc™ Graphics (Xe-LPG integrated GPU; the Xe-core count and XMX/DPAS presence are not captured by
probe.jsonand are unverified) - Host NPU: Intel® AI Boost NPU 3720 (power not measured on this testbed)
- Host RAM: 15.43 GB total physical RAM (from
scratch/live_real_models_benchmark_results.json, not fromprobe.json; Windows 11 AMD64 build 26220 perprobe.json) - Driver Stack: driver versions are not recorded (
driver_versionsis empty in every result file andprobe.jsonhas no driver version); OpenVINO GenAI 2026.4.1.0 perprobe.json - Measured so far (result files): Qwen2.5-1.5B (LLM, GPU/NPU/CPU), Whisper-tiny (STT, GPU/CPU), Qwen2-VL-7B (VLM, GPU/CPU). Embeddings, image, video and an Ollama baseline are not yet measured, and no real 32B/72B run exists.