Skip to content

Latest commit

 

History

12 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Inference Optimization Mastery

A production-grade, measured study of LLM inference optimization — serving, batching, KV cache, quantization, distillation, speculative decoding, compiled runtimes, and GPU capacity sizing.

Method, not marketing: every result is a delta vs a baseline with its quality tradeoff attached, produced by a real GPU run and stamped with {GPU, model, precision, sample size, date}. No number appears here that didn't come from an actual run. See docs/research/ for the algorithm/decision-logic notes and docs/specs/ for the design.

Scope honesty: this is a rigorous benchmark study on a laptop GPU (RTX 3060, 6 GB, Ampere) plus a cloud appendix — not a live production service. P0–P5 run locally; P6 is sizing math validated by one cloud point; P7 (FP8 / TensorRT-LLM / Triton / 7B) requires a rented Hopper/Ada GPU.

Architecture

workloads/bench/loadgen → swappable engines/ (HF · vLLM · TensorRT-LLM) → bench/metricsbench/report (+ quality anchor) → results/<phase>/ (committed reports + charts).

Headline results (measured)

Phase Optimization Result GPU
P0 baseline (HF vs vLLM), DeepSeek-R1-Distill-Qwen-1.5B vLLM 187.7 vs HF 13.7 tok/s = 13.7×; TPOT 15 ms vs 314 ms (20.9×); GSM8K 0.75 = 0.75 lossless RTX 3060 6GB
P1 continuous batching + KV cache max_num_seqs 1→16 = 5.8× throughput (71.7→414.9 tok/s), TTFT 57s→0.32s, quality flat; prefix caching −20% TTFT (lossless) RTX 3060 6GB
P2 INT4 quantization (AWQ/GPTQ) AWQ/GPTQ int4 = 1.9× throughput + 2.6× KV-cache headroom, quality = fp16 (lossless) RTX 3060 6GB
P3 knowledge distillation (1.5B→0.5B SeqKD) eval-gate caught a regression (0.40→0.27, teacher–student capacity-gap); reliable win = 2.1× throughput / ⅓ memory of the smaller model RTX 3060 6GB
P4 speculative decoding measured regression (2.5× slower TPOT, lossless) — wrong tool for a 1.5B target on 6GB; spec-decode needs a large target + high acceptance (→ P7) RTX 3060 6GB
P5 agentic e2e @ SLO (stacked) INT4-AWQ + prefix caching = 1.7× throughput / 40% lower $/token / lower p95 at equal concurrency & preserved quality (fair baseline; the big 13.7× is vLLM-vs-HF in P0) RTX 3060 6GB
P6 GPU sizing unit-tested sizing calculator anchored on measured 3060, validated (KV math reproduces measured capacity; H100 proj ~11k tok/s matches published); projects L40S/A100/H100 fleet size + $/1M-token calc + measured anchor
P7 FP8 / TensorRT-LLM / Triton (7B) code + runbook complete; numbers PENDING a cloud run (FP8 needs Hopper/Ada — honest gap) cloud (L40S/A100)
+X cross-arch: Liquid LFM2 (hybrid) vs transformer LFM2 = 2.33× less KV/token → 2.36× more concurrency (6 attn + 10 conv vs 28 attn), widens with context; config-derived. Live GPU serving driver-gated (needs CUDA-13 driver) config analysis

Run

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e ".[dev]"
.venv/bin/pytest                       # 20 GPU-free unit tests
.venv/bin/python scripts/run_baseline.py        # P0; other phases: scripts/run_*.py

What this demonstrates (the through-line)

A reproducible, honesty-stamped methodology applied across the inference-optimization surface, with the judgment to tell wins from mis-applications:

  • Big honest win: naive HF Transformers → vLLM = 13.7× throughput (P0, conservative — HF never batches); then INT4-AWQ + prefix caching add 1.7× more / 40% lower $/token at equal concurrency and preserved quality (P5). Decomposed and individually measured (P1 batching scaling curve, P2 1.9×+2.6× KV) — not a single inflated multiple against a strawman baseline.
  • Honest negatives that show seniority: distillation into a 0.5B regressed quality — caught by the eval-gate (P3); speculative decoding regressed a 1.5B target — recognized as the wrong tool, not shipped (P4). Both with diagnosed root causes + "when it flips."
  • Capacity engineering: a unit-tested GPU-sizing calculator, validated against the measured anchor (P6); a cloud appendix (FP8/TensorRT-LLM) delivered as runnable code + runbook with numbers honestly pending (P7).

Every result: stamped report.json + chart + RESULTS.md + a staff-level ADR (context → measured options → decision → when it flipswhat we skipped). 9 ADRs, 20 GPU-free unit tests, per-phase commits.

About

Measured study of LLM inference optimization on real GPU hardware — vLLM serving, continuous batching, INT4 quant, distillation, speculative decoding, and GPU capacity sizing. Every number from an actual run, with its quality tradeoff attached.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages