feat: lmcache-turbo-quant-serde — tqai KV-cache compression plugin for LMCache v1

[pbertsch]

Summary

Adds integrations/lmcache/ — a pip-installable package (lmcache-turbo-quant-serde) that bridges tqai's PolarQuantizer into LMCache's v1 distributed serde plugin system.

pip install -e integrations/lmcache   # or PyPI once published

import lmcache_turbo_quant_serde
lmcache_turbo_quant_serde.register()   # call once at startup
# Then configure: {"type": "tqai", "head_dim": 128, "bits": 4}

What's in the package

File	Purpose
_wire.py	Self-describing binary format: magic TQ01, bits/seed/shape/dtype embedded — no config needed to decompress
_codec.py	Standalone TurboQuantSerializer / TurboQuantDeserializer (to_bytes / from_bytes) — usable without LMCache
_v1_codec.py	TqaiSerializer / TqaiDeserializer conforming to lmcache.v1.distributed.serde.base
_register.py	register("tqai") via register_serde_factory; idempotent, non-conflicting with LMCache's own turboquant
tests/	47 tests: wire / codec / integration / e2e
benchmarks/bench_serde.py	Compression ratio, cosine quality, throughput table

Key design decisions

• Reshape before quantize: LMCache stores KV as [2, L, T, H×D]; PolarQuantizer expects head_dim as the last axis, so TqaiSerializer.serialize reshapes to [2, L, T, H, head_dim], stores that shape in the wire, and TqaiDeserializer merges back
• Self-describing wire format: deserializer needs zero config — bits, seed, head_dim, dtype, shape are all embedded in the compressed blob
• Graceful import: _v1_codec.py falls back to local ABCs when lmcache isn't installed, so the package imports cleanly without lmcache's full dep chain
• @lru_cache on quantizer: PolarQuantizer holds a head_dim×head_dim Haar rotation matrix; cache avoids rebuilding it across calls

Benchmark results (CPU, bfloat16, LLaMA-3 8B)

Note: 3-bit perf numbers below assume PR #12 (bitstream vectorization) is merged first. Without it, 3-bit is ~1900× slower.

Bits	Size vs fp16	Cosine ↑	Ser (256 tok)	Des (256 tok)
4-bit	25.8%	0.9954	80 ms	17 ms
3-bit	19.5%	0.9831	~80 ms	~17 ms
2-bit	13.3%	0.9405	48 ms	16 ms

Test plan

• pytest integrations/lmcache/tests/ — 47/47 pass
• E2E roundtrip via real create_serde_processor (LMCache v1 async path)
• Compression ratios: 4-bit < 38%, 3-bit < 29%, 2-bit < 22%
• Cosine similarity ≥ 0.95 for bits ≥ 3
• register() is idempotent; does not affect existing serdes (fp8)
• Package imports cleanly without lmcache installed (falls back to local ABCs)

Depends on: PR #12 for full 3-bit performance

🤖 Generated with Claude Code