The README chart is a simple memory-scaling model. It compares linear KV-cache compression systems against Catalyst Brain's fixed public SDK state.
Sources used:
- TurboQuant: Google Research / arXiv, "Online Vector Quantization for
Efficient High-Throughput LLM Inference" (
arXiv:2504.19874, https://arxiv.org/abs/2504.19874). The chart uses the reported 3.5 bits/channel quality-neutral point as the modeled memory ratio versus FP16. - KIVI: "A Tuning-Free Asymmetric 2bit Quantization for KV Cache"
(
arXiv:2402.02750, https://arxiv.org/abs/2402.02750). The chart models raw 2-bit KV storage versus FP16. - PyramidKV: "Dynamic KV Cache Compression based on Pyramidal Information
Funneling" (
arXiv:2406.02069, https://arxiv.org/abs/2406.02069). The chart uses the paper's 12% KV retention setting. - Catalyst Brain HKVC: fixed 4096-dim state measured through the public
catalyst-brainSDK APIs inscripts/generate_scaling_chart.py.
This repository deliberately exposes only adapter code, public benchmark models, and public SDK calls. The closed-source Catalyst Brain SDK remains the engine.