Catalyst KV Cache

Research drop-in KV-cache adapter backed by the closed-source, monetized catalyst-brain SDK.

This repository is source-available for non-commercial research and evaluation. Production, enterprise, hosted, revenue-generating, or pilot use requires a written license or pilot agreement. Contact:

hello@strategic-innovations.ai

What It Is

catalyst-kv-cache gives researchers a minimal drop-in adapter for testing Catalyst's fixed-state holographic KV-cache approach without source access to the closed SDK. It installs the public catalyst-brain wheel from PyPI and uses only public SDK APIs. The public adapter shows integration behavior and results; the core Catalyst engine remains inside the monetized SDK.

Scaling Results

The breakthrough is not another linear compression curve. TurboQuant, KIVI, and PyramidKV are strong recent KV-cache compression systems, but they still scale with context length. Catalyst Brain HKVC uses fixed holographic state through the public catalyst-brain SDK surface, so the modeled state stays flat as tokens grow.

Model assumptions for the chart:

• Baseline FP16 KV cache: tokens * 40 layers * 4096 hidden * 2 K/V tensors * 2 bytes
• TurboQuant: modeled from the published 3.5 bits/channel quality-neutral point
• KIVI: modeled as 2-bit KV versus FP16
• PyramidKV: modeled from the published 12% KV retention setting
• Catalyst Brain HKVC: measured from the public SDK as fixed 4096-dim Catalyst state

At 128K tokens under this model:

Method	Modeled memory	Shape
FP16 KV cache	83.89 GB	Linear
TurboQuant	18.35 GB	Linear compressed
KIVI 2-bit	10.49 GB	Linear compressed
PyramidKV	10.07 GB	Linear retained
Catalyst Brain HKVC	0.016 MB	Fixed holographic state

The generated source data is in docs/kv_cache_scaling.csv; source notes are in docs/SOURCES.md. This repo intentionally shows only adapter code, integration behavior, and public SDK results. It does not expose Catalyst Brain trade secrets.

The first adapter is intentionally conservative:

• passthrough mode preserves model behavior by returning original key/value states from update(...) while recording fixed-size Catalyst state.
• refs mode returns compact references for serving stacks that explicitly opt into Catalyst-managed cache materialization.
• Hugging Face style methods such as update, get_seq_length, get_usable_length, reorder_cache, and reset are provided for research integration work.

Install the SDK

python -m pip install catalyst-brain

This repo is the strategically available adapter/demo layer. The monetized package users need from PyPI is catalyst-brain.

To run this adapter repo locally:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install catalyst-brain
python -m pip install -e ".[dev]"
pytest -q

Quick Drop-In Sketch

from catalyst_kv_cache import CatalystKVCache, CatalystKVConfig

cache = CatalystKVCache(CatalystKVConfig(mode="passthrough", dim=4096))

# Hugging Face style update signature.
key_states, value_states = cache.update(
    key_states=key_states,
    value_states=value_states,
    layer_idx=0,
    cache_kwargs={"position": 128, "token_count": 1},
)

print(cache.compression_report())
print(cache.to_rain_header(agent_id="research-run"))

Why Passthrough First

Passthrough mode is the safe research default. It lets a model run normally while Catalyst records compact HDC/Rain/HKVC state next to the normal cache. That creates apples-to-apples traces before a serving stack switches to refs mode and starts using Catalyst-managed cache references directly.

License Boundary

Allowed without a commercial agreement:

• Non-commercial research
• Academic experiments
• Personal evaluation
• Benchmark reproduction
• Pull requests and issue reports

Requires a written license or pilot agreement:

• Production inference
• Enterprise/internal commercial deployment
• Hosted APIs or SaaS
• Revenue-generating products
• Customer pilots
• Any use where Catalyst KV Cache is part of a paid workflow

Contact hello@strategic-innovations.ai.

Claim Discipline

This package exposes the drop-in adapter and fixed-state instrumentation powered by catalyst-brain. The benchmark suite lives at:

https://github.com/CrewRiz/catalyst-brain-benchmarks

Use the benchmark repo to reproduce payload reduction, HKVC query scaling, bind/unbind correctness, and memory-model comparisons for catalyst-brain.