Skip to content

Latest commit

 

History

History
46 lines (27 loc) · 3.3 KB

File metadata and controls

46 lines (27 loc) · 3.3 KB

exp-attn-fft-perplexity — What happens if you actually compress attention using FFT?

Round 2 · Compression attempts


What we were asking

Given that attention patterns are spectrally concentrated, can you actually compress them — keep only the low-frequency FFT coefficients, discard the rest — without the model falling apart? This is the direct practical test of the compression idea.

The analogy

A JPEG image compressed at 90% quality looks almost identical to the original. Compressed at 10% quality it becomes blocky and blurred but is still recognisable. The question here is whether attention patterns behave like images — with a graceful degradation curve where you can trade quality for compression — or like encryption keys, where removing even a tiny fraction destroys everything.

Why this matters

KV-cache compression is a real problem in deployed LLMs. At long context lengths, storing all the key-value pairs from every attention head consumes enormous memory. If you could keep only 10% of the FFT coefficients per attention matrix, you'd get a 10× reduction in KV-cache memory with acceptable quality loss. This experiment tests whether that tradeoff exists.

What was tested

We passed GPT-2 through its normal forward pass, but at each layer intercepted the attention score matrices, applied FFT compression at varying retention ratios (keeping 50%, 20%, 10%, 5%, 2%, 1% of coefficients), and measured the resulting perplexity on a held-out text corpus. Baseline (uncompressed): perplexity ~48.

Results

Retention ratio Resulting perplexity
100% (baseline) 48
50% ~850
20% ~900
10% ~920
5% ~920

Every compression ratio tested produced catastrophic perplexity collapse. There is no "graceful degradation" curve — the model breaks immediately, and the damage does not grow with further compression. The failure happens at 50% and doesn't get meaningfully worse at 1%.

What this means

The spectral structure of attention patterns is an illusion of exploitability. Yes, 95% of the energy lives in 5% of the coefficients — but the remaining 5% of energy, spread across the discarded high-frequency components, is not noise. It carries the token-specific routing decisions: which exact token this position should attend to, among all the tokens in context. That information is high-frequency (it changes sharply from token to token) and cannot be discarded.

Think of it this way: the low-frequency envelope of an attention pattern encodes "this head generally attends to recent tokens." The high-frequency detail encodes "at this specific position, it should attend to token 7 rather than token 6." The second piece of information is what makes the model work. Discarding it for compression breaks the model immediately, regardless of how little spectral energy it carries.

The compression hypothesis is falsified. Attention patterns cannot be compressed in the frequency domain without destroying the model.

Open questions

  • Could a trained compression scheme — where the model learns to reconstruct high-frequency attention from compressed low-frequency information — succeed where post-hoc FFT truncation fails?
  • What is the minimum information needed per attention head per token to preserve model quality?