Round 1 · Spectral diagnostics
Do GPT-2's internal activations (the hidden states that flow through the model layer by layer) change in their spectral character across depth? Does information get compressed, expanded, or scrambled as it moves through the network?
Imagine listening to a conversation being passed through a relay of twelve people. The first person hears a complex, detailed message. By the time it reaches the last person, has it been simplified into a few key points, or is it still carrying the full detail? Spectral analysis of activations is one way to measure this — high-frequency content corresponds to fine-grained detail; low-frequency content corresponds to broad structure.
The spectral profile of activations tells you something about what information different layers specialise in. If early layers are high-frequency and late layers are low-frequency, the network is progressively compressing information into global summaries. This has implications for where to place architectural interventions — and for understanding what different layers are actually doing.
We extracted hidden state activations at each layer for 50 sentences through GPT-2, treating each activation vector as a 1D signal and computing its FFT. We measured what fraction of spectral energy lives in the top 1% and top 10% of frequency coefficients at each layer.
The profile is U-shaped, not monotonically compressing:
- Early layers: moderately compressible (~50% energy in top 1%)
- Middle layers (L5–L9): similar, slightly improving
- Layer 11 anomaly: 92.3% of energy in the top 1% of coefficients — dramatically more concentrated than any other layer
For context, positional embeddings (which encode position rather than meaning) show 93.9% energy in the top 1% — nearly identical to the L11 anomaly.
The final layer of GPT-2 produces activations that look spectrally almost identical to positional embeddings — highly regular, globally structured, low-frequency. This is likely driven by the final layer norm before the output head, which normalises the activations and may collapse their spectral content into a more uniform form.
The U-shape means the network is not simply compressing information as depth increases. Middle layers may be doing the bulk of the semantic work, with the final layer acting more like a normalisation gateway before vocabulary prediction.
- Is the Layer 11 anomaly caused by the final LayerNorm, or by the layer's learned weights?
- Does the U-shaped profile persist in larger models, or does it flatten out?