logit lens

Normally you only see the model's guess at the very end. The logit lens is a way to peek at what the model would guess if you stopped it partway up, layer by layer. In the early layers the guesses are mostly nonsense; they sharpen towards the end.

Reading the residual stream partway up the model as if it were the top, to see what the model would predict at that point.

The model turns the residual stream into next-token probabilities only at the top: a final normalisation, then the unembedding, which scores every token in the vocabulary. The logit lens applies those same two steps to the stream after an earlier layer. It shows what the model "would say" there, which is often nothing sensible until late; it is a readout, not a claim about what the layer is doing.

The model's last step is a normalisation followed by the unembedding, which scores the stream against every token (logits). The logit lens (nostalgebraist, 2020) applies those same two steps to the stream after an earlier layer, giving a distribution at every depth. Because the stream is only ever added to, later layers often keep the output's way of encoding things, and the readout there is informative. Earlier it can mislead, since the stream may be organised for the layers still to come rather than for the output. A readout shows what the output head would make of the stream at that depth, not what the layer computes or "thinks".

$\mathrm{lens}(x_l) = \mathrm{softmax}(E \, \mathrm{RMSNorm}(x_l))$, with the final norm's learned scale and, in Qwen3.5-4B, the tied table $E$ (tied weights). Introduced by nostalgebraist (2020, interpreting GPT: the logit lens). It is biased in early layers, whose basis differs from the final one; the tuned lens (Belrose et al. 2023, Eliciting Latent Predictions from Transformers with the Tuned Lens) learns an affine map per layer to correct for it. In the lab's workbench (raindrop-water-qwen, raindrop-methane-qwen), the top 5 at the answer position in the middle depths are mostly formatting, Chinese and partial tokens, which the lab shows as they are. A readout is a measurement, not a causal claim: what a depth contributes is tested by intervening on it, as activation patching does.