project

project

project: the logit lens on vLLM, with whichever module a family unembeds with.

project

project(model: Any, hidden: Tensor, head: Any, bias: Tensor | None = None) -> Tensor

Logits for hidden ([..., hidden_size]): the model's final norm, then vLLM's logits processor over head.

head is the module whose weight unembeds: lm_head where the model has one, the embedding where the two are tied into it (Gemma, Cohere). bias is the head's own, for a model whose logits add one (GPT-J, Phi). The processor applies the model's logit scale and softcap. The norm's kernel takes [rows, hidden_size], and a norm that takes the residual beside the hidden state hands back a pair.