cohere2¶
Cohere Command R7B and Aya on vLLM (vllm.model_executor.models.commandr).
vLLM runs Command R (cohere) and Command R7B / Aya (cohere2) through
one implementation. The names are Llama's. The block is parallel and is not
fused, whatever its signature says: it is called forward(positions,
hidden_states, residual), ignores the residual it is handed, runs the
attention and the MLP on the one normed input, and returns (hidden_states,
residual) with the whole stream first. There is no lm_head module
(the logits processor unembeds with embed_tokens' weight, which the
checkpoint ties, and applies the model's logit_scale), so
project_on_vocab is defined here.
module-attribute
¶
module-attribute
¶
project_on_vocab(model: StandardizedVLLM, hidden: Tensor) -> Tensor
The final norm, then vLLM's logits processor over embed_tokens, which is the unembedding here.