exaone4¶
EXAONE 4.0 on vLLM (vllm.model_executor.models.exaone4).
Llama's names, but the block is not fused, whatever its signature says: it
is called forward(positions, hidden_states, residual), ignores the
residual it is handed, and returns (hidden_states, residual) with
the whole stream first (the second element is the stream after the
attention). It is post-norm: each sublayer runs on the stream itself and what
the block adds is the sublayer's norm's output, so the contributions point at
the sibling norms, as on transformers. The attention norms its queries and
keys, and rotates them only on the blocks the checkpoint's layer types say.