exaone4¶
EXAONE 4.0 (Exaone4ForCausalLM).
Llama's names plus a post-norm block, OLMo-2's layout: every sublayer is normed
after, x + post_attention_layernorm(attn(x)) and then
+ post_feedforward_layernorm(mlp(x)). What the block adds is the post-norm's
output, not the module's, so the contributions point at the sibling norms.
EXAONE 4.0 has only the post-norms: no input_layernorm exists to alias.
The attention runs the shared eager forward with per-head q/k norms. A checkpoint
with a sliding_window mixes sliding-window layers, which apply the rotary, and
full-attention layers, which apply none (NoPE); both reach the interface the same
way, so the values are the same on every block.