olmo3

olmo3

OLMo 3 (Olmo3ForCausalLM).

Llama's names plus a post-norm block: every sublayer is normed after, not before, x + post_attention_layernorm(attn(x)) and then + post_feedforward_layernorm(mlp(x)). What the block adds is the post-norm's output, not the module's, so the contributions point at the sibling norms. OLMo 3 has only the post-norms, and mixes sliding-window and full attention layers: no input_layernorm exists to alias.

RENAME module-attribute

RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm'}

ENVOYS module-attribute

ENVOYS = {Olmo3DecoderLayer: Layer, Olmo3Attention: Attention, Olmo3MLP: Mlp}