ernie4_5

ernie4_5

ERNIE 4.5 dense on vLLM (vllm.model_executor.models.ernie45).

vLLM runs ERNIE 4.5 as a Llama: Ernie4_5ForCausalLM subclasses its LlamaForCausalLM, builds Llama's own modules and then switches each attention's rotary embedding to the interleaved (not NeoX) layout and drops the output projection's bias, which is what transformers' module computes. So the names, the fused block and the classes are Llama's.

RENAME module-attribute

RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm'}

ENVOYS module-attribute

ENVOYS = {LlamaDecoderLayer: Layer, LlamaAttention: Attention, LlamaMLP: Mlp}

Layer

Layer(*args: Any, **kwargs: Any)

Bases: FusedLayer

The decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual).