ernie4_5_moe¶
ERNIE 4.5 MoE on vLLM (vllm.model_executor.models.ernie45_moe).
Llama's names and Llama's fused block: it takes and returns the stream as
(hidden_states, residual). A block's mlp is the mixture of experts on
the blocks from moe_layer_start_index to moe_layer_end_index (every
moe_layer_interval-th), a dense MLP on the others; either way its output is
what the block adds, the shared experts' included, as on transformers. The
routing is inside vLLM's fused MoE kernel and has no values here.
module-attribute
¶
module-attribute
¶
ENVOYS = {Ernie4_5_MoeDecoderLayer: Layer, Ernie4_5_MoeAttention: Attention, Ernie4_5_MoeMoE: Mlp, Ernie4_5_MoeMLP: Mlp}
Bases: FusedLayer
The decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual).