qwen2_moe

qwen2_moe

Qwen2-MoE (Qwen1.5-MoE) on vLLM (vllm.model_executor.models.qwen2_moe).

Llama's names and Llama's fused block. A block's mlp is the sparse mixture of experts, or a dense MLP on the blocks mlp_only_layers names; either way its output is what the block adds. The routing is inside vLLM's fused MoE kernel and has no values here.

RENAME module-attribute

RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm'}

ENVOYS module-attribute

ENVOYS = {Qwen2MoeDecoderLayer: Layer, Qwen2MoeAttention: Attention, Qwen2MoeMLP: Mlp, Qwen2MoeSparseMoeBlock: Mlp}

Layer

Layer(*args: Any, **kwargs: Any)

Bases: FusedLayer

The decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual).