deepseek_v3¶
DeepSeek-V3 on vLLM (vllm.model_executor.models.deepseek_v2).
vLLM runs DeepSeek-V2 and -V3 through one implementation. The names are
Llama's and the block is fused: it takes and returns the stream as
(hidden_states, residual). A block's mlp is the mixture of experts,
or a dense MLP on the first first_k_dense_replace blocks; either way its
output is what the block adds, and the routing is inside vLLM's fused MoE
kernel.
The attention is latent (MLA). By default vLLM runs it in a kernel of its own
(mla_attn), which works on the compressed keys and values and serves
nothing of the per-head queries, keys and values transformers reads; with
VLLM_MLA_DISABLE=1 it runs its ordinary attention layer on padded
tensors. Either way the attention's interior is not mapped here, and its
contribution, which is the module's output on both paths, is.
module-attribute
¶
module-attribute
¶
LATENT = "vLLM runs DeepSeek's latent attention in its own kernel, on compressed keys and values; not mapped on vLLM"
module-attribute
¶
ENVOYS = {DeepseekV2DecoderLayer: Layer, DeepseekV2MLAAttention: Attention, DeepseekV2Attention: Attention, DeepseekAttention: Attention, DeepseekV2MoE: Mlp, DeepseekV2MLP: Mlp}
Bases: FusedLayer
The decoder block: forward(positions, hidden_states, residual, llama_4_scaling) -> (hidden_states, residual).