hunyuan_v1_dense¶
Hunyuan dense V1 on vLLM (vllm.model_executor.models.hunyuan_v1).
Llama's names and a fused block, with a third element: it is called
forward(positions, hidden_states, residual, kv_states) and returns
(hidden_states, residual, kv_states), where the first two are the halves
of the stream and the third the attention's keys and values, handed to the
next block for cross-layer attention sharing (use_cla). The attention
returns (output, (keys, values)); the output is what the block adds. It
norms its queries and keys per head after the rotary embedding
(use_qk_norm), and the attention layer receives them after both, as on
transformers.
On a checkpoint with use_cla the blocks that reuse an earlier block's keys
and values run another attention module (HunYuanCrossAttention), which
this family does not map; the released dense checkpoints do not set it.
module-attribute
¶
module-attribute
¶
ENVOYS = {HunYuanDecoderLayer: Layer, HunYuanAttention: Attention, HunYuanMLP: Mlp}
Bases: FusedLayer
The decoder block: forward(positions, hidden_states, residual, kv_states) -> (hidden_states, residual, kv_states).