nemotron¶
Nemotron (Nemotron-4 / Minitron) on vLLM (vllm.model_executor.models.nemotron).
Llama's names and Llama's fused block, with vLLM's own classes: the block
takes and returns the stream as (hidden_states, residual), and its
NemotronLayerNorm1P norms (a LayerNorm whose weight is stored minus one)
take the residual beside the hidden state and add the two, as Llama's RMS
norms do. The rotary embedding covers partial_rotary_factor of each head
and the MLP has no gate, as on transformers.
module-attribute
¶
module-attribute
¶
ENVOYS = {NemotronDecoderLayer: Layer, NemotronAttention: Attention, NemotronMLP: Mlp}
Bases: FusedLayer
The decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual).