gpt_oss¶
GPT-OSS on vLLM (vllm.model_executor.models.gpt_oss).
vLLM's names are its own: model.embedding is embed_tokens and each
block's attention is attn (aliased to self_attn), whose own attn
child is vLLM's attention layer, as on every family. The block is fused but
takes the stream first: forward(hidden_states, positions, residual) ->
(hidden_states, residual). Its MLP is a mixture of experts whose output is
what the block adds; the routing is inside vLLM's fused MoE kernel.
The attention has a sink per head: a learned logit that joins the softmax
as one extra key column, inside vLLM's kernel, and is dropped afterwards. The
recomputed pattern does the same with the module's sinks, so its rows sum
to less than one, as on transformers; attention_scores are the masked
scores before the sink joins them, also as on transformers.
module-attribute
¶
RENAME = {'model.embedding': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm', 'attn': 'self_attn'}
module-attribute
¶
ENVOYS = {TransformerBlock: Layer, OAIAttention: Attention, MLPBlock: Mlp}
Bases: FusedLayer
The decoder block: forward(hidden_states, positions, residual) -> (hidden_states, residual).