gpt2

gpt2

GPT-2 on vLLM (vllm.model_executor.models.gpt2).

Its tree is transformers' (transformer.{wte, wpe, h[i].{ln_1, attn, ln_2, mlp}, ln_f} plus lm_head) and its block is not fused: it is called with the residual stream and returns it, adding both contributions itself.

RENAME module-attribute

RENAME = {'transformer.wte': 'embed_tokens', 'transformer.h': 'layers', 'transformer.ln_f': 'norm', 'ln_1': 'input_layernorm', 'attn': 'self_attn', 'ln_2': 'post_attention_layernorm', 'attn.c_proj': 'attn.o_proj', 'mlp.c_proj': 'mlp.down_proj', 'act': 'act_fn'}

ENVOYS module-attribute

ENVOYS = {GPT2Block: Layer, GPT2Attention: Attention, GPT2MLP: Mlp}

intermediate_size

intermediate_size(model: StandardizedVLLM) -> int

The MLP width is n_inner, None meaning four times the hidden size.