gptj¶
GPT-J on vLLM (vllm.model_executor.models.gpt_j).
Its tree is transformers' (transformer.{wte, h[i].{ln_1, attn, mlp},
ln_f} plus lm_head). The block is parallel: it is called with the
positions and the residual stream, runs the attention and the MLP on the one
normed input, and returns the stream with both added. The head has a bias,
which project_on_vocab adds as the model does.
module-attribute
¶
RENAME = {'transformer.wte': 'embed_tokens', 'transformer.h': 'layers', 'transformer.ln_f': 'norm', 'ln_1': 'input_layernorm', 'attn': 'self_attn', 'out_proj': 'o_proj', 'act': 'act_fn', 'fc_out': 'down_proj'}
project_on_vocab(model: StandardizedVLLM, hidden: Tensor) -> Tensor
The final norm, then vLLM's logits processor over lm_head with its bias, as the model's own logits are.