phi¶
Phi-1 and Phi-2 on vLLM (vllm.model_executor.models.phi).
Its tree is transformers' (model.{embed_tokens, layers[i].{input_layernorm,
self_attn, mlp}, final_layernorm} plus lm_head). The block is parallel:
it is called with the positions and the residual stream, runs the attention
and the MLP on the one normed input, and returns the stream with both added.
The head has a bias, which project_on_vocab adds as the model does.
module-attribute
¶
RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.final_layernorm': 'norm', 'dense': 'o_proj', 'act': 'act_fn', 'fc2': 'down_proj'}
project_on_vocab(model: StandardizedVLLM, hidden: Tensor) -> Tensor
The final norm, then vLLM's logits processor over lm_head with its bias, as the model's own logits are.