phi3

phi3

Phi-3 on vLLM (vllm.model_executor.models.phi3).

vLLM runs Phi-3 as a Llama: Phi3ForCausalLM subclasses its LlamaForCausalLM and builds Llama's own modules, so the names, the fused block and the classes are Llama's.

RENAME module-attribute

RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm'}

ENVOYS module-attribute

ENVOYS = {LlamaDecoderLayer: Layer, LlamaAttention: Attention, LlamaMLP: Mlp}

Layer

Layer(*args: Any, **kwargs: Any)

Bases: FusedLayer

The decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual).