qwen2¶
Qwen2 / Qwen2.5 (Qwen2ForCausalLM).
Llama's tree and Llama's block: model.{embed_tokens, layers[i].{input_layernorm,
self_attn, post_attention_layernorm, mlp}, norm} and lm_head, the residual
added in the block, attention through the shared eager forward.
llava-interleave (a llava wrapper) and LLaVA-OneVision put a SigLIP tower at
model.vision_tower, which is vision (its post_layernorm over the patches
is vision.norm), and the projector model.multi_modal_projector, which is
projector. OneVision feeds the projector the image's crops and then unpads
its output and adds newline tokens, so vision.image_features, read at the
scatter (ImageScatter), is not the projector's output there. Both set
vision_feature_layer -1 with strategy "full": the projector reads the last
block's layer_output before post_layernorm, so a vision.tower_output
write does not reach the text model.
module-attribute
¶
RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm', 'model.language_model.embed_tokens': 'embed_tokens', 'model.language_model.layers': 'layers', 'model.language_model.norm': 'norm', 'model.vision_tower': 'vision', 'model.multi_modal_projector': 'projector', 'embeddings.patch_embedding': 'patch_embed', 'encoder.layers': 'layers', 'post_layernorm': 'norm', 'layer_norm1': 'input_layernorm', 'layer_norm2': 'post_attention_layernorm', 'out_proj': 'o_proj', 'activation_fn': 'act_fn', 'fc2': 'down_proj'}
module-attribute
¶
ENVOYS = {Qwen2DecoderLayer: Layer, Qwen2Attention: Attention, Qwen2MLP: Mlp, SiglipVisionModel: Vision, SiglipEncoderLayer: VisionLayer, SiglipAttention: VisionAttention, SiglipMLP: VisionMlp, LlavaModel: ImageScatter, LlavaOnevisionModel: ImageScatter}