qwen2_vl_text¶
Qwen2-VL, text (Qwen2VLTextModel, model_type qwen2_vl_text).
The language model inside Qwen2VLForConditionalGeneration (model_type
qwen2_vl): Qwen2's block under multimodal rotary embeddings (M-RoPE). The
position ids are three streams (temporal, height, width; a fourth, the text
positions, rides in front for the mask), which the model's rotary_emb
folds into one cos/sin pair by mrope_section before any block
runs. The attention then applies that pair the way Llama does, inside its
forward and before the shared attention interface, so attention_queries
and attention_keys are the rotated queries and keys, as on every rotary
family, and the base Attention holds. No causal-LM class is registered for
the config, so every checkpoint loads as the wrapper (task="image-text-to-text"),
whose text stack sits at model.language_model.
The wrapper's Qwen ViT model.visual is vision (a QwenVision: packed,
its patches [1, patches, vision_hidden_size]) with its blocks vision.layers;
its merger is projector, inside the tower.
module-attribute
¶
RENAME = {'model.language_model.embed_tokens': 'embed_tokens', 'model.language_model.layers': 'layers', 'model.language_model.norm': 'norm', 'model.visual': 'vision', 'model.visual.merger': 'projector', 'blocks': 'layers', 'attn': 'self_attn', 'norm1': 'input_layernorm', 'norm2': 'post_attention_layernorm', 'attn.proj': 'attn.o_proj', 'act': 'act_fn', 'fc2': 'down_proj'}
module-attribute
¶
ENVOYS = {Qwen2VLModel: ImageScatter, Qwen2VLDecoderLayer: Layer, Qwen2VLAttention: Attention, Qwen2MLP: Mlp, Qwen2VisionTransformerPretrainedModel: QwenVision, Qwen2VLVisionBlock: VisionLayer, Qwen2VLVisionAttention: QwenVisionAttention, Qwen2VLVisionMlp: VisionMlp}