qwen3_vl_moe_text¶
Qwen3-VL-MoE, text (Qwen3VLMoeTextModel, model_type qwen3_vl_moe_text).
The language model inside Qwen3VLMoeForConditionalGeneration (model_type
qwen3_vl_moe): qwen3_vl_text's block, M-RoPE and DeepStack
(Layer.deepstack_output on the first blocks), with Qwen3-MoE's sparse
mixture of experts as the MLP (a softmax router, gate, aliased router;
dense blocks, Qwen3VLMoeTextMLP, where mlp_only_layers or
decoder_sparse_step say). Every checkpoint loads as the wrapper
(task="image-text-to-text"), whose text stack sits at model.language_model.
The wrapper's Qwen ViT model.visual is vision (a QwenVision, as on
qwen3_vl_text); its merger is projector.
module-attribute
¶
ENVOYS = {Qwen3VLMoeModel: ImageScatter, Qwen3VLMoeTextModel: TextModel, Qwen3VLMoeTextDecoderLayer: Layer, Qwen3VLMoeTextAttention: Attention, Qwen3VLMoeTextMLP: Mlp, Qwen3VLMoeTextSparseMoeBlock: Moe, Qwen3VLMoeVisionModel: QwenVision, Qwen3VLMoeVisionBlock: VisionLayer, Qwen3VLMoeVisionAttention: QwenVisionAttention, Qwen3VLMoeVisionMLP: VisionMlp}
Bases: Layer
Qwen3-VL-MoE's decoder block: qwen3_vl_text's, deepstack_output on the first blocks.