qwen3_5_text¶
Qwen3.5 / 3.6 text (Qwen3_5ForCausalLM, model_type qwen3_5_text).
Llama's tree with a hybrid block: three blocks in four carry a gated DeltaNet
mixer, linear_attn, the fourth ordinary attention, self_attn
(config.layer_types). Each block has one or the other, never both, so on
a linear block every self_attn value is reported missing and the linear
values live at layers[i].linear_attn (see nnterp.LinearAttention). Every
released checkpoint is a qwen3_5 wrapper (Qwen3_5ForConditionalGeneration):
task="text-generation" (the default) builds Qwen3_5ForCausalLM out of it, and
task="image-text-to-text" the wrapper, whose text stack sits at
model.language_model; RENAME carries both spellings.
On the wrapper the Qwen ViT model.visual is vision (a QwenVision: packed,
[1, patches, vision_hidden_size], with a learned pos_embed added after patch_embed
and no deepstack), its blocks vision.layers; its merger is projector.
module-attribute
¶
RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm', 'model.language_model.embed_tokens': 'embed_tokens', 'model.language_model.layers': 'layers', 'model.language_model.norm': 'norm', 'model.visual': 'vision', 'model.visual.merger': 'projector', 'blocks': 'layers', 'attn': 'self_attn', 'norm1': 'input_layernorm', 'norm2': 'post_attention_layernorm', 'attn.proj': 'attn.o_proj', 'linear_fc2': 'down_proj'}
module-attribute
¶
ENVOYS = {Qwen3_5Model: ImageScatter, Qwen3_5DecoderLayer: Layer, Qwen3_5Attention: Attention, Qwen3_5GatedDeltaNet: LinearAttention, Qwen3_5MLP: Mlp, Qwen3_5VisionModel: QwenVision, Qwen3_5VisionBlock: VisionLayer, Qwen3_5VisionAttention: QwenVisionAttention, Qwen3_5VisionMLP: VisionMlp}