qwen2_5_vl_text¶
Qwen2.5-VL's language model on vLLM (vllm.model_executor.models.qwen2_5_vl), text-only prompts.
As on qwen2_vl_text: vLLM runs the whole
Qwen2_5_VLForConditionalGeneration, whose language model is vLLM's own
Qwen2ForCausalLM at language_model, so the classes and the fused
block ((hidden_states, residual)) are qwen2's under that prefix. The
positions are M-RoPE's [3, tokens]; on a text-only prompt the three rows
are the same 1-D positions, so the queries and keys are plain Qwen2's.
Text-only. The vision tower is not mapped and images are out of scope here;
run the engine as a language model (language_model_only=True, or every
limit_mm_per_prompt at 0). In its multimodal mode vLLM embeds every
prompt outside the model's forward and calls it with inputs_embeds and
no ids, so input_ids and token_embeddings are not there to read.
module-attribute
¶
RENAME = {'language_model.model.embed_tokens': 'embed_tokens', 'language_model.model.layers': 'layers', 'language_model.model.norm': 'norm', 'language_model.lm_head': 'lm_head', 'language_model.logits_processor': 'logits_processor'}
module-attribute
¶
ENVOYS = {Qwen2DecoderLayer: Layer, Qwen2Attention: Attention, Qwen2MLP: Mlp}
Bases: FusedLayer
Qwen2's decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual), with M-RoPE positions.