Skip to content

kimi_k25

kimi_k25

Kimi K2.5 on vLLM (model_type kimi_k25): Kimi K2's language model inside the wrapper (vllm.model_executor.models.kimi_k25), text-only prompts.

Kimi K2's language model is DeepSeek-V3's, under Moonshot's kimi_k2 model type (see the transformers family). The wrapper's text_config says deepseek_v3, so the wrapper resolves by its own type, to this module (nnterp.families.resolve). vLLM runs a K2.5 checkpoint as KimiK25ForConditionalGeneration, whose language model is vLLM's own DeepseekV2ForCausalLM at language_model: the stack at language_model.model.{embed_tokens, layers, norm}, the head and the logits processor beside it. The block, the latent attention and the mixture of experts are deepseek_v3's classes and envoys, unchanged, under that prefix; what deepseek_v3 says of them (the fused block, the latent attention's interior not mapped, VLLM_MLA_DISABLE=1 for float32) holds here.

vLLM registers the wrapper as KimiK25ForConditionalGeneration; a checkpoint saved by transformers says Kimi_K25ForConditionalGeneration, so the engine needs hf_overrides={"architectures": ["KimiK25ForConditionalGeneration"]} to load it.

Text-only. The vision tower is not mapped and images are out of scope here; run the engine as a language model (language_model_only=True). In its multimodal mode vLLM embeds every prompt outside the model's forward and calls it with inputs_embeds and no ids, so input_ids and token_embeddings are not there to read.

RENAME module-attribute

RENAME = {'language_model.model.embed_tokens': 'embed_tokens', 'language_model.model.layers': 'layers', 'language_model.model.norm': 'norm', 'language_model.lm_head': 'lm_head', 'language_model.logits_processor': 'logits_processor'}