deepseek_v3¶
DeepSeek-V3 (DeepseekV3ForCausalLM).
Llama's tree and Llama's block: model.{embed_tokens, layers[i].{input_layernorm,
self_attn, post_attention_layernorm, mlp}, norm} and lm_head, the residual
added in the block, attention through the shared eager forward. Multi-head latent attention as in V2; dense MLPs for the first first_k_dense_replace blocks, a mixture of experts after.
Moonshot's Kimi K2.5 / K2.6 / K2.7-Code (Kimi_K25ForConditionalGeneration) wrap this text
model under model.language_model with lm_head at the root, so RENAME carries both
spellings; their text_config says kimi_k2, which kimi_k2.py maps to this family.
module-attribute
¶
RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm', 'model.language_model.embed_tokens': 'embed_tokens', 'model.language_model.layers': 'layers', 'model.language_model.norm': 'norm', 'gate': 'router'}
module-attribute
¶
ENVOYS = {DeepseekV3DecoderLayer: Layer, DeepseekV3Attention: Attention, DeepseekV3MLP: Mlp, DeepseekV3MoE: Moe}