gemma¶
Gemma 1 (GemmaForCausalLM).
Llama's tree and Llama's block: model.{embed_tokens, layers[i].{input_layernorm,
self_attn, post_attention_layernorm, mlp}, norm} and lm_head, the residual
added in the block, attention through the shared eager forward.
PaliGemma puts a SigLIP tower at model.vision_tower, which is vision (its
post_layernorm over the patches is vision.norm), and a linear projector
model.multi_modal_projector, which is projector; vision.image_features
is read at the scatter (ImageScatter). Its processor refuses a prompt without
an image: a text-only trace takes the tokenizer's encoding.
module-attribute
¶
RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm', 'model.language_model.embed_tokens': 'embed_tokens', 'model.language_model.layers': 'layers', 'model.language_model.norm': 'norm', 'model.vision_tower': 'vision', 'model.multi_modal_projector': 'projector', 'embeddings.patch_embedding': 'patch_embed', 'encoder.layers': 'layers', 'post_layernorm': 'norm', 'layer_norm1': 'input_layernorm', 'layer_norm2': 'post_attention_layernorm', 'out_proj': 'o_proj', 'activation_fn': 'act_fn', 'fc2': 'down_proj'}
module-attribute
¶
ENVOYS = {GemmaDecoderLayer: Layer, GemmaAttention: Attention, GemmaMLP: Mlp, SiglipVisionModel: Vision, SiglipEncoderLayer: VisionLayer, SiglipAttention: VisionAttention, SiglipMLP: VisionMlp, PaliGemmaModel: ImageScatter}