bloom¶
BLOOM on vLLM (vllm.model_executor.models.bloom).
Its tree is transformers' (transformer.{word_embeddings,
word_embeddings_layernorm, h[i].{input_layernorm, self_attention,
post_attention_layernorm, mlp}, ln_f} plus lm_head). The block is
called with the positions and the residual stream and returns the stream,
adding both contributions itself; vLLM's attention and MLP modules return
their contributions alone, where transformers' add the residual inside. The
attention is ALiBi's: no rotary embedding, a per-head slope on the scores.
module-attribute
¶
RENAME = {'transformer.word_embeddings': 'embed_tokens', 'transformer.h': 'layers', 'transformer.ln_f': 'norm', 'self_attention': 'self_attn', 'dense': 'o_proj', 'gelu_impl': 'act_fn', 'dense_4h_to_h': 'down_proj'}