falcon¶
Falcon on vLLM (vllm.model_executor.models.falcon).
Its tree is transformers' (transformer.{word_embeddings, h[i].{input_layernorm
or ln_attn + ln_mlp, self_attention, mlp}, ln_f} plus lm_head). The
block is called with the positions and the residual stream and returns the
stream. Its attention and its MLP each return (output, bias): vLLM keeps
the output projection's bias apart so that a parallel block can sum the two
outputs before one all-reduce, and adds it in the block. The contributions
are the first element, which is the whole contribution on a checkpoint whose
projections have no bias (Falcon-7B and -40B); on one that has (bias in
the config, Falcon-RW) they are unavailable.
On a parallel block the block adds the attention into the MLP's output
tensor in place; mlp_output is read before that, as a copy, like every
value on this engine.
module-attribute
¶
RENAME = {'transformer.word_embeddings': 'embed_tokens', 'transformer.h': 'layers', 'transformer.ln_f': 'norm', 'self_attention': 'self_attn', 'dense': 'o_proj', 'act': 'act_fn', 'dense_4h_to_h': 'down_proj'}
module-attribute
¶
BIAS = "the output projection has a bias, which vLLM's block adds apart from the module's output; not mapped yet"
module-attribute
¶
Where the MLP's normed input is read: its own norm on a sequential block, else the norm the attention shares or ln_mlp.