gpt_bigcode¶
GPT-BigCode on vLLM, through its transformers backend (GPTBigCodeForCausalLM -> TransformersForCausalLM).
vLLM has no GPT-BigCode of its own: it runs transformers' modules, so the
block is transformers' (see nnterp.components.vllm.transformers_backend),
with the stream [1, tokens, hidden_size] and the engine's attention layer
mounted as the attention's attn child. The tree is GPT-2's under
model rather than transformer (the backend builds the bare
GPTBigCodeModel): model.{wte, wpe, h[i].{ln_1, attn, ln_2, mlp}, ln_f}
and lm_head. The block adds both contributions itself and returns the
stream. Under multi_query (every released checkpoint) the keys and
values are one head shared by every query head, as the engine's layer is
built. token_embeddings is wte's output, before the positions are added.