arcee

arcee

Arcee (AFM) on vLLM (vllm.model_executor.models.arcee).

Llama's names and Llama's fused block: it takes and returns the stream as (hidden_states, residual). The block is vLLM's own class, its attention is Llama's LlamaAttention, and its MLP is vLLM's ungated one (down_proj(relu2(up_proj(x)))), as on transformers.

RENAME module-attribute

RENAME = {'model.embed_tokens': 'embed_tokens', 'model.layers': 'layers', 'model.norm': 'norm'}

ENVOYS module-attribute

ENVOYS = {ArceeDecoderLayer: Layer, LlamaAttention: Attention, ArceeMLP: Mlp}

Layer

Layer(*args: Any, **kwargs: Any)

Bases: FusedLayer

The decoder block: forward(positions, hidden_states, residual) -> (hidden_states, residual).