standard

standard

Standard, the envoy every component derives from, the axis names every layout is spelled with, the two helpers for tuple outputs, and the walk support makes over a block list.

HIDDEN module-attribute

HIDDEN = 'hidden_size'

VOCAB module-attribute

VOCAB = 'vocab_size'

HEADS module-attribute

HEADS = 'num_heads'

KV_HEADS module-attribute

KV_HEADS = 'num_kv_heads'

HEAD_DIM module-attribute

HEAD_DIM = 'head_dim'

QK_HEAD_DIM module-attribute

QK_HEAD_DIM = 'qk_head_dim'

INTERMEDIATE module-attribute

INTERMEDIATE = 'intermediate_size'

EXPERTS module-attribute

EXPERTS = 'num_experts'

TOP_K module-attribute

TOP_K = 'top_k'

BATCH module-attribute

BATCH = 'batch'

SEQ module-attribute

SEQ = 'seq'

TOKENS module-attribute

TOKENS = 'tokens'

QUERY module-attribute

QUERY = 'query'

KEY module-attribute

KEY = 'key'

STREAMS module-attribute

STREAMS = 'streams'

MIXER_HEADS module-attribute

MIXER_HEADS = 'heads'

KEY_DIM module-attribute

KEY_DIM = 'key_dim'

VALUE_DIM module-attribute

VALUE_DIM = 'value_dim'

STATE_DIM module-attribute

STATE_DIM = 'state_dim'

GROUPS module-attribute

GROUPS = 'groups'

CHANNELS module-attribute

CHANNELS = 'channels'

IMAGES module-attribute

IMAGES = 'images'

PATCHES module-attribute

PATCHES = 'patches'

IMAGE_TOKENS module-attribute

IMAGE_TOKENS = 'image_tokens'

VISION_HIDDEN module-attribute

VISION_HIDDEN = 'vision_hidden_size'

Standard

Standard(*args: Any, **kwargs: Any)

Bases: Envoy

An envoy carrying standard values: what Layer, Attention and Mlp share.

A value that is an operation inside a forward is served only on a call whose forward was instrumented before the call began. Instrumenting on first read is enough when the read comes before the module runs, which is the usual case; it is too late when the worker is already parked inside the module, as it is when a block's mlp_output (an operation in the block's forward) is read after the block's attention_output. A family whose value sits in such a forward sets sourced = True on that envoy, and the forward is instrumented when the envoy is built and again when real weights replace meta ones, which reinstalls the plain forward.

sourced class-attribute instance-attribute

sourced = False

values classmethod

values() -> dict[str, EProperty]

This class's standard values by name, base classes first.

support

support() -> dict[str, str | None]

Each standard value here -> None when available, else the reason.

first_tensor

first_tensor(value: Any) -> Tensor

The tensor of a module output: the first element when it is a tuple.

rewrap

rewrap(envoy: Envoy, value: Tensor) -> Any

value in the shape the module returned: back in its tuple, if any.

module_int

module_int(module: Any, *names: str) -> int | None

The first of names the module holds as an integer, or None.

in_width

in_width(module: Any, *names: str) -> int | None

The input width of the first of names the module has as a projection (nn.Linear, or transformers' Conv1D), or None.

absent

absent(envoy: Envoy, name: str, where: str = 'block') -> str | None

Why a value at the child module name is unavailable: the envoy has no such child or alias. None when it has.

unsized

unsized(envoy: Envoy, name: str) -> NotImplementedError

The error a size raises when the module spells it no way the base reads.

axis_size

axis_size(envoy: Envoy, axis: str) -> int

The size of layout axis axis on a value envoy serves.

An axis that is a fact about the model is named as the size that holds it (HIDDEN is "hidden_size", HEADS "num_heads"), so its size is that attribute: on envoy when its class declares it (an attention's num_heads, a mixture's num_experts), else on the first envoy above it that does, up to the model's root (hidden_size, vocab_size)::

axis_size(model.layers[3], HIDDEN)          # model.hidden_size
axis_size(model.layers[3].self_attn, HEADS)  # this block's heads

Only an envoy's class counts (a property, the root's StandardizedProperty), never an attribute of the wrapped module, which can share a name and hold another number.

RAISES	DESCRIPTION
ValueError	no envoy from envoy up to the root declares axis: it is not a fact about the model (the batch, the sequence, the pattern's keys), and its size is the tensor's.

values

values(cls: type) -> dict[str, EProperty]

A class's standard values by name, base classes first.

standard_children

standard_children(block: Envoy) -> dict[str, Standard]

The block's children that carry standard values, by standard name (the alias where one is bound).

block_hosts

block_hosts(blocks: Any) -> dict[str, list[str]]

Standard-value hosts across every block: module name -> value names, in first-seen order.

The union over the blocks, so a hybrid lists both self_attn and linear_attn and a block lacking one reports it as missing; a module no block has (OPT's mlp) is not listed.

block_support

block_support(block: Any, hosts: dict[str, list[str]]) -> dict[str, str | None]

One block's values by dotted name, over hosts: None when available, else the reason.

blocks_support

blocks_support(blocks: Any, layer: int | None = None) -> dict[str, Any]

The block values of a block list, the way support reports them.

With layer, that block's values by dotted name ("layer_output", "self_attn.attention_probabilities"). Without, each value -> None when available on every block, else {layer: reason} for the blocks where it is not. Shared by the text model's blocks and a vision tower's.