Skip to content

Standardizing Interpretability

nnterp gives every transformer family the same module names and the same standard values, on top of nnsight. An experiment written once runs on GPT-2, Llama, Qwen, Gemma, Mamba and the other supported families.

Get Started Guide Tutorials Documentation About

What is NNterp?

StandardizedTransformer is an nnsight TransformersModel, so trace, generate, .save(), tracer.iter and remote=True work as they do in nnsight. What changes is the names that you write and the values that you read.

Every family answers to Llama's names: model.embed_tokens, model.layers[i], model.layers[i].self_attn, model.layers[i].mlp, model.norm, model.lm_head. Because they are aliases, GPT-2's model.transformer.h[5] and model.layers[5] are the same envoy and the native names keep working.

A name says which module; a value says which tensor, and it means the same thing on every family. layer_output is the residual stream leaving a block whether the block returns a tensor or a tuple. attention_output and mlp_output are what each sublayer adds to the stream, so on every family

layers[i].input + attention_output + mlp_output == layers[i].layer_output

Every supported family has a page in the encyclopedia: its block drawn with nnterp's names, every value and where it is read, and what support() says for its checkpoints.

Not every checkpoint has every value. Before you trace, model.support() says what this one has and, for anything missing, why.

pip install nnterp

from nnterp import StandardizedTransformer

model = StandardizedTransformer("openai-community/gpt2")   # or a Llama, a Pythia, ...

with model.trace("The Eiffel Tower is in"):
    attn = model.layers[5].self_attn.attention_output.save()   # what attention adds
    resid = model.layers[5].layer_output.save()                # the residual stream leaving block 5
    logits = model.logits.save()

print(attn.shape, resid.shape, logits.shape)
# torch.Size([1, 7, 768]) torch.Size([1, 7, 768]) torch.Size([1, 7, 50257])

Add remote=True to run the same block on a model hosted by NDIF. An agent learns nnterp from the skills in ndif-team/skills, and every page here is also served as Markdown (llms.txt).