Start with a row of numbers

Read the phrase grid load rises. The word “rises” is more useful when connected to what rises. In a model, that connection has to become a numerical operation. Attention lets one token state combine information from other token states.

If you have seen an embedding, start there: a vector is an ordered list of coordinates. Put one vector on each row of a matrix $X$. In a deeper layer, these rows are already contextual states produced by earlier computation. They need not remain the original embedding-table entries.

A recurrent model carries state along a sequence. Here we will instead calculate a set of weights from one row to the available rows, then form a mixture. We are tracing a forward calculation; the stored model parameters stay fixed.

TOY ASSUMPTION · CL-004

Our labels are grid, load, and rises, with invented states $[1,0]$, $[0,1]$, and $[1,1]$. They were chosen for arithmetic, not learned from text. Neither coordinate is a measured electrical quantity or a verified semantic feature. The example makes no claim about forecasting or document understanding.

Compare with keys; carry values

For each row, create three projections. A query compares with keys. A value is the vector that will enter the mixture. These are roles played by numeric states, not three kinds of token.

A projection is matrix multiplication by a parameter matrix. We use row vectors throughout. For our toy, choose these fixed matrices:

For “rises,” multiply $[1,1]$ by $W_K$: the first output is $1\cdot1+1\cdot0=1$; the second is $1\cdot1+1\cdot1=2$. So its key is $[1,2]$. The same row has query $[1,1]$ and value $[1,2]$.

The matching key and value coordinates for “rises” are a coincidence of these constants. Keys and values need not have the same width. In a trained model, learning changes the projection parameters; a new input changes the projected states even when those parameters are fixed.

S1 · §§3.2.2–3.2.3 CL-002, CL-003, CL-018

Read the shapes before the numbers

Let $n$ be the number of query rows, $m$ the number of key/value rows, $d_k$ the query/key width, and $d_v$ the value width. The transpose symbol $\mathsf T$ swaps a matrix’s rows and columns.

The first multiplication gives one comparison for every query/key pair. The second uses one weight per value row. Query and key widths must match; key and value row counts must match. $d_v$ may differ from $d_k$. In our self-attention example, $n=m=3$ and both widths happen to be 2.

We have candidates and comparisons. Next we need to turn a row of scores into a controlled mixture.

Derived shape contract · CL-003, from S1 equation (1).

Score one query against every key

Take the last query, “rises”: $q=[1,1]$. A dot product multiplies matching coordinates, then adds. Its raw scores against our three keys are:

For example, the third score is $1\cdot1+1\cdot2=3$. These are compatibility scores under the chosen projections. They are not probabilities or verified statements of importance. Dot products also depend on vector magnitude; they are not automatically cosine similarity.

The original scaled dot-product rule divides each score by $\sqrt{d_k}$. Here $d_k=2$, so the scaled scores are approximately $[1.414,0.707,2.121]$.

Why the square root? Under the illustrative assumption of independent, zero-mean, unit-variance query/key components, the variance of their dot product grows as $d_k$. Dividing by $\sqrt{d_k}$ keeps that variance at one. Large score differences can push softmax toward saturation. Learned states need not satisfy those assumptions, so the scaling is a motivated design choice, not a universal bound on scores or gradients.

S1 · §3.2.1 and its variance footnote CL-001, CL-005, CL-006, CL-025

Normalize across the available keys

Softmax turns the scaled scores $s_j$ into positive weights. Exponentiation makes the numbers positive; division makes the row sum to one. For now, all three positions are allowed.

The denominator is shared. A weight depends on its competitors’ scores as well as its own. For “rises,” this gives $a\approx[0.284,0.140,0.576]$. These probabilities are over key positions, not over the vocabulary and not over whether a sentence is true.

One row at a time, the matrix formula is:

For numerical stability, the lab subtracts the largest allowed score before taking exponentials. Multiplying every numerator and the denominator by the same factor leaves the weights unchanged. All displayed values are rounded to three decimals; the calculation keeps full floating-point precision.

S1 · §3.2.1, equation (1); stability is an algebraic derivation CL-001, CL-020

Mix vectors, not token labels

Now multiply each value vector by its weight and add the resulting vectors:

The output is approximately $[0.860,1.432]$. It is a new two-coordinate state. Nothing in this tiny operation decodes a word or decides what happens to a grid.

Because the weights are nonnegative and sum to one, the result is a convex mixture of the allowed values: a point inside their convex hull. Each output coordinate lies between that coordinate’s smallest and largest allowed value. The values themselves can be negative or exceed one; probability bounds belong to the weights.

Before the lab, make one prediction: if the query becomes the zero vector, will the output become zero too?

Derived arithmetic and convex-mixture property · CL-001, CL-005, CL-008, CL-022.

A zero query still receives something

With a zero query, every finite key gets score zero. Equal scores produce equal weights across the allowed positions. That need not make the weighted values cancel. Try the query multiplier at zero, then explain the two output coordinates.

This is a direct intervention on the selected $q$ after projection. A real change to the input could affect queries, keys, and values together. Here they are separated deliberately. The next control similarly replaces one value coordinate after its projection. Neither slider learns a parameter.

Three small experiments

1. Change the query. Keep “rises” selected and set its multiplier to zero. Every weight is $1/3$; the output is $[2/3,4/3]$. Increase the multiplier again. The largest fixed score gets a larger share, while tied scores remain tied. This is concentration, not a switch to hard selection.

2. Change what is visible. Select “grid.” With causal visibility, only its own value is allowed, so the output is exactly $[1,0]$. Disable causal visibility and later values can contribute. Return to “rises”: switching the mask has no effect because every position is already at or before the final query.

3. Change the carried content. Reset, then move the third value’s second coordinate from 2 to 4. The scores and weights stay identical. The second output coordinate increases by $2a_3\approx1.152$. The first output coordinate stays fixed. If that third key is blocked, the override has no effect on the output.

The attention map describes the mixing coefficients. The map alone does not tell you the resulting vector. That distinction will matter when we interpret a complete model.

Toy interventions and their consequences · CL-008, CL-009, CL-018. LAB-001 is the canonical calculation behind the live view and static comparisons.

Run the same calculation locally

The companion is a dependency-free JavaScript module. From this guide’s site folder, run node model.mjs to print the baseline and contrasts. To inspect one state in your own module, import the same pure function used by this page:

import { compute } from './model.mjs';
const before = compute();
const after = compute({ valueY: 4 });
console.log(before.weights, after.weights);
console.log(before.output, after.output);

The first pair matches; the second pair differs only in its second coordinate. Complete source and checks are linked in the notes. This run demonstrates the stipulated algebra, not model training or empirical prediction quality.

Remove a position before normalization

For a left-to-right prediction, a position must not consult future input tokens. The causal rule we use permits a key at position $j$ exactly when $j\le i$, where $i$ is the query position. The current input token is included.

In the formula, add a mask $M$ to the scaled scores: zero for allowed entries and negative infinity for blocked ones. Operationally, the lab exponentiates only allowed entries.

Blocking is not merely setting a score to zero: a zero score still gets $\exp(0)=1$ in the numerator. A blocked key gets exactly zero weight and is removed from the denominator. Renormalizing can change every remaining weight.

Every row in our causal mask has at least one allowed position. If you block all positions, there is no distribution to normalize. The companion rejects that case explicitly. This is a mathematical boundary, not a claim that all software libraries handle it identically.

S1 · §§3.1, 3.2.3 CL-007, CL-020

What a position-free mixture cannot notice

Imagine keeping a query fixed and moving each key together with its matching value. The scores and contributions simply move with their pairs. Adding the same terms in a different order gives the same output.

For all-visible self-attention with no positional information, permuting all input rows also permutes the query rows. The output therefore follows the same permutation: it is permutation-equivariant. It is not necessarily identical at each fixed row index.

For a short derivation, let $P$ be a matrix that reorders rows. Then the reordered weights are $PAP^{\mathsf T}$, and:

This argument requires all-visible attention, or a mask reordered consistently with the rows. A fixed triangular causal mask changes which token may consult which other token when the order changes. Do not apply the unrestricted permutation argument to that different case.

Derived from shared row-wise projections and CL-001 · CL-011.

Supply a position-dependent signal

The original Transformer adds a positional vector to each input embedding. Its chosen encoding uses sine/cosine pairs at different frequencies. If $p$ is the position index and $r$ indexes a coordinate pair, let $d_{\mathrm{model}}$ be the token-state width. The original formula is:

In width two, the pair at position zero is $[0,1]$ and at position one is $[\sin(1),\cos(1)]$. These are position-dependent numbers; adding them can make the same token at two positions produce different queries, keys, and values.

The paper also tested learned positional embeddings. Other choices exist; the sinusoidal design is the original choice described here. Our arithmetic lab does not add positional vectors. Its fixed rows and causal mask are deliberately exposed so there is no hidden position computation.

A mask and a position vector solve different parts of the problem. One restricts access; the other supplies features. Now we can place attention inside the rest of the architecture.

S1 · §3.5 CL-010

Several views of the same states

Our lab has one head. In multi-head attention, each head has its own query, key, and value projections. Each computes a separate mixture, then the outputs are concatenated along their feature dimension.

Here $H$ is the head count; $H_h$ is head $h$’s output. If each head returns width $d_v$, concatenation has width $Hd_v$. The output matrix $W_O$ maps that width back to $d_{\mathrm{model}}$, so it can be added to the input state.

Heads provide separately parameterized comparisons and mixtures. The architecture does not require one head to mean “grammar” and another “facts.” A readable semantic role would be a finding to investigate, not something the formula guarantees.

S1 · §3.2.2 CL-012, CL-024

Follow the bypass as well as the mixture

An encoder block in the original Transformer contains all-visible self-attention followed by a position-wise feed-forward sublayer. Each sublayer has a residual addition and then layer normalization. “Residual” means that the incoming state has an additive path around the sublayer.

These equations show the original Post-LN order, with dropout omitted for the inference-time schematic. The original training setup uses dropout on sublayer outputs before the addition and normalization. Pre-LN places normalization inside the residual branch; it is a distinct variant, not the layout drawn here.

S1 · §3.1, Figure 1; §5.4 CL-013

S4 · abstract: normalization placement CL-016

Layer normalization acts across a token’s feature coordinates in this block. It centers and rescales them, then applies learned coordinate-wise gain $\gamma$ and bias $\beta$. For token vector $z$ of width $D$, one stabilized convention is:

The small positive $\epsilon$ avoids division by zero. This normalizes a feature vector; it is separate from softmax normalizing weights across key positions. For example, $z=[1,3]$ has mean 2 and variance 1 before stabilization.

S5 · §3, equation (3); §5, equation (5) CL-015

Mix positions, then transform features

Attention combines information across positions. The original feed-forward network then applies a two-layer transformation independently to each position, using the same parameters across positions within that layer:

The first linear map changes feature width; the elementwise maximum is the ReLU nonlinearity; the second map returns to the model width. If $u$ has width $d_{\mathrm{model}}$, $W_1$ is $d_{\mathrm{model}}\times d_{\mathrm{ff}}$ and $W_2$ is $d_{\mathrm{ff}}\times d_{\mathrm{model}}$. Different layers may have different parameters. This per-position processing does not introduce another direct token-to-token mixing step.

The residual paths, output projection, and feature transformations mean the single-head convex-mixture property does not describe the final Transformer state. Even the attention output is combined with information that bypassed it.

S1 · §3.3, equation (2) CL-014, CL-023, CL-024

Locate the prediction boundary

The original decoder differs from the encoder diagram above. It uses causal self-attention over decoder states, adds cross-attention to the encoder output, and has its own feed-forward sublayer. In cross-attention, queries come from the decoder; keys and values come from the encoder output. The two sequence lengths can differ, which is why we kept $n$ and $m$ separate in the shape rule.

Training targets are shifted: the state at one decoder input position is used to predict the next target token. Including the current input token in a causal mask does not reveal that future label. At the top of the decoder, a learned linear map and vocabulary softmax turn states into next-token probabilities. The attention softmax inside a head is a different distribution.

Learning would require a loss and parameter updates through this computation. Our lab performs neither. It stops at one attention output so that the mechanism remains hand-checkable.

S1 · §§3.1–3.4 CL-002, CL-007, CL-013, CL-018, CL-021

Keep the claim as small as the evidence

In the default lab, the “rises” value receives about 0.576 of the mixture weight. That is an exact statement about one head, one row, and these invented states. It does not mean “57.6% of the model’s reasoning came from rises.”

The value intervention gives a narrower, demonstrable result: identical weights can carry different content and produce different outputs. A complete Transformer adds heads, projections, bypass paths, normalization, and nonlinear transformations. A heatmap omits these.

There is also empirical and methodological debate. Jain and Wallace found different attention distributions with similar predictions in the NLP models they tested. Wiegreffe and Pinter challenged the scope and definition of explanation and argued for task-dependent tests. Those studies largely concern recurrent attention classifiers, so they are not a universal experimental verdict on every Transformer.

Our inference is deliberately bounded: attention weights alone do not establish a faithful explanation of a model’s final prediction. They can still be informative measurements of an internal computation. A stronger explanatory claim needs a definition, a target prediction, and evidence from suitable interventions or other checks.

S2 · §4.2; pp. 3548–3550 CL-017

S3 · §§3.4–6; pp. 15–19 CL-017

Try a case the controls do not answer

A new attention row has query $q=[0,0]$. Only the first and third keys are allowed. The keys can be any finite two-coordinate vectors, and the values are:

What are the three weights and the output? Would replacing the middle value by $[100,-100]$ change it? What if every key were blocked?

New synthetic transfer and derivation · CL-019, CL-020. It uses an unfamiliar mask and negative values, not the lab’s default case.

One more change: reorder the tokens

Remove positional features and use all-visible self-attention. Swap the first and third rows of $X$, keeping the same projection parameters. Should every output stay at its old row index?

You can now reconstruct the mechanism from a shape check, a visibility rule, and a weighted sum—and state exactly where that explanation stops. The notes keep the symbols, source locators, and runnable lab together for a later reread.

Derived transfer · CL-011.