Change one thing

Keep the parameters fixed. Intervene on a query, a visibility rule, or a carried value.

A zero query still receives something

With a zero query, every finite key gets score zero. Equal scores produce equal weights across the allowed positions. That need not make the weighted values cancel. Try the query multiplier at zero, then explain the two output coordinates.

This is a direct intervention on the selected $q$ after projection. A real change to the input could affect queries, keys, and values together. Here they are separated deliberately. The next control similarly replaces one value coordinate after its projection. Neither slider learns a parameter.

EXP-001 · ATTENTION LAB

Change one part of the mixture.

Synthetic single head · fixed projection matrices · direct state interventions · no training

Predict first. If the last query becomes zero, must the output become zero too? Then change only the third value: which quantities should stay exactly the same?

Current queryrises q = [1, 1]

May readgrid, load, rises

One query row · scaled scores divide the dot product by √2
Key token	Dot product	Scaled score	Visibility	Weight · 0 to 1	Value	Contribution
gridkey 0	2.000	1.414	Allowed	0.284	[1, 0]	[0.284, 0.000]
loadkey 1	1.000	0.707	Allowed	0.140	[0, 2]	[0.000, 0.280]
riseskey 2	3.000	2.121	Allowed	0.576	[1, 2]	[0.576, 1.152]

Weighted output
sum of the three contributions[0.860, 1.432]

This is the final token: causal and all-visible modes permit the same three positions. Choose an earlier query to see a visibility change.

Blocked scores remain visible for comparison; their logits are excluded before normalization. Display rounds to three decimals, so displayed weights may not add to exactly 1.

The slider changes q or a value coordinate after projection. It does not edit X, keys, or the fixed projection matrices. All UI masks allow at least one key. This experiment computes attention only; residual paths, normalization, feed-forward layers, and positional information are discussed separately.

Five states to read side by side

These computed comparisons are always available. Compare the first and last rows: identical weights carry different values and produce different outputs.

Derived from the same fixed model · vector order is grid, load, rises
State	Query	Weights	Third value	Output
Baselinerises · gain 1 · causal	[1, 1]	[0.284, 0.140, 0.576]	[1, 2]	[0.860, 1.432]
Zero queryrises · gain 0 · causal	[0, 0]	[0.333, 0.333, 0.333]	[1, 2]	[0.667, 1.333]
First query, causalgrid · gain 1 · causal	[1, 0]	[1.000, 0.000, 0.000]	[1, 2]	[1.000, 0.000]
First query, all visiblegrid · gain 1 · all visible	[1, 0]	[0.401, 0.198, 0.401]	[1, 2]	[0.802, 1.198]
Third value changedrises · gain 1 · causal; V₃ y = 4	[1, 1]	[0.284, 0.140, 0.576]	[1, 4]	[0.860, 2.584]

Three small experiments

1. Change the query. Keep “rises” selected and set its multiplier to zero. Every weight is $1/3$ ; the output is $[2/3,4/3]$ . Increase the multiplier again. The largest fixed score gets a larger share, while tied scores remain tied. This is concentration, not a switch to hard selection.

2. Change what is visible. Select “grid.” With causal visibility, only its own value is allowed, so the output is exactly $[1,0]$ . Disable causal visibility and later values can contribute. Return to “rises”: switching the mask has no effect because every position is already at or before the final query.

3. Change the carried content. Reset, then move the third value’s second coordinate from 2 to 4. The scores and weights stay identical. The second output coordinate increases by $2a_3\approx1.152$ . The first output coordinate stays fixed. If that third key is blocked, the override has no effect on the output.

\Delta o_2=a_3\,\Delta v_{3,2}

The attention map describes the mixing coefficients. The map alone does not tell you the resulting vector. That distinction will matter when we interpret a complete model.

Toy interventions and their consequences · CL-008, CL-009, CL-018. LAB-001 is the canonical calculation behind the live view and static comparisons.

Run the same calculation locally

The companion is a dependency-free JavaScript module. From this guide’s site folder, run node model.mjs to print the baseline and contrasts. To inspect one state in your own module, import the same pure function used by this page:

import { compute } from './model.mjs';
const before = compute();
const after = compute({ valueY: 4 });
console.log(before.weights, after.weights);
console.log(before.output, after.output);

The first pair matches; the second pair differs only in its second coordinate. Complete source and checks are linked in the notes. This run demonstrates the stipulated algebra, not model training or empirical prediction quality.