A graph gives context

A missing row is still attached to its neighbors. That is the opportunity—and the constraint.

Start with six rows

Imagine six sites connected by a small network. You know two feature values at each site, but deliberately cover the row at B. What could a model use to reconstruct it? Nearby sites offer one source of context. The connections say which sites are nearby in the graph.

We use a power-grid-shaped synthetic graph. Its two feature channels, $u$ and $v$ , are dimensionless invented numbers. They are not voltages, power injections, sensor readings, or simulated grid states. The shape supplies a memorable example; it supplies no evidence about electrical behavior.

FIG-001Six rows, one connected context.

Original feature rows (u, v)

A: (2, 1)
B: (1, 2)
C: (2, 2)
D: (3, 1)
E: (1, 3)
F: (2, 3)

B connects to A, C and D. Links are undirected; position and line length carry no physical units.

A feature vector is an ordered list. At B it is $x_B=(1,2)$ ; swapping the entries changes its meaning. Stack the six vectors as rows to get $X\in\mathbb{R}^{6\times2}$ . The graph $G=(V,E)$ adds a node set $V$ and an edge set $E$ . An edge joins two rows; it is not another feature value.

CL-012 · All six feature rows and graph connections are teaching assumptions.

One message-passing step

B has three neighbors: A, C and D. A message-passing layer collects their vectors, combines them, and transforms the result using shared learned weights. “Shared” means the same rule is applied at different nodes; it does not mean every node receives the same result.

Let $\mathcal{N}(i)$ be the neighbors of $i$ , $h_i^{(0)}=x_i$ its starting vector, and $W_{\rm self},W_{\rm nbr}$ learned matrices. One illustrative layer is:

h_i^{(1)}=\sigma\!\left(W_{\rm self}h_i^{(0)}+W_{\rm nbr}\frac{1}{|\mathcal{N}(i)|}\sum_{j\in\mathcal{N}(i)}h_j^{(0)}\right)

The sum collects neighbors, division makes a mean, and $\sigma$ is a nonlinear transformation. With two input features and $q$ output features, both matrices have shape $q\times2$ , treating each vector as a column in this equation.

Before the transformation, B's original neighbor mean is $((2,1)+(2,2)+(3,1))/3=(7/3,4/3)$ . That is not the target $(1,2)$ . Graph context offers information to learn from; adjacency alone does not specify a perfect reconstruction rule.

This mean layer is a refresher, not a reproduction of GraphMAE's architecture. Graph attention can instead learn neighbor-dependent weights. The authors' main GraphMAE recipe uses graph attention, while studying other architecture choices.

CL-002 · Gilmer et al., message-passing formulation, §2

CL-002 · Veličković et al., Graph Attention Networks, §2

Targets without manual labels

A training objective compares a prediction with a target. Self-supervision changes where that target comes from: here, the original feature row supplies it. We hide a row from the predictor while retaining its original values to score the prediction.

GraphMAE, introduced by Hou and colleagues in 2022, centers its original objective on masked node-feature reconstruction. Its graph supplies context, and its encoder learns a representation through this pretext task. This is a particular graph method, not a new name for every masked autoencoder.

CL-001 · GraphMAE, original paper, §3

The next question is more precise than “what is masked?” We need to follow the original row, the input token, and the hidden representation separately.