When mixing stops helping

More context is useful only if the representation keeps the distinctions the task needs.

Repeat this exact average

Return to the original connected graph, all gates 1 and $a=b=1/2$ . Each new state is a convex average: nonnegative coefficients sum to one. It stays between the previous smallest and largest values. The overall range therefore cannot increase under this exact rule.

Keep applying the same operator. The node values approach a common value. The bars below show a deterministic computation from the original input; they are not a training curve and report no prediction accuracy.

FIG-MIXRepeated fixed averaging erases the original scalar distinctions.

DETERMINISTIC TOY OUTPUT · bars share a 0–3 scale

Layer 0

2.000

0.000

1.000

3.000

0.000

Layer 1

1.250

0.750

1.333

1.750

1.500

Layer 4

1.173

1.165

1.267

1.479

1.563

Layer 20

1.294

1.298

1.308

1.313

Layer 100

1.300

All gates 1, a=b=½, original connected graph. The limiting value is 1.300. Rounded values may look equal before exact convergence.

Why 1.3? This neighbor-mean rule preserves a degree-weighted average. A and B each have degree 2, C degree 3, D degree 2, E degree 1. Thus the limit is:

\frac{\sum_i d_i x_i}{\sum_i d_i}=\frac{4+0+3+6+0}{10}=\frac{13}{10}

The ordinary mean is 6/5, a different number. The denominator used in the update determines which weighted quantity is preserved. On this finite connected undirected graph, the self term makes the averaging walk lazy, avoiding an alternating two-cycle. Repeated application converges to its unique degree-weighted consensus.

Why degree weighting is preserved

Multiply each node update by its degree and sum over nodes. The self term contributes half of the previous degree-weighted sum. In the neighbor term, each node's old value appears once for each of its incident edges, so it contributes the other half. The total stays fixed. Connectivity and positive self retention make all nodes approach one value, which must equal that preserved sum divided by total degree.

CL-013 · Derived for this fixed, connected, undirected, unit-gate toy. Removing C–D gives separate limits 1 on ABC and 1.5 on DE. Halving a gate while retaining the old degree breaks the convex-average premise.

What this says about oversmoothing

Initially, A=2 and E=0 were easy to distinguish by their state. At consensus they both carry 1.3. A downstream head that sees only this scalar has lost the distinction. This illustrates a mechanism behind oversmoothing: repeated neighborhood mixing can make node representations less discriminative.

The scope matters. Li, Han and Wu analyze smoothing in graph convolutional networks; Oono and Suzuki give conditions under which particular GNNs approach a restricted representation subspace. These analyses do not establish that every deep GNN makes every vector identical or that an extra layer always reduces task accuracy.

CL-006 · Li, Han & Wu: “When GCNs Fail,” Figure 2 and Theorem 1

CL-006 · Oono & Suzuki, §3.3 and §4: conditions on expressive-power loss

Nor is our update the exact Kipf–Welling GCN layer. That layer uses added self-connections, symmetric degree normalization, a trainable matrix and a nonlinearity. Our neighbor mean and separate half-self term are deliberately simpler. Do not transfer the number 1.3, or our scalar consensus shape, to another normalization.

CL-005 · Kipf & Welling, §2, equation (2)

A second limitation is already visible at one layer: a mean of messages [0,2] equals a mean of [1,1]. This toy cannot recover which pair produced that aggregate if all other inputs agree. Aggregation is a modeling decision about which distinctions to retain.

Derived counterexample from CL-008: equal means do not imply equal neighbor multisets. This is a limitation of the chosen mean, not every possible GNN.

Try a graph that is not a grid

A document relation graph has five text regions P, Q, R, S, T. Its undirected edges are PQ, QR, QS and ST. Imagine the same positive self-plus-neighbor rule and unit gates. Increase only R's initial scalar feature by 2.

Which regions must change after two synchronous rounds? Can T change yet? If Q–S is removed, can R's change ever reach T?

You do not need an OCR model to answer. Use the relation graph and the computation, not the apparent closeness or meaning of the text regions.

Reveal the worked reasoning

From R, the distances are R=0, Q=1, P=2, S=2 and T=3. Positive coefficients and a positive single-source change make R, Q, P and S change after two rounds. T is unchanged until round three. Removing Q–S separates P/Q/R from S/T, so R's change never reaches T through these local layers.

If the message coefficient were zero, paths would still exist but neighbor changes would have no effect. This is why the general claim is “can depend on”; the exact changed set needs the positive toy assumptions.

Back at the grid, the same discipline applies. Choose what nodes and edges mean, define message and update functions, locate any learned parameters, and check the output against the intended task. A graph-shaped model supplies a pattern of information access. Physical validity comes only from additional modeling and evidence that this toy does not provide.

CL-014 · No physical simulation or real-grid recommendation. The transfer answer is derived from CL-004/010's local-dependency argument.