Direction is a choice

Scaled cosine error rewards aligned features. Squared error also sees their magnitude.

One row, by hand

Take B's target, $x_B=(1,2)$ , and prescribe $\widehat x_B=(2,1)$ . This is a chosen vector, not a trained decoder's output. Their dot product is $1\cdot2+2\cdot1=4$ . Each vector's Euclidean norm, or length, is $\sqrt{5}$ .

c_i=\frac{x_i^\top\widehat x_i}{\|x_i\|_2\,\|\widehat x_i\|_2}

Cosine similarity $c_i$ measures alignment. For nonzero vectors it lies between −1 and 1: opposite directions at −1, a right angle at 0, the same direction at 1. Here $c_B=4/5=0.8$ . The angular residual $1-c_B$ is 0.2.

GraphMAE's scaled cosine error raises the residual to an exponent $\gamma\ge1$ , then averages over selected nodes:

L_{\rm SCE}=\frac{1}{|M|}\sum_{i\in M}(1-c_i)^\gamma

With only B selected and $\gamma=2$ , the score is $(1-0.8)^2=0.04$ . The exponent acts before the mean. “Scaled” here refers to this power, not an extra fitted multiplier.

CL-006 · Author SCE implementation: normalize, dot, residual power, mean

Inspect the objective

Predict before changing the controls: if you double the prescribed prediction from $(2,1)$ to $(4,2)$ , does SCE change? Does squared error change? Start with one variable; the mask and exponent can stay fixed.

FIG-004 · LAB-001

Which errors count?

Synthetic features. Prescribed reconstructions. No encoder, decoder, or weights are trained here. Changing the mask changes input visibility and scored rows; it does not train new predictions.

Masked nodesPrescribed prediction scaleSCE exponent gamma

SELECTED-NODE MEANS

SCE0.04

MSE1

Dashed node = hidden input row. Edges remain. All values are dimensionless; MSE first averages the two feature errors.

Masked B · scale 1 · gamma 2. SCE 0.04; MSE 1. Mean over 1 selected node only.

B · scored
Encoder input
[MASK]
Withheld target
(1, 2)
Prescribed prediction
(2, 1)
Cosine / residual
0.8 / 0.2
SCE term / MSE term
0.04 / 1
Visible input neighbors
A, C, D

Unscored nodes: A, C, D, E, F. They may supply context, but have no direct term in this mean.

Deterministic comparison · gamma 2, fixed prescribed directions
Case / selected	SCE	MSE
BaselineB · scale 1	0.04	1
Doubled predictionB · scale 2	0.04	4.5
Two selectedB, D · scale 1	0.020051	0.75
Three selectedB, C, D · scale 1	0.013367	0.5

Values rounded to six decimals; computation uses full precision. Scale changes prescribed prediction magnitude only. The complete reading edition retains these comparisons.

For comparison, this lab uses per-feature mean squared error, then averages over selected nodes. The feature dimension is $d=2$ :

L_{\rm MSE}=\frac{1}{|M|}\sum_{i\in M}\frac{1}{d}\sum_{k=1}^{d}(\widehat x_{ik}-x_{ik})^2

At B the baseline is $((2-1)^2+(1-2)^2)/2=1$ . Doubling gives $((4-1)^2+(2-2)^2)/2=4.5$ . SCE remains 0.04: positive rescaling cancels between the cosine numerator and denominator.

Now select B, C and D. The average can fall even though more features are withheld. C's prescribed direction is exactly right and D's nearly right; their small terms lower the mean. The predictions were not learned under the new mask. A lower number here does not imply easier training or a better representation.

CL-013 · Deterministic derivations from CL-012's prescribed vectors. Browser, static table and runnable lab share one numerical module.

What the exponent changes

Increasing $\gamma$ from 1 to 2 changes B's term from 0.2 to 0.04. For a residual $r=1-c$ between zero and one, higher powers make its numerical contribution smaller. Larger residuals become more prominent relative to smaller ones.

That is not a promise that every individual loss decreases. If $c=-0.5$ , then $r=1.5$ : squaring raises the term from 1.5 to 2.25. Changing the loss scale changes optimization behavior; a smaller displayed value does not itself mean better training.

Boundary: what if a feature vector is zero?

The mathematical cosine divides by both vector lengths and is undefined for a zero vector. Our teaching functions reject that case. The author's code uses PyTorch's epsilon-protected F.normalize, so zero inputs have a defined numerical path: a zero normalized row gives dot product zero and residual one. Extremely small vectors also depend on epsilon. These are different domain policies; the lab is not a bit-for-bit PyTorch replacement.

CL-007 · PyTorch normalize, denominator max(norm, epsilon)

A perfect angle is not a perfect row

There is a stronger counterexample: let $\widehat x_B=2x_B=(2,4)$ . The direction is exact, so SCE is zero for every positive exponent. Yet the values are wrong. Per-feature squared error is $((2-1)^2+(4-2)^2)/2=2.5$ .

This is a property of the objective, not a defect discovered in a trained model. If absolute magnitude matters downstream, this distinction belongs in your experimental design. Evaluate magnitude recovery directly or compare a suitable magnitude-sensitive objective. Whether doing so improves performance remains an empirical question.