A loss that compares

Similarity becomes useful to this objective only in relation to the other candidates.

Read the geometry

The following vectors are hand-positioned toy representations, not encodings of our arrow. Fix the anchor at $a=(1,0)$ . Place the positive at angle $\alpha$ on a unit circle, negative A at $(0,1)$ , and negative B at $(-1,0)$ . Every vector has length one.

Cosine similarity measures alignment. For unit vectors it is simply their dot product, so the anchor reads each candidate's horizontal coordinate. The positive score is $s_+=\cos\alpha$ ; the two negative scores are $s_A=0$ and $s_B=-1$ . Moving the positive toward the anchor increases its score. This uses SimCLR's cosine-similarity convention.

Turn three scores into one loss

Divide each score by a positive temperature $\tau$ , exponentiate, then normalize the three weights so they sum to one. This normalization is called softmax. Let $Z$ be the sum of those weights:

\begin{aligned}Z&=e^{s_+/\tau}+e^{s_A/\tau}+e^{s_B/\tau}\\p_+&=\frac{e^{s_+/\tau}}{Z}\\\ell&=-\ln p_+\end{aligned}

The positive is included in the denominator; the anchor is not. Our single-anchor InfoNCE-style loss is the negative natural logarithm of the positive's probability, measured in nats. A larger $p_+$ means a smaller loss. This is a teaching subset, not the complete SimCLR minibatch objective. See CPC's candidate-classification formulation and SimCLR Eq. 1.

At $\alpha=60^\circ$ and $\tau=0.5$ , the scores are $(0.5,0,-1)$ . Their scaled values are $(1,0,-2)$ . The resulting positive probability is about 0.705385 and the loss is 0.349012 nats. These are deterministic calculations from the stated toy values.

Try a sharper comparison

Predict first: at $\alpha=120^\circ$ , would reducing temperature from 1 to 0.1 lower the positive's loss? Set that angle, change only temperature, and watch the probability bars and loss. The vectors and their raw similarities should stay still.

EXP-001 · A CONTRASTIVE LOSS LAB

Geometry becomes a loss.

Toy: hand-positioned unit vectors, not representations learned from the arrow. Three candidates, one anchor, no training.

Before you touch the controls

At an angle of 120°, Negative A scores higher than the positive. Predict: will reducing temperature from 1.0 to 0.1 lower the positive’s loss?

Try those two settings. Then read the explanation below the lab.

Positive angle α 60°Moves the positive, changing its similarity. Temperature τ 0.5Reweights scores; does not move vectors.

SHARE AMONG THREE CANDIDATES · 0 TO 1

● Positive0.705

□ Negative A0.259

◇ Negative B0.035

SINGLE-ANCHOR LOSS0.349 nats

● Positive ▷ Anchor □ Negative A ◇ Negative B
Axes x and y are vector coordinates. All vectors have length 1. The positive’s horizontal projection is its similarity to the anchor.

At 60° and temperature 0.5, the positive receives 0.705 of the candidate probability. Loss is 0.349 nats. The positive has the highest similarity.

Same candidates, three calculation stages
Candidate	Similarity s	Scaled s / τ	Probability
Positive	0.500	1.000	0.705
Negative A	0.000	0.000	0.259
Negative B	-1.000	-2.000	0.035

The positive is in the denominator; the anchor is not. These probabilities are relative to this candidate set, not confidence in an image category. Values round to three decimals.

Read the same experiment without sliders

Change one factor at a time
Angle	Temperature	Positive p	Loss, nats
60°	0.5	0.705	0.349
0°	0.5	0.867	0.143
120°	1.0	0.307	1.180
120°	0.1	0.007	5.007

60° → 0° at τ = 0.5: closer positive, lower loss. At 120°, τ = 1.0 → 0.1: the higher-scoring negative takes nearly all the probability, so positive loss increases.

Separate hypothetical: if every representation were identical, all scores would tie. Then positive probability is 1/3 and loss is ln(3) ≈ 1.099 at any positive temperature.

Lower temperature sharpens score differences, favoring the highest-scoring candidate. At 120°, negative A scores 0 while the positive scores −0.5. In this specific comparison, sharpening raises the positive's loss from about 1.180270 to 5.006760 nats. Lower temperature does not universally lower positive loss; the effect depends on the scores and temperature.

These probabilities are relative weights among these three candidates—not calibrated confidence that an image belongs to a semantic category. Nor does moving a slider train a model: angle directly changes the prescribed geometry, while temperature changes only the score-to-probability calculation.

What if every score ties?

Transfer: all three candidate scores are 0. What can temperature change?

Nothing about their relative weights. Each exponential is $e^{0/\tau}=1$ , so $p_+=1/3$ and $\ell=\ln3\approx1.098612$ for every positive temperature. Any three equal scores give this result. Identical representations are one way to produce equal scores; tying just two points in the plot is not full collapse.

This arithmetic explains what the objective scores, not a guarantee about where training will converge. Even a low loss leaves another question unanswered: what information will the representation retain for a later task?