↗SSL
Chapter 3 of 5
Notes & sources
Contents ≡
Contents / Chapter 3

03

Learning from two views

Common origin supplies a relationship. The pixels do not have to be identical.

Change the view, keep the source

Make two intensity variants of our arrow. Their pixels differ, but both were constructed from the same source image. We call this known relationship a positive pair. The pairing requires a record of common origin, not the semantic label “up.” Our intensity-only change is an illustrative augmentation, not SimCLR's complete transformation recipe.

In SimCLR's architecture, the two views pass through the same encoder. “Same” means shared parameters: changing an encoder weight changes the function used on both branches. A small network called a projection head then maps the encoder's features into the space used by the contrastive objective.

h=fθ(v)z=gϕ(h)\begin{aligned}h&=f_\theta(v)\\z&=g_\phi(h)\end{aligned}

Here vv is a view, fθf_\theta the encoder with parameters θ\theta, and gϕg_\phi the projector with parameters ϕ\phi. The encoder representation hh and projected representation zz are different objects. SimCLR applies its contrastive loss to zz and retains the encoder representation hh for downstream use.

FIG-003Different views, shared parameters.
Two views share an encoder and a projection headTwo intensity views of one source pass through the same encoder f theta, producing h, then the same projection head g phi, producing z for the contrastive objective.SOURCEView 1View 2fθfθsharedENCODERh₁h₂gφgφsharedPROJECTORz₁z₂Two views share an encoder and a projection headThe two paths use identical encoder parameters and identical projector parameters. Encoder representations h and projected representations z have different roles.ONE SOURCEView 1View 2fθfθsharedh₁h₂gφgφz₁z₂

A SimCLR-style schematic: fθ is the shared encoder, gφ the shared projection head. The objective compares projected z; the encoder’s h is retained for downstream use. Intensity-only views are illustrative. No encoder runs on this page.

Positive is a construction rule

Take one view as an anchor. Its paired view is the positive candidate. In the contrastive setup we will illustrate, views from other source images supply negative candidates. The objective compares the positive's similarity to the anchor with the competing similarities; matching the positive is not judged in isolation.

A negative here does not mean “known to belong to a different human class.” Two different images could both show upward arrows. Treating them as negatives can create a semantic false negative: the construction rule separates sources that a later task might want to group. Source identity and semantic category are not interchangeable.

Matching is not enough

Suppose every input receives exactly the same representation. Positive pairs match perfectly, but so does everything else. This is representation collapse: that representation preserves no distinction between inputs. The next chapter's three-candidate loss makes the problem concrete—equal scores give the positive only one third of the probability, not a perfect score.

Does every self-supervised method need negatives?

No. BYOL uses an online predictor, a moving-average target network and a stop-gradient target, without negative pairs in its objective. It is a different complete design. Its existence does not show that simply deleting negatives from this contrastive loss will produce useful learning, or that one component guarantees freedom from collapse.

Let's now isolate candidate competition. We will prescribe four tiny vectors directly, leaving the encoder out of the calculation so that every score is visible.