SELF-SUPERVISED LEARNING Learning from Where does the target come from when nobody labels the example? Hide a piece. Make two views. Learn what the objective asks a representation to keep.
Start reading → The missing piece becomes the target An invented arrow with its top-right region masked. The sixteen original values are shown separately as a target, not supplied to a trained model. THE INPUT THE TARGET One example. Two different roles. Withhold part of an invented arrow. Its original pixels become a target—but not an input to the predictor.
Five short chapters, from the source of a target to the information a representation keeps. Basic ML and arithmetic are enough; vectors, cosine similarity and contrastive loss are introduced along the way.
01 Where targets come from A learning signal does not have to arrive as a human-written category.
↗ 02 Predicting the missing part The original data can supply the answer without giving that answer to the predictor.
↗ 03 Learning from two views Common origin supplies a relationship. The pixels do not have to be identical.
↗ 04 A loss that compares Similarity becomes useful to this objective only in relation to the other candidates.
↗ 05 What the representation keeps A useful invariance for one task can erase the answer to another.
↗