Distillation
A six-chapter field guide
Glossary
Contents

Reading companion

Glossary & sources

Definitions, source status, assumptions, and any optional marked additions behind the field guide.

A small vocabulary

Policy
A conditional distribution over the next token in this guide.
Trajectory
A generated response sequence; its prefixes provide the states evaluated during tokenwise distillation.
Prefix
The prompt plus response tokens generated before the current position.
On-policy
Here, the response occurrence was sampled from the student behavior policy used for collection.
Behavior policy
The policy that generated the collected response; it may differ from the current trainable parameters later.
Teacher
The model supplying supervision at the selected prefix; teacher probability is not automatically correctness.
Forward KL
D_KL(teacher || student) at one held-fixed prefix in this notation.
Reverse KL
D_KL(student || teacher) at one held-fixed prefix in this notation.
Nat
The information unit produced by natural logarithms.
Stop-gradient
An estimator choice that treats a dependency as fixed during differentiation; here it applies to the sampled trajectory.

Primary sources and status

  1. SRC-GKD-ICLR-2024 · Peer-reviewed · ICLR 2024 official record

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

    Canonical publication anchor for GKD. It does not turn this toy into model evidence.

  2. SRC-GKD-ARXIV · Author manuscript v3 · 2024-01-17 · accepted at ICLR 2024

    Agarwal et al. author manuscript, arXiv:2306.13649v3

    Navigable equations, Algorithm 1, source mixture, divergence choices, and stop-gradient statement.

  3. SRC-MINILLM-ICLR-2024 · Peer-reviewed · ICLR 2024 official record

    MiniLLM: Knowledge Distillation of Large Language Models

    Sequence-distribution reverse KL and its policy-gradient decomposition; distinct from the canonical local toy.

  4. SRC-OPD-RETHINKING-2026 · Recent preprint evidence · arXiv 2026 · not peer reviewed

    Rethinking On-Policy Distillation of Large Language Models

    Preprint evidence used only to mark current feedback-granularity questions; no empirical conclusion is imported into the guide.

  5. SRC-SEQKD-EMNLP-2016 · Peer-reviewed · EMNLP 2016

    Sequence-Level Knowledge Distillation

    Fixed teacher beam outputs used as hard sequence targets.

  6. SRC-IMITKD-EMNLP-2020 · Peer-reviewed · EMNLP 2020

    Autoregressive Knowledge Distillation through Imitation Learning

    Sequence-source mixture, oracle supervision, schedule, and sample-pooling boundaries.

What this guide does not establish

The probabilities in OPD-TOY-001 are invented teaching inputs. They are not a trained language model, a benchmark, or evidence that one method wins. No learner study or controlled learning outcome is claimed. Cross-tokenizer alignment, EOS inclusion, production length weighting, behavior-policy lag, and teacher correctness remain implementation questions beyond the fixture.

Variant 04 · passive media notes

Three local chapter replays

The clips are supplemental renderings of facts already established in the canonical prose. They contain no audio stream, use local English WebVTT captions, and retain six distinct still frames plus transcripts when motion or media is unavailable.

  • V04-VID-001 · 15 seconds · A draw becomes a visited prefix. MP4 · WebVTT
  • V04-VID-002 · 16 seconds · The teacher follows the visited prefix. MP4 · WebVTT
  • V04-VID-003 · 20 seconds · A loss is not yet an update. MP4 · WebVTT

Rendered locally with Remotion 4.0.506 at 1920 by 1080, 30 fps. The variant interaction gate fails: transport controls do not manipulate the toy model.

← Return to contents