Distillation
A six-chapter field guide
Glossary
Contents

Reading companion

Glossary & sources

Definitions, source status, assumptions, and any optional marked additions behind the field guide.

A small vocabulary

Policy
A conditional distribution over the next token in this guide.
Trajectory
A generated response sequence; its prefixes provide the states evaluated during tokenwise distillation.
Prefix
The prompt plus response tokens generated before the current position.
On-policy
Here, the response occurrence was sampled from the student behavior policy used for collection.
Behavior policy
The policy that generated the collected response; it may differ from the current trainable parameters later.
Teacher
The model supplying supervision at the selected prefix; teacher probability is not automatically correctness.
Forward KL
D_KL(teacher || student) at one held-fixed prefix in this notation.
Reverse KL
D_KL(student || teacher) at one held-fixed prefix in this notation.
Nat
The information unit produced by natural logarithms.
Stop-gradient
An estimator choice that treats a dependency as fixed during differentiation; here it applies to the sampled trajectory.

Primary sources and status

  1. SRC-GKD-ICLR-2024 · Peer-reviewed · ICLR 2024 official record

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

    Canonical publication anchor for GKD. It does not turn this toy into model evidence.

  2. SRC-GKD-ARXIV · Author manuscript v3 · 2024-01-17 · accepted at ICLR 2024

    Agarwal et al. author manuscript, arXiv:2306.13649v3

    Navigable equations, Algorithm 1, source mixture, divergence choices, and stop-gradient statement.

  3. SRC-MINILLM-ICLR-2024 · Peer-reviewed · ICLR 2024 official record

    MiniLLM: Knowledge Distillation of Large Language Models

    Sequence-distribution reverse KL and its policy-gradient decomposition; distinct from the canonical local toy.

  4. SRC-OPD-RETHINKING-2026 · Recent preprint evidence · arXiv 2026 · not peer reviewed

    Rethinking On-Policy Distillation of Large Language Models

    Preprint evidence used only to mark current feedback-granularity questions; no empirical conclusion is imported into the guide.

  5. SRC-SEQKD-EMNLP-2016 · Peer-reviewed · EMNLP 2016

    Sequence-Level Knowledge Distillation

    Fixed teacher beam outputs used as hard sequence targets.

  6. SRC-IMITKD-EMNLP-2020 · Peer-reviewed · EMNLP 2020

    Autoregressive Knowledge Distillation through Imitation Learning

    Sequence-source mixture, oracle supervision, schedule, and sample-pooling boundaries.

What this guide does not establish

The probabilities in OPD-TOY-001 are invented teaching inputs. They are not a trained language model, a benchmark, or evidence that one method wins. No learner study or controlled learning outcome is claimed. Cross-tokenizer alignment, EOS inclusion, production length weighting, behavior-policy lag, and teacher correctness remain implementation questions beyond the fixture.

← Return to contents