READING COMPANION

Notation, evidence, and the edges of the model.

The proof is only as clear as its fixed quantities. Keep this ledger beside the derivation.

Epistemic key

source-backed externally checkable statement with a primary locator

derived follows from displayed premises and arithmetic

toy invented input used only to expose a mechanism

boundary assumption or implementation choice that limits the result

unresolved not established by inspected evidence

Compact glossary

Policy: A conditional distribution over the next token in this article; it does not by itself imply scalar reward optimization.
Trajectory: A generated response sequence. Its prefixes are the states at which local token distributions can be compared.
On-policy: For canonical GKD here, the response was sampled from the current student behavior policy at collection time. It describes provenance, not KL direction.
Behavior policy: The policy that generated collected data. $q_{\bar\theta}$ exposes that it is held fixed during the canonical local update.
Forward KL: $D_{KL}(p_{teacher}\|q_{student})$ at a fixed state in this notation.
Reverse KL: $D_{KL}(q_{student}\|p_{teacher})$ at a fixed state in this notation.
Logit: An unnormalized student score. Softmax maps the length- $M$ logit vector to a probability vector.
Nat: The unit of information produced by a natural-log ratio. KL is dimensionless, but “nats” records the logarithm base.
Stop-gradient: An estimator choice that holds one dependency fixed during differentiation. Here it applies to the sampled trajectory, not the local student logits.
Feedback granularity: Which teacher information is exposed: a full vocabulary distribution, top-k truncation, one sampled token's score, a hard action, or a scalar reward.

Primary sources and status

Inspected 5 September 2026. Citations support the nearby bounded claims; further-reading links are not used as generic evidence. Empirical results from these sources are intentionally not transferred into the toy.

SRC-GKD-ICLR-2024 · Peer-reviewed · ICLR 2024 official recordOn-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Canonical publication-status anchor. Supports the GKD framing, not this toy or a universal superiority claim.
SRC-GKD-ARXIV-V3 · Author manuscript v3 · 2024-01-17 · accepted at ICLR 2024Agarwal et al. author manuscript, arXiv:2306.13649v3
Navigable source for Eqs. 2–5, Algorithm 1, lambda cases, divergence choices, temperature, and the explicit stop-gradient statement.
SRC-SEQKD-EMNLP-2016 · Peer-reviewed · EMNLP 2016Sequence-Level Knowledge Distillation
Primary source for fixed teacher beam outputs used as hard sequence targets.
SRC-IMITKD-EMNLP-2020 · Peer-reviewed · EMNLP 2020Autoregressive Knowledge Distillation through Imitation Learning
Primary source for ImitKD sequence-level mixtures, oracle supervision, schedule, and practical sample pooling.
SRC-MINILLM-ICLR-2024 · Peer-reviewed · ICLR 2024 official recordMiniLLM: Knowledge Distillation of Large Language Models
Conference anchor for sequence-level reverse KL and its policy-gradient decomposition. The later arXiv v6 title changed in 2026; equations here remain pinned to the conference version.
SRC-TML-OPD-2025 · Author engineering report · Thinking Machines Lab · 2025-10-27On-Policy Distillation
Primary author report for the sampled-token reverse-log-ratio recipe and explicit statement that its experiments did not use top-k logit distillation. Not peer-reviewed.
SRC-TINKER-TOPK · Official engineering documentation · version-sensitiveTinker Cross-Entropy documentation
Primary documentation for renormalized teacher top-k soft targets. Not canonical GKD evidence.

Contract and independent artifacts

Shared contract: Verbatim build copy (local copy) · SHA-256 9AAD0A2560BFF8696A65B49F5AA72058A75C102AA2C9E17535C40471FFD0E697
Derivation: Markdown source sheet (local copy)
Numerics: Generated JSON receipt (local copy) · Python source (local copy) · Python tests (local copy)
Claim ledger: Evidence ledger (local copy)

Residual limits

Aligned [A,B] vocabulary; no cross-tokenizer mapping.
Two evaluated positions; EOS inclusion and variable-length weighting are not generalized.
Sampling temperature 1 is absorbed into supplied probabilities.
Fresh behavior policy is assumed at collection; replay staleness is not modeled.
Teacher distribution is supervision, not guaranteed truth or causal/process feedback.
The canonical GKD-style stop-gradient does not define every method called OPD.
No real-model or learner-outcome claim is tested.