05
Which similarly named recipes differ?
Classify the source, feedback, objective, reduction, and gradient semantics before trusting a shared label.
Which similarly named recipes differ?
| Recipe | Response / prefix source | Teacher feedback | Objective and gradient boundary | Evidence status |
|---|---|---|---|---|
| SeqKD | fixed teacher beam output | hard teacher-generated sequence | student likelihood on fixed targets | peer-reviewed · EMNLP 2016 |
| GKD / canonical toy | fixed, mixed, or current-student outputs via λ | full next-token distributions at selected prefixes | declared local divergence; sampled trajectory detached | peer-reviewed · ICLR 2024 + toy boundary |
| ImitKD | scheduled mixture of fixed and student-generated sequences | oracle action or full token distribution | oracle NLL or cross-entropy; implementation schedule matters | peer-reviewed · EMNLP 2020 |
| MiniLLM | theory uses student response distribution; practical sampler is teacher-mixed | sequence and token-level log-ratio terms | sequence reverse KL with score-function terms and stabilizers | peer-reviewed · ICLR 2024 |
| Scalar-reward RL | algorithm-dependent trajectory source | external scalar reward | expected reward or surrogate; not token-distribution KL by definition | separate objective family |
A method name is a starting point, not a complete estimator. Ask what generated the response, what teacher information is visible, how it is reduced, and which dependencies reach the update.
Canonical GKD keeps source and divergence configurable
GKD can mix fixed and student-generated outputs with a coefficient , then choose a token-distribution divergence independently. Its canonical sampling path is stopped for the local update.
Peer-reviewed anchor: Agarwal et al., ICLR 2024. Navigable equations and Algorithm 1: author manuscript v3.
Sequence-level reverse KL retains different terms
MiniLLM begins from reverse KL between complete response distributions and derives likelihood-ratio terms involving sampled tokens. Its practical estimator adds further approximations and stabilizers. That is not the same gradient semantics as freezing a sampled GKD trajectory and differentiating only local token-distribution losses.
Peer-reviewed anchor: Gu et al., MiniLLM, ICLR 2024. See §2 and Appendix A.2 in the linked proceedings paper.
Six details the label still does not answer
- Prompt source: fixed collection, live environment, or another distribution?
- Response source: teacher, current student, stale behavior policy, or a mixture?
- Feedback granularity: full vocabulary, top-k, one sampled token, hard target, or scalar reward?
- Objective and reduction: which arguments, positions, lengths, and units?
- Gradient semantics: detached collection, score-function term, importance weighting, or another estimator?
- Vocabulary and EOS: how are supports aligned and sequence boundaries counted?
Use those questions on two recipes whose axes deliberately disagree.