Field notes

Similar labels, different estimators

Use this page after the case. It keeps publication status, state source, feedback granularity, objective, and update semantics in separate columns.

Method matrix

These rows are not interchangeable recipes.
Method	State / trajectory source	Feedback	Objective	Update semantics	Source status
Canonical GKD / fully on-policy	Current student completions when λ = 1	Full teacher/student token distributions at identical prefixes	Declared divergence; forward, reverse, JSD tested	Detached student sampling in Eq. 4	Peer-reviewed ICLR 2024
SeqKD	Fixed teacher-decoded sequence	Sequence target used for student likelihood training	Sequence-level teacher target / NLL setup	Supervised update on fixed sequence	Peer-reviewed EMNLP 2016
ImitKD	Mixture of initial data and student generations; generated pool may be older	Teacher best token or distribution at visited prefixes	Imitation cross-entropy variants	Mixed-data imitation-learning loop	Peer-reviewed EMNLP 2020
MiniLLM	Student sequence distribution with practical teacher-mixed sampling	Sequence-level reverse-KL estimator terms	Reverse KL	Policy-gradient derivation, IS and length terms	Peer-reviewed ICLR 2024; later arXiv v6 retitled
Scalar-reward RL	Policy trajectories under the algorithm contract	Scalar reward / return signal	Reward objective, optionally regularized	Policy-gradient family dependent	Separate term in Agarwal Eq. 5
Thinking Machines implementation	Student rollouts	Teacher/student log-probs on sampled tokens	Negative sampled reverse log-ratio as per-token advantage	RL importance-sampling loss	2025 engineering report; not peer reviewed

Compact comparison only; read the primary papers before implementing a trainer. OPD-CL-007 DETECTIVE-CL-003 DETECTIVE-CL-004 DETECTIVE-CL-005 DETECTIVE-CL-006 DETECTIVE-CL-007 DETECTIVE-CL-008

What is sourced, derived, toy, or unresolved?

sourced

Sourced

Student-generated GKD trajectories, same-prefix distribution comparison, independent data-source/divergence axes, fixed-prompt compatibility, detached GKD sampling, and method-specific estimator distinctions.

derived

Derived

The categorical draw, three reverse-KL values, unrounded mean, operation decomposition, and trace-order invariants.

toy

Toy

Every probability, draw, version label, complication record, and narrative case detail.

unresolved

Unresolved until specified

Cross-tokenizer mapping, top-k tail handling, allowed policy lag/correction, EOS inclusion, production length weighting, and teacher correctness.

Exact claim index

Each point-of-use citation lands on one of these stable records. Expand a record for its status, source locator, and wording boundary.

OPD-CL-001In canonical GKD, “on-policy” identifies output trajectories sampled from the student and the prefixes visited along those trajectories.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, §3.1, Eq. 4 and Algorithm 1 lines 5–10; ICLR abstract/record.
Wording boundary: Say “student-generated output trajectories/prefixes,” not that every prompt is newly sampled online.

OPD-CL-002The teacher is evaluated at the same student-visited prefix used for the local student prediction; the returned feedback granularity is method-specific.

Classification: Sourced fact.
Status: Qualified.
Source locator: SRC-GKD-ARXIV-V3, §3 Eq. 2 and §3.1 Eq. 4; SRC-TML-OPD-2025, Implementation / Pseudocode steps 2–3.
Wording boundary: Do not call all feedback a full distribution and do not equate sampled-token feedback with full-vocabulary KL.

OPD-CL-003Trajectory provenance and divergence choice are independent axes; an on-policy student trajectory can be paired with forward KL, reverse KL, JSD, or another declared divergence.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, §3.1, Algorithm 1 (lambda and divergence D are separate hyperparameters), boxed GKD objective, and GKD-variants paragraph in §4.
Wording boundary: Do not infer update semantics from the word “on-policy.”

OPD-CL-004Student sampling, prefix visitation, teacher evaluation, local loss computation, loss reduction, and parameter update are separable conceptual operations.

Classification: Derivation from sourced algorithms.
Status: Supported.
Source locator: Premises: SRC-GKD-ARXIV-V3, §3 Eq. 2 and Algorithm 1; SRC-TML-OPD-2025, four-step implementation outline.
Wording boundary: Say “conceptual operations”; do not market it as a universal architecture.

OPD-CL-005The canonical GKD-style toy does not backpropagate through the student sampling distribution or discrete sampled token.

Classification: Local toy contract aligned with a sourced GKD property.
Status: Supported for canonical GKD and this toy.
Source locator: SRC-GKD-ARXIV-V3, §3.1 immediately after Eq. 4 and after the boxed GKD objective.
Wording boundary: Never generalize to MiniLLM, the 2025 engineering implementation, or every method called OPD.

OPD-CL-006A fixed input/prompt collection can coexist with on-policy student-generated completions.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, Eq. 4 (x ~ X, y ~ p_S(.|x)) and Algorithm 1 line 6.
Wording boundary: State which portion has fixed provenance and which portion is student-generated.

OPD-CL-007Canonical GKD's detached trajectory with token-distribution losses and MiniLLM's sequence-level reverse-KL policy-gradient formulation are distinct estimators with different gradient semantics.

Classification: Sourced comparison.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, §3.1 Eq. 4; SRC-MINILLM-ICLR-2024, §2.1 Eq. 1 and §2.2 Eqs. 2–7; SRC-GKD-ARXIV-V3, related-work discussion.
Wording boundary: Attribute each estimator to its paper/version.

DETECTIVE-CL-001The canonical paper is an ICLR 2024 peer-reviewed conference paper; its current readable author manuscript is arXiv v3 dated 17 January 2024.

Classification: Sourced bibliographic fact.
Status: Supported.
Source locator: SRC-GKD-ICLR-2024, title/authors/conference record; SRC-GKD-ARXIV-V3, header/version line.
Wording boundary: Do not call later 2026 OPD preprints the canonical source.

DETECTIVE-CL-002GKD's tokenwise sequence discrepancy is a declared length reduction over divergences between teacher and student next-token distributions at identical prefixes.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, §3 Eq. 2.
Wording boundary: The article must still declare its own EOS inclusion and length convention; the toy uses two visited positions and no EOS row.

DETECTIVE-CL-003Scalar-reward RL and the distillation divergence are separate objective terms that can be combined but are not synonyms.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, §3.2 Eq. 5.
Wording boundary: Do not label the toy “RL”; it has no scalar reward or policy-gradient update.

DETECTIVE-CL-004SeqKD trains on a fixed teacher-generated sequence rather than gathering current-student trajectory prefixes for feedback.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-SEQKD-EMNLP-2016, §3.2, pp. 1319–1320; corroborated by SRC-GKD-ARXIV-V3, §3 baseline descriptions.
Wording boundary: Describe the paper's sequence-level teacher decoding/training setup; do not reduce all offline distillation to SeqKD.

DETECTIVE-CL-005ImitKD uses a sequence-level mixture of initial data and student generations, and its generated pool may contain outputs from earlier iterations.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-IMITKD-EMNLP-2020, §3.1–3.4, Eqs. 4–6 and Algorithm 1, pp. 6123–6125.
Wording boundary: Call it mixed and potentially stale under its stated pool strategy; do not relabel all ImitKD states as current on-policy.

DETECTIVE-CL-006The peer-reviewed MiniLLM formulation optimizes a sequence-level reverse KL using a policy-gradient derivation and practical estimator terms that are absent from OPD-TOY-001.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-MINILLM-ICLR-2024, §2.1 Eq. 1; §2.2 Eqs. 2–7; Appendix B.
Wording boundary: The 2026 retitled arXiv v6 is a post-publication revision; use the ICLR paper for this peer-reviewed claim.

DETECTIVE-CL-007The 2025 Thinking Machines implementation samples student trajectories, retrieves student and teacher log-probabilities on sampled tokens, assigns the negative sampled log-ratio as a per-token advantage, and updates through an RL importance-sampling loss.

Classification: Sourced engineering behavior.
Status: Supported for that report's described implementation.
Source locator: SRC-TML-OPD-2025, Implementation / Pseudocode steps 1–4 and code excerpt.
Wording boundary: Engineering report, not peer-reviewed consensus or a universal OPD recipe.

DETECTIVE-CL-008Full-vocabulary teacher distributions, sampled-token log-ratios, and top-k teacher feedback expose different information and are not interchangeable without an estimator/renormalization contract.

Classification: Derivation plus source-bounded comparison.
Status: Qualified.
Source locator: Premises: SRC-GKD-ARXIV-V3, Eq. 2 uses full distributions; SRC-TML-OPD-2025, Pseudocode step 3 uses sampled-token log-probabilities and explicitly says top-k/logit distillation was not tested in its experiments.
Wording boundary: Do not attribute an empirical top-k result to the engineering report; tail-mass handling remains implementation-specific.

DETECTIVE-CL-009A direct tokenwise divergence requires teacher and student probabilities over an aligned vocabulary/token event at the same prefix.

Classification: Derivation from the definition.
Status: Qualified.
Source locator: Premises: SRC-GKD-ARXIV-V3, §2 autoregressive policy definition over vocabulary V, §3 Eq. 2.
Wording boundary: Cross-tokenizer mapping is not specified by the canonical paper; label the mismatch case unresolved until a mapping is declared.

DETECTIVE-CL-010Sampling temperature changes the distribution of student-visited trajectories; canonical GKD reports training with student temperature 1.

Classification: Sourced fact.
Status: Supported.
Source locator: SRC-GKD-ARXIV-V3, §2 temperature definition and §3.1 after Eq. 4.
Wording boundary: Record rather than universalize temperature; the toy's fixed categorical draw is not a temperature experiment.

DETECTIVE-CL-011EOS inclusion, truncation, and sequence-length reduction affect the implemented objective and must be declared.

Classification: Inference from source differences.
Status: Qualified.
Source locator: Premises: SRC-GKD-ARXIV-V3, Eq. 2 averages over L_y and task appendices use output caps; SRC-MINILLM-ICLR-2024, §2.2 includes length normalization.
Wording boundary: The case file declares two visited positions, excludes EOS, and uses an unweighted mean; no broader default is claimed.

DETECTIVE-CL-012Stored data is only strictly current on-policy relative to the behavior policy that produced it; a later update policy may differ.

Classification: Sourced/inferred implementation condition.
Status: Qualified.
Source locator: SRC-GKD-ARXIV-V3, Algorithm 1 samples from current student at each step; SRC-IMITKD-EMNLP-2020, §3.4 permits generated samples from earlier iterations; SRC-MINILLM-ICLR-2024, Appendix C.1 describes inner optimization over collected samples.
Wording boundary: Record behaviorPolicyId and updatePolicyId; do not call every lag invalid without the method's correction contract.

DETECTIVE-CL-013For prefix x, D_KL([0.75,0.25] || [0.5,0.5]) = 0.13081203594113696… nats.

Classification: Derivation.
Status: Supported.
Source locator: OPD-CORE-v1 fixture; recomputed by lab/record_trace.py and an independent decimal calculation.
Wording boundary: Display 0.130812; retain full precision for the mean.

DETECTIVE-CL-014For prefix xB, D_KL([0.6,0.4] || [0.9,0.1]) = 0.3112386795830576… nats.

Classification: Derivation.
Status: Supported.
Source locator: Same as DETECTIVE-CL-013.
Wording boundary: Display 0.311239.

DETECTIVE-CL-015The full-precision unweighted mean of the two included local losses is 0.2210253577620973… nats, displayed 0.221025.

Classification: Derivation.
Status: Supported.
Source locator: Same as DETECTIVE-CL-013.
Wording boundary: Do not average the six-decimal displays: that gives 0.2210255, which rounds differently.

DETECTIVE-CL-016The off-policy comparison row xA has reverse KL 0.5826853020432398… nats and is excluded from the canonical mean.

Classification: Derivation.
Status: Supported.
Source locator: OPD-CORE-v1; independent computation and tests.
Wording boundary: Its numerical value is a diagnostic comparison, not part of OPD-TOY-001's canonical loss.

DETECTIVE-CL-017The shared kernel, extra cases, and generated illustrations are teaching devices and provide no empirical evidence about OPD efficacy or learner outcomes.

Classification: Provenance statement.
Status: Supported.
Source locator: OPD-CORE-v1 experiment boundaries; image-generation.md; project files.
Wording boundary: Label at the point of use, not only in this ledger.

Open the complete Markdown claim ledger copied into this local build (local copy).

Primary sources and versions

Agarwal et al., ICLR 2024 official record · canonical peer-reviewed source.
Agarwal et al., arXiv v3 · current author manuscript, revised 17 January 2024; §3 Eq. 2, §3.1 Eq. 4 and Algorithm 1.
Kim & Rush, SeqKD, EMNLP 2016 · §3.2, pp. 1319–1320.
Lin et al., ImitKD, EMNLP 2020 · §3.1–3.4, Eqs. 4–6 and Algorithm 1.
Gu et al., MiniLLM, ICLR 2024 conference paper · §2.1 Eq. 1, §2.2 Eqs. 2–7.
MiniLLM author manuscript v6 · post-publication revision, retitled in 2026; not silently substituted for the conference source.
Lu & Thinking Machines Lab, 2025 engineering report · sampled-token implementation; explicitly did not test top-k/logit distillation in those experiments; living page, not peer reviewed.

Implementation disclosures

Contract: OPD-CORE-v1; SHA-256 9AAD0A…0E697, verified before build.
Sampling: Already-declared toy probabilities, temperature recorded as 1, half-open categorical intervals, u=.8; draw detached.
Alignment: Teacher and student use DETECTIVE-ALIGNED-TOY-v1. The mismatch case fails closed.
Length / EOS: Exactly two included prefixes, x and xB; EOS excluded; unweighted mean of unrounded values.
Freshness: Canonical behavior and loss policies are both student-v0. A separate case exposes v0 → v1 lag.
Feedback granularity: Canonical case is full-vocabulary. Sampled-token and top-k variants are described, never equated.
Update: No optimizer, learning rate, logits, or parameter update is invented.
State runtime: Pure reducer; no state-machine library. The trace is generated by Python and replayed, not recomputed, in the browser.

policy.json (local copy)trace.ndjson (local copy)static report (local copy)lab manifest (local copy)

What this case cannot establish

It does not show that OPD universally beats SFT, offline distillation, or RL; that a stronger or more confident teacher is correct; that tokenwise feedback supplies causal/process supervision; that full-vocabulary, sampled-token, and top-k feedback are equivalent; or that completing this case demonstrates mastery. No real model was trained or downloaded.

The experimental hypothesis about diagnostic reconstruction remains unproven. A learner study would be needed to compare transfer, retention, or speed across variants.