02
Who generated the supervised state?
The sampled token becomes part of the next state. Provenance travels with that collection event.
Who generated the supervised state?
OPD-TOY-001 · invented teaching fixture At prefix x, vocabulary order is [A,B] and the student distribution is [0.75,0.25]. The contract fixes a uniform draw at 0.8. Its inverse-CDF interval selects B, so the next visited prefix is xB.
| Token | Student probability | Sampling interval | Result |
|---|---|---|---|
| A | 0.75 | [0, 0.75) | not selected |
| B | 0.25 | [0.75, 1) | 0.8 → B → xB |
The same contract also supplies a fixed teacher demonstration whose first token is A, reaching xA. In this comparison, the occurrence of xA is fixed/off-policy and the occurrence of xB is student-generated/on-policy at collection. A prefix string alone cannot reveal who generated it.
Keep collection, supervision, and update distinct
- Sample: student draw
0.8 → B. - Visit: the next prefix is
xB. - Query: teacher and student both return distributions at
xB. - Compare: the chosen divergence returns nats.
- Update: local student parameters receive a gradient; the sampled path stays detached.
The five labels are a teaching decomposition. Implementations may fuse forward passes, but the logic remains: sampling determines the states; the teacher supplies feedback at those states; the objective turns feedback into a scalar; the update differentiates the declared estimator.
Detach the draw, not the student
The canonical GKD-style toy does not backpropagate through the discrete 0.8 → B draw. During the local update, the realized path [x,xB] is treated as fixed. The student distribution evaluated at x or xB is still differentiable through its logits and parameters.
Boundary. A fixed prompt collection is compatible with on-policy completion sampling. Reusing old responses after the student changes may not be. Record the behavior-policy snapshot instead of inferring freshness from a method name.
Now that xB exists, what exactly does the teacher provide there?