The rollout lab · Rollouts

Follow draw 3

The most useful thing to track is not the entire table. Track draw 3.

At $T=1$ , that draw is The answer is 5. and always receives reward 0.75 from our fixed verifier. Before changing anything, predict: if we give it different companions, must its relative signal keep the same sign?

Whole-completion toy. A centered group-relative teaching signal (not full GRPO). No model is loaded and no policy update occurs.

Rollouts K 6Try 3 → 6 → 8 first

Temperature T 1.0Then explore the distribution

Live comparisonFollow the ring: draw 3

Dots are rewards; the dashed line is their mean. The segment from mean to dot is the signed difference. Reward scale stays fixed at 0–1.

Group mean 0.7500 · Sum of centered signals +0.0000

Sampling viewProbabilities are not counts

Stems show exact toy probabilities, not empirical frequencies. Counts describe this finite group. Raising K does not change the probability distribution.

See every draw and probability interval

Computed toy draws; display rounded to four decimals
Draw	u	Index	Completion	Reward	Centered A
1	0.3982	0	5	1.0000	+0.2500
2	0.4663	0	5	1.0000	+0.2500
3	0.7087	1	The answer is 5.	0.7500	+0.0000
4	0.6670	1	The answer is 5.	0.7500	+0.0000
5	0.5855	0	5	1.0000	+0.2500
6	0.9719	2	6	0.0000	-0.7500

One reward, three comparisons

Keep T = 1. Draw 3 stays at reward 0.7500 while its companions move the mean.

K = 3

Rewards: 1, 1, 0.75
Mean 0.9167
Draw 3: -0.1667
negative

K = 6

Rewards: 1, 1, 0.75, 0.75, 1, 0
Mean 0.7500
Draw 3: +0.0000
zero

K = 8

Rewards: 1, 1, 0.75, 0.75, 1, 0, 0.75, 0
Mean 0.6563
Draw 3: +0.0938
positive

The reward does not change; the shared baseline does. These are derived snapshots of the seeded whole-completion toy, not training results. At K = 1, the only reward equals its mean and its centered value is zero.

Frozen inputs and the sampling rule

These are invented whole-completion categories, not token-level model outputs.

Fixed inputs · candidate indices are zero-based
Index	Completion	Logit	Reward
0	5	1.2	1
1	The answer is 5.	0.4	0.75
2	6	-0.2	0

Seed 20260903; generator LCG32-v1. Restart the same stream for every prospective setting. K takes its first K draws.

state = (1664525 * state + 1013904223) mod 2^32
u = state / 2^32

Choose the first candidate whose cumulative full-precision probability is strictly greater than u. Compute at full precision; round only displayed values. A seed alone does not make different generators equivalent.

Exact probability derivations from fixed logits (rounded)
T	p(5)	p(The answer is 5.)	p(6)
0.5	0.7919	0.1599	0.0482
1	0.5896	0.2649	0.1454
1.5	0.5051	0.2963	0.1986
2	0.4615	0.3093	0.2292

At T = 1 the rounded values sum to 0.9999; the unrounded probabilities sum to one. No weights, token likelihoods, KL terms or optimizer exist in this sandbox.

The original CPython lab capture is a measured seeded toy-process output, not measured LLM behavior. Other displayed snapshots are computed from the frozen mathematical model. Inspect the capture (local copy).

Read the code or run it yourself

The core comparison needs only a mean and a subtraction. This self-contained arithmetic example is not the sampler:

rewards = [1, 1, 0.75, 0.75, 1, 0]
mean = sum(rewards) / len(rewards)
centered = [reward - mean for reward in rewards]
# mean: 0.75
# centered: [0.25, 0.25, 0, 0, 0.25, -0.75]

Arithmetic output follows from the shown inputs. For the complete seeded sampler, use the canonical Python lab (standard library only).

Complete Python lab (local copy) · Tests (local copy) · Lab manifest (local copy)

Canonical listings: LIST-LAB-001 configuration; LIST-LAB-002 probabilities; LIST-LAB-003 draws; LIST-LAB-004 centering; LIST-LAB-005 boundary checks.

Change the group, not the reward rule

Leave temperature at 1. Try group sizes $K=3$ , $K=6$ , and $K=8$ . Here $K$ counts separately drawn whole completions; it does not count tokens.

Draw 3 at temperature 1; values rounded to four decimals
Group size	Group mean	Reward	Centered signal
3	0.9167	0.7500	−0.1667
6	0.7500	0.7500	0.0000
8	0.6563	0.7500	+0.0938

These values follow from the frozen sampler and reward rule. The same 0.75 score is below one mean, equal to another, and above the third. The baseline is part of the answer whenever you discuss a relative signal.

At the default $T=1$ , $K=6$ , seed 20260903, the recorded local toy run drew candidate indices $[0,0,1,1,0,2]$ . Their rewards are $[1,1,0.75,0.75,1,0]$ , their mean is 0.75, and their centered signals are $[0.25,0.25,0,0,0.25,-0.75]$ . This is output of the disclosed seeded toy process, not samples from a language model. Recorded lab output (local copy)

Now change temperature

Predict whether increasing $T$ from 1 to 2 makes 5 more or less probable. Then try it. Follow the chain: temperature changes probabilities; probabilities and the fixed random draws determine the new group; the reward rule determines its scores; those scores determine the mean.

Every setting starts a prospective draw from the same initial seed. Reset restores $T=1$ , $K=6$ , and seed 20260903. Nothing edits the verifier's reward for a recorded completion, and nothing updates model weights.

This remains a whole-completion toy and centered teaching signal—not full GRPO.

Check your understanding

Set $K=1$ . Why is the only centered signal zero, whatever completion appears?

Reveal answer

The only reward is also the group mean: $r_1-\bar r=r_1-r_1=0$ .

A useful score still does not tell us which policy produced the data.