Keep the definitions close.

Notation, primary sources, and the exact boundary around our experiment.

A compact glossary

LSTM

Long Short-Term Memory, a recurrent neural-network architecture. This guide uses separate forget, input, and output gates, a tanh candidate, and no peepholes or projection.

Recurrent state

A numerical state passed between sequence steps. Shared parameters update it from new input and previous state.

Input $x_t$

The current input vector, with $D$ coordinates in this guide. The lab's event labels are not encoded text.

Cell state $c_t$

The carried vector updated by adding a retained contribution and a gated candidate. It is not globally constrained to the tanh range.

Hidden state $h_t$

The exposed vector $o_t\odot\tanh(c_t)$ in the standard non-projected form. It also feeds future gate and candidate computations.

Forget gate $f_t$

A coordinatewise multiplier on the previous cell state. Larger values retain more of that value at the current step.

Input gate $i_t$

A coordinatewise multiplier on the candidate write. It need not be the complement of the forget gate.

Candidate $g_t$

A proposed signed value, computed with tanh in the standard cell. It is distinct from the sigmoid gates.

Output gate $o_t$

A multiplier controlling exposure as current hidden state—not the model's final answer. A change can affect later learned gates through hidden-state feedback.

Elementwise product $\odot$

Multiplication of matching vector coordinates, preserving the vector's shape.

Sigmoid and tanh

Activation functions. For finite real inputs, sigmoid lies strictly in $(0,1)$ and tanh in $(-1,1)$. Exact endpoints in the lab are prescribed idealizations.

Direct-path sensitivity

The derivative along the explicit cell carry route when gates and candidates are held fixed. Its forget-gate product is not the full recurrent derivative.

BPTT

Backpropagation through time: differentiating the unrolled recurrent computation to obtain loss gradients for shared parameters.

Task readout

A separate transformation from chosen hidden features to predictions appropriate to a task. No task head is executed here.

Sources behind the explanation

Primary papers and pinned documentation inspected on 4 September 2026. These support the concepts, not measured learning effectiveness or this toy's invented gate schedule.

1. PyTorch 2.8 — LSTM

   Opening equations, input/output definitions, and projection distinction. Source for the standard no-peephole, non-projected update taught here; not a claim about the latest release.

2. PyTorch 2.8 — RNN

   Plain tanh recurrent update and parameter reuse in the time loop. LSTM is also an RNN architecture.

3. On the difficulty of training recurrent neural networks

   Pascanu, Mikolov, and Bengio (2013), §§1–2, equations (1)–(5), and Fig. 2. Unrolled computation, BPTT, and temporal gradient products; not evidence that this toy trained a network.

4. Long Short-Term Memory

   Hochreiter and Schmidhuber (1997), §4 and Fig. 1. Historical original architecture, not the exact modern forget-gate equations.

5. Learning to Forget: Continual Prediction with LSTM

   Gers, Schmidhuber, and Cummins (2000). Journal record for the later forget-gate extension; this date is not a claim of first public appearance.

6. Learning Precise Timing with LSTM Recurrent Networks

   Gers, Schraudolph, and Schmidhuber (2002), §§2–3. Forget-gate history, cell-output feedback, and a separate output layer. Its own output equation differs from this guide, so it is not the source for our tanh hidden-state equation.

7. PyTorch 2.8 — Sigmoid

   Function definition supporting the finite-real range of gate multipliers.

8. PyTorch 2.8 — Tanh

   Function definition supporting the finite-real candidate and squashed-state ranges.

Reproduce the six-step trace

The canonical JavaScript module (local copy) defines the fixed candidates, gates, update arithmetic and validation. Run node model.mjs from the source folder for baseline, comparisons and transfer values. No framework, randomness or learned weights are involved.

Numerical tests (local copy) check the hand cases, invariants, invalid inputs and all 1,331 gate combinations at six inspected positions. Calculations use full precision; the display rounds to three decimals.

The experiment directly assigns gates, including idealized 0/1 values, and fixes candidates independently of hidden state. It does not train an LSTM, process text, perform BPTT, or evaluate sequence predictions. Real learned gates can depend on earlier hidden states, so this toy's invariances do not all carry over.