Nothing here is pre-trained. Both models begin blank and learn the corpus in front of you, generating a continuous stream the whole time. Watch each stream climb from noise toward structure as the model fills in. Same corpus, two ways of learning it.
Bigram — live counts. An empty 27×27 table. Each tick it reads the next corpus character and increments one count, then speaks a few characters by sampling the current, partial table. Early on most rows are empty, so it stammers and emits ? for unseen letters; as coverage climbs past ~200 pairs the e/t/a/space rhythm appears. This is learning as accumulation — no gradient, just tallying what follows what.
Neural — live SGD. A softmax model over learnable logits, weights initialised to zero (uniform predictions = pure noise). Each tick runs one real gradient step of cross-entropy on a random corpus pair and then speaks from the current weights. The loss meter falls from ~3.3 toward the corpus entropy; the speech gains word-length spacing and real letter runs as it descends. This is learning as optimisation — the same mechanism that trains the big transformer, shrunk to something you can watch converge in seconds.
Why two? Same target, two epistemologies. Counting reaches the statistics instantly for pairs it has seen and knows nothing about the rest; gradient descent knows a little about everything from the first step and refines all of it together. Run them side by side and you can watch the difference in how they get smart.