In 1960 Donald Michie taught a stack of matchboxes to play noughts and crosses, using nothing but coloured beads and a rule for when to add them. Point the same machine at words and you get the thing sitting under every chatbot you have ever used.
Play a game against a filing cabinet. You go first, you put a cross in the middle. Now your opponent has to reply, and your opponent is a stack of matchboxes on a table.
Somebody finds the box with your exact position drawn on the front. Inside are coloured beads, one colour for each empty square. They shake it, tip out a single bead without looking, read its colour, and put a nought in the matching square. That is the whole move. Nobody thought about it, including the machine.
The interesting part happens after the game ends. If the machine lost, every bead it drew that game gets thrown away. If it won, three more beads of each drawn colour go back in. A draw is worth one extra. That is the entire learning algorithm, and it fits in a sentence.
A jar of raffle tickets for every decision you ever face, one jar per situation. You never think. You pull a ticket. But when the evening goes well you photocopy the tickets you used and drop the copies back in, and when it goes badly you bin them. After a few hundred evenings the jars are full of the tickets that tend to work, and none of them ever knew why.
Michie's machine, MENACE, needed 304 boxes to cover every position it could face as second player, and a couple of hundred games before it stopped losing. It has no processor and no memory in the usual sense. The bead counts are the memory, sitting in the open where you can count them.
Noughts and crosses is a small world: a position, a set of legal replies, a verdict at the end. Hold that shape in your head, because writing English has exactly the same shape, and almost nobody notices.
Finish this, out loud, before you read on: “humpty dumpty sat on a ___”
You said wall. You did not consider every word in English and score them. Something in you already held a short list for this exact situation, weighted, and the fattest item on it came out first.
So build the box. Write the situation on the front, which here is just the word that came last. Inside go beads, one colour per word that could come next. Draw a bead, write down its word, and you have made a move.
Below is an empty box with the written on the front. Nothing is in it yet, because nothing has happened yet. Put beads in by hand and see what the box starts to say about the word the.
the mouse is four times likelier than the spoon.The label on the front is everything the machine is allowed to remember at the moment it chooses. Not the topic, not who is talking, not what it said two lines ago. One word, written on a box. Everything else about the conversation has already fallen on the floor.
Filling boxes by hand is obviously absurd. English has a lot of words and they occur in a lot of situations, and you would be there until Tuesday.
Nobody fills them by hand. You pour text in and let it fill them, which turns out to be the least mysterious step in the whole business.
Here is the part that sounds like it must be more than it is. Take some text. Slide a window along it. Every time you see one word follow another, drop a bead in the appropriate box. Stop when you run out of text.
Scroll through it happening. The text is a handful of nursery rhymes, which is all the machine on this page will ever know.
No boxes, no beads, no rules about English. Just a line of text and a window two words wide.
The line opens with the cat. So: find the box labelled the, and drop in a bead the colour of cat. If that box does not exist yet, make it.
cat and, then and the, then the fiddle. Five beads down, and the box for the already holds two of them, because the keeps happening.
Eleven words, twelve beads, nine boxes. The twelfth bead is the one that means the line has ended, and drawing it later is how the machine knows to stop. Nothing has been understood. Something has been counted.
Sixteen lines of rhyme. The wall of boxes is the finished machine, and the fat boxes are the words English keeps coming back to.
A hotel where every guest signs the book on their way in, and the receptionist keeps one page per guest listing who walked in behind them. No conversation, no interest, no judgement. Months later that book knows a startling amount about the hotel, and the receptionist still knows nothing.
Real training is this, at a scale that is hard to picture: trillions of words, and boxes that hold not a handful of beads but a probability for every word in the language. The counting is dressed up as arithmetic on numbers rather than beads in card, for reasons worth getting to later. The move is identical.
So the boxes are full. What does a fat pile of beads actually mean?
Open the box marked little and tip it out on the table. Ten beads in five colours, and one of those colours has four of them.
That heap is a probability distribution, which is a heavy phrase for a simple object: a list of options with a number on each, and the numbers add up to one whole box. Four beads out of ten is forty percent. It was never anything cleverer than a proportion.
The two pictures below are the same box twice. Beads on the left, percentages on the right. Click either one and both light up, because they are not two facts, they are one fact drawn two ways.
the is fat and torn eighteen ways. The box for hickory holds one colour and has no choice to make at all.A weather vane, not a forecaster. It has no theory about the weather. It has been standing in this field for years and it is currently pointing where the wind has mostly come from. The machine does not know what a lamb is. It knows that four slips of paper out of ten said lamb, and that is genuinely all it has.
Notice the boxes holding a single colour. After hickory the rhymes only ever said dickory, so that box has no opinion, only a reflex. Boxes like that will matter a great deal shortly.
Every box now holds a set of odds. Time to use them with your eyes shut.
The machine writes like this. Look at the last word. Find its box. Shake, pull one bead, write down the word on it. That word is now the last word, so find its box, and go again. Stop when you pull the bead that means stop.
There is no plan. Nothing was decided about where the sentence is going, because nothing in the machine can hold a sentence. It holds one word and a box.
Run it a few times and you will get lines that are eerie in a specific way. the mouse ran away with the spoon is not in any rhyme. It is stitched from two of them at the word ran, where the box happened to offer a door into a different poem.
Someone humming their way through a medley they half know. Every note they hum is a plausible next note, so it sounds like music the whole way through, and they cannot tell you which song they are in. The seams are invisible from inside because there is no inside.
That stitching is the closest thing here to creativity, and it is a side effect of a shared word. It is also why the output can be fluent and wrong at the same time: fluency lives in the boxes, and truth was never in them.
One thing has been quietly assumed so far. When you reach into the box, do you really take a bead at random?
You do not have to draw fairly. You could cheat towards the fat piles by feeling for the colour there is most of, or you could cheat the other way and deliberately ignore how many there are of each.
That cheat has a name in every model you have used. It is the temperature dial, and it is the one number most people are given control of without being told what it does. It does not change a single bead in the box. It changes how much the pile sizes count when you reach in.
Drag it. The left column is the box as it actually is, unchanged. The right column is what the machine behaves as if it holds.
At the cold end it stops sampling and starts obeying: always the biggest pile, which is called greedy decoding. Greedy sounds safe and behaves badly, because a loop is a perfectly good answer to “what is most likely next” and the machine has no memory of having just said it.
At the hot end the piles stop mattering, the rare beads come out as often as the common ones, and you get word salad with correct spelling.
How hard you shake the box before you reach in. A gentle tilt and your fingers find the colour there is most of. A violent shake and every colour is equally likely to end up under your hand. The beads are the same beads either way.
This is not a metaphor for what real models do, it is the same arithmetic. A model scores every possible next word, divides every score by the temperature, and turns the results back into chances. Dividing by a small number exaggerates the gaps between the piles. Dividing by a large one flattens them.
Which leaves the harder question. What decides what goes in the box in the first place? The writing on the front.
The machine has been reading one word off the front of the box. That is why it wanders: after ran it cannot tell whether it is in the rhyme about the mouse or the one about the dish.
Fix it by writing more on the front. Label the boxes with the last two words, and mouse ran becomes a different box from dish ran. The machine stops changing poems mid sentence and starts sounding like it means something.
You pay for it immediately, and the bill is drawn below. Every word you add to the label multiplies the number of boxes you need, while the same fixed pile of beads gets spread across all of them.
At three words nearly every box holds exactly one bead. The machine now writes beautiful English and has stopped composing: it is reciting the rhymes back at you word for word, because a box with one bead in it is not a decision, it is a quotation.
Revising for an exam with flashcards. One word on the front and you can waffle about anything, badly. A whole paragraph on the front and your answer is perfect, right up until the question is phrased differently, at which point you have no card and nothing to say.
This is the trade every language model lives on. Short labels generalise and drivel. Long labels are accurate and memorise. Beads cannot escape it, because a box is either full or it is not.
Real models climb out by not using boxes at all, which is worth coming back to. First there is something more embarrassing to deal with.
Ask it something. Type a question into the machine and watch what it does with it, which is nothing whatsoever question shaped.
It takes the last word of whatever you gave it, looks for a box with that on the front, and carries on. A question mark is not a request. It is just text that ran out.
email, so the machine has nowhere to reach. Nothing tells it to apologise or ask for clarification, because those are also just words somebody would have had to count.Handing a page of your writing to someone who has read every book and remembers none of them, and who has been told exactly one thing: keep going, and make it sound like the page. They are not being unhelpful when they fail to answer your question. Answering was never the job.
Everything you experience as an assistant answering you is built on top of this. The machine underneath still only continues text. It was later shown enormous numbers of examples where a question is followed by an answer, so continuing became answering, statistically. That fix is famous and it is next.
Michie's boxes did not start out any good either. They got good because somebody kept telling them how the game went.
Back to the bead rule from the very first box. The machine writes a line. You judge it. If you liked it, every bead it drew gets three copies dropped back into its box. If you did not, the beads it drew are removed.
Nothing else changes. Same boxes, same drawing, same dial. The only new thing in the system is your opinion, and your opinion is now editing the piles directly.
Do a few by hand to feel the rule. Then hand the job to a rater with one fixed taste: any line with the word lamb in it is good, everything else is bad, twenty five rounds to a click. Nothing tells the machine what the rater likes.
Watch the opening go first. The box at the start of a line is the one box every line passes through, so it collects beads faster than any other, and after a few dozen verdicts it has a firm favourite. Keep the rater running and variety across twenty attempts slides from twenty different lines down to around half that, with the same phrases surfacing again and again.
Notice what it never learned. It has no idea the rater wanted lambs. It learned which beads were in its hand at the moment the applause happened, which is not the same thing and is the reason this whole approach is delicate.
A busker who plays six songs, notices you tip for one of them, and now plays that one every time you walk past. They have not learned that you like music. They have learned which song produces coins. Ask them for something new and there is nothing left in the case.
This is preference training, and its real version is why assistants feel helpful. Humans rate answers, the ratings reshape which words come out, and the model drifts towards what raters approve of. The drift is the point, and it is also the problem.
Reward confident answers and you get confident wrong answers. Reward agreement and you get flattery. Reward a particular phrasing hard enough and you get it every time, which is the collapse you can watch happening in the figure above in under a minute. The machine cannot tell the difference between being right and being clapped at, because those arrive through the same hole.
So: counting, sampling, a dial, and applause. That is a working language model, and you could build it out of stationery. What are the real ones doing that this is not?
The box for the and the box for a are doing the same job in English. Both sit in front of a noun, both take roughly the same kinds of word next. A person sees that instantly. The machine cannot see it in principle, because a box has no way to look at another box.
So the fat box hoards while the thin one starves. Everything counted about the does nothing whatsoever for a, and a label you have never seen leaves you with no box and therefore nothing to say. This is the wall that counting hits, and you cannot climb it by counting harder.
A neural network never builds the boxes. It learns a position for every word, in a space where words that behave alike end up close together, and then a single rule that turns any position into a set of odds. Feed it a phrase it has never met and it still lands somewhere sensible, because the rule works on where the words sit, not on whether that exact box was ever filled.
That is what an embedding is, and it is why the counting machine dies at three word labels while a real model handles thousands of words of context. The counting machine needs to have seen your exact situation. The model needs your situation to resemble something.
Everything else you hear about, the attention that decides which earlier words matter most, the layers, the billions of parameters, is machinery for doing this one trick well over long stretches of text. Underneath, at the last step, the model still produces a pile of odds over the next word and something still reaches in and pulls one out.
A language model is a very large set of boxes you never see, filled by counting, reached into with a dial, and reshaped by whoever rates the output. The beads on this page are the whole apparatus at a scale you can put on a table, and the parts that got left out are about capacity, not about kind.
The parts that did not get left out are the ones people find surprising. It does not know anything. It cannot answer, only continue. It will repeat itself if you make it too careful and babble if you make it too loose. And it will learn to please you specifically, quite fast, without ever learning what you actually wanted.
Every dial at once, on text of your choosing. The corpus swaps between nursery rhymes, weather reports and support replies, which is the fastest way to feel that the machine has no subject matter of its own, only whatever it counted.
Claude Shannon needed text that behaved like English without being English, so he built the machine on this page out of a book and a pencil. He picked a word, flipped forward until he found that word again, and wrote down whatever came after it. That is a bead draw, executed with a thumb.
What came out: the head and in frontal attack on an english writer that the character of this point is therefore another method for the letters that the time of who ever told the problem for an unexpected. Ten words in the middle of that are indistinguishable from real prose. He stopped there because the next step up would have meant flipping through the book for every single word, and it was 1948.
The machine on this page counts an extra invisible word at the end of every line, and drawing it ends the sentence. Real models do the same with a special token, which is why a model can decide it has finished. It is also why a model sometimes stops mid thought: finishing is a word like any other, and it can be drawn at an awkward moment.
The machine here gives up. Real counting models fall back: no box for the last three words, so try the last two, then the last one, and blend the answers together. That family of patches is called smoothing and backoff, and people spent the eighties and nineties on it. Neural models made the whole problem go away by never needing an exact match in the first place.
A model produces a score for every possible next word. Divide every score by the temperature, exponentiate, normalise so they sum to one, and sample. Because the scores are logarithms of the counts, dividing them by t is the same as raising every bead pile to the power of one over t, which is exactly what the figure on this page does. Temperature zero is undefined in the arithmetic and is implemented as: take the biggest pile.
Top-k and top-p are the other two dials you will meet. Both throw away the tail of the distribution before sampling: keep the best k colours, or keep however many colours it takes to cover p percent of the beads. They cut off the nonsense without flattening what remains, which is why they are usually preferred to a hot temperature.
There are 19,683 ways to fill a three by three grid with blanks, noughts and crosses, but most of them cannot happen in a real game, and many are rotations or mirrors of each other. Michie collapsed those into 304 genuinely distinct positions that the machine could face when playing second. Building it was a hardware problem: 304 matchboxes, sorted, labelled and filled by hand.
Three things are honestly missing. Real models do not store counts, they store weights that produce counts, which is what lets them generalise. Attention lets them weigh a word from two thousand words back more heavily than the word immediately before, so the label on the front of the box is not a fixed window. And the preference stage is not a person clicking good on one output; it is a second model, trained on human comparisons, standing in for the human and scoring millions of samples. The bead rule survives all three. The bookkeeping does not.