A machine that produces instead of choosing

The old machines chose from a list. This one does not have a list.

Before any mechanism, the shape of the change, because it is easy to miss how specific it is.

For about fifteen years the useful, working, everywhere-in-your-life kind of artificial intelligence was a sorting machine. Artificial intelligence just means getting a computer to do something we would call intelligent if a person did it, and until recently the thing it did was decide which box something belonged in.

Your email provider looked at a message and put it in one of two boxes: spam or not spam. Your bank looked at a payment and put it in one of two boxes: fraud or fine. Your phone looked at a photo and put a name to the face. A system like this is a classifier, and the box it picks is a label.

Classifiers are genuinely clever and they run the world. But notice the ceiling built into them: the answer already exists before the question is asked. Somebody wrote down the list of boxes. The machine's entire job is to pick one. Ask a spam filter to write you a poem and the request is not hard, it is meaningless, because a poem is not on the list.

Generative systems have no list. Asked for a poem about a lighthouse, the machine produces a poem about a lighthouse, and that particular arrangement of words has, in all likelihood, never existed before. Not retrieved. Not assembled from a template. Produced.

Both kinds learn from training data: a large pile of examples they are shown while being built. The difference is what they learn to do with it. One learns to sort the pile. The other learns to make more of it.

So the interesting question is not what generative systems can produce. It is what would have to be true for a machine to produce anything at all, given that it cannot copy and has no list. Answering that takes four ideas, and the first is that the machine never sees your words in the first place.

Text has to become numbers before anything can happen to it

A computer multiplies numbers. That is the whole of what it does at the bottom. So the first thing done to any text you send is to convert it into numbers, and it happens in two steps that get conflated constantly.

Step one is chopping. Text is cut into tokens: chunks from a fixed list called the vocabulary, decided once before the model was built and never changed after. A token is usually a common short word, or a fragment of a longer word, or a space plus the word following it, or a single mark of punctuation. Roughly four characters on average.

It matters that these are tokens and not words, because almost every explanation of this technology says "word" and then everything downstream is slightly wrong. The model has never seen a word. lighthouse may well be two tokens. A rare surname may be five. The model's world is made of these chunks and its whole sense of language is built on them.

Step two is turning each token into an embedding: a list of numbers, of fixed length, standing for that token. A list of numbers like this is called a vector, and ours are 4,096 numbers long. No single position in the list means anything you could name. What matters is the arrangement: tokens used in similar ways end up with similar lists, so harbour and port sit near each other and neither sits near saxophone.

That nearness is not decoration. It is the only reason the machine can handle a sentence it has never seen: a new arrangement of familiar regions is still somewhere it can reason about. And the text you type in, the thing that starts all this, is called the prompt.

Text is now numbers arranged so that closeness means similarity. That is the raw material. What the machine actually does with it is one operation, repeated, and it is far narrower than most people expect.

The machine does exactly one thing, and it is smaller than you think

Given some text, produce a guess about what comes next. That is the entire operation. Everything a generative text system appears to do, every explanation, every translation, every argument, is that one move performed over and over.

But the guess is not a single answer, and this is the detail that almost every plain-language account leaves out, taking most of the real understanding with it.

The model does not decide the next token is waited. It produces a number for every single token in its vocabulary, all 100,000 of them, saying how likely each is to come next. A probability is just a number between 0 and 1 saying how likely something is, and a complete set of them, one per possibility, adding up to 1, is a probability distribution.

So the output of the machine, at every step, is a distribution over a hundred thousand possibilities. Almost all of them get essentially zero. A handful get most of the weight. The figure below shows the top few, which is all anyone ever looks at, but keep the shape of the real thing in mind: an enormous list, nearly all of it flat.

To produce more than one token, the chosen token is added to the end of the text and the same question is asked again. Feeding its own output back in as input is called autoregressive generation, and the text it is looking at when it answers is its context.

Two things in that figure deserve to be said out loud.

First, the distribution is sometimes sharp and sometimes flat. After the lighthouse keeper waited until, the leading candidate takes 41 percent, because the grammar of the sentence has narrowed things considerably. One step earlier it took 34 percent, spread across many more plausible options. The model is, in effect, telling you how constrained the moment is.

Second, nothing here is being retrieved. There is no stored poem, no library of sentences to copy. There is a set of numbers that, given some text, produce a guess about the next piece of text. Everything else is that guess, run repeatedly.

Which raises the question that the "take the top one" button quietly begs. If the model always hands back a distribution, who decides which token actually gets used? And why, if you ask the same question twice, do you get two different answers?

Choosing from the spread is a separate decision, and it is yours

The model produces the distribution. Something else picks a token out of it, and that something is a rule you can change without touching the model at all.

The simplest rule is: always take the highest one. That is called greedy decoding, and it makes the system completely deterministic. The same prompt gives the same answer every time, word for word, forever. It also, in practice, produces flat and repetitive text that has a habit of getting stuck in loops.

The usual rule instead is sampling: pick at random, but weighted by the probabilities. A candidate at 60 percent gets chosen about six times in ten; a candidate at 4 percent gets chosen about four times in a hundred. Not chosen often, but not impossible either, and that is where the variety comes from.

This is the real answer to why the same prompt gives different answers. Not that the machine is moody or thinking afresh. A weighted die is being rolled at every token, and the same first roll going differently sends the entire rest of the text down another path.

And there is a dial on the die. Temperature reshapes the distribution before the roll. Below 1 it exaggerates the differences, making likely candidates even more likely and the output more predictable. Above 1 it flattens them, giving unlikely candidates a real chance, and the output gets more surprising and less reliable in equal measure. At 0 it collapses to greedy decoding.

Sit with the twenty-roll tally for a moment, because it demolishes a common misreading. The leading candidate is not "the answer". It is a candidate that wins about seven times out of twenty and loses the other thirteen. Over a 300-token reply that die is rolled 300 times, and the chance of any two replies coming out identical is effectively nil.

Variety is not the model thinking differently. It is the same guess, rolled again.

This also explains a common frustration honestly. When a system gives a good answer and then a worse one to the same question, nothing has broken and nothing has been changed. You are seeing two draws from one distribution, which is the design working as intended.

Everything so far describes a machine that already has its opinions. Where did they come from? Nobody wrote the probability of keeper following the lighthouse into a table anywhere.

Nobody wrote the rules. The machine was corrected until it stopped being wrong.

Here is where "it learns the patterns" usually appears and the explanation stops. It is worth pushing through, because the actual method is simple enough to hold in your head and it explains several later things that otherwise look like magic.

A model is a very large collection of numbers called parameters. Ours has twelve billion of them. They are arranged in a structure called a neural network, which for our purposes means: the input numbers get multiplied by the parameters in a fixed order, layer after layer, and whatever falls out the far end is the answer. The structure never changes. Only the values of the parameters do, and only during training.

At the start those twelve billion numbers are set randomly. The model is not ignorant in an interesting way; it is noise. Shown the lighthouse, it produces a distribution that is essentially flat across all hundred thousand tokens.

Training is then a loop, and it has four steps that repeat until the electricity bill becomes intolerable.

Take a real piece of text from the training data and hide what comes next. Let the model guess: it produces a distribution. Measure how wrong that guess was, by checking how much probability it gave to the token that actually came next; that measure of wrongness is called the loss. Then nudge every one of the twelve billion parameters a tiny amount in whichever direction would have made the loss smaller. That nudging is gradient descent.

Then do it again with the next piece of text. And again, trillions of times.

Two consequences of that loop are worth stating, because both get misunderstood constantly.

The first: nobody chose what the model learned. No one wrote a grammar rule, a fact, or a style guide into it. The parameters arrived at their values purely by being wrong slightly less often, over and over. Grammar, facts, tone and the shape of an argument all appear because getting them right lowers the loss on real text. They were never specified, only rewarded.

The second: this is why nobody can point inside a model and show you where something is stored. Knowing that lighthouses are on coasts is not a line in a file. It is a faint tilt spread across billions of numbers that also do a thousand other jobs. The model can be examined, but not read.

One loop, repeated. What turns it into something that can write a passable cover letter is not a cleverer loop. It is the size of what you point it at.

The loop is old. What is new is how obscenely much of it there is.

Guess-measure-nudge dates to the 1980s. It was understood, published, and mostly disappointing for thirty years. What changed was not the idea but its scale, meaning simply how much of everything is thrown at it, and the change is hard to feel without numbers.

Our illustrative model reads four trillion tokens during training. That is roughly three billion pages: more than any person could read in ten thousand lifetimes, and a real fraction of the readable public internet plus a great many books.

It carries twelve billion parameters, each one nudged on essentially every example. And the training run consumes a quantity of compute, which is the total number of arithmetic operations performed, that takes thousands of specialised chips several weeks and costs millions of pounds in electricity and hardware.

This first, enormous, general phase is called pretraining. Its only goal is the next-token guess on ordinary text. Nobody is teaching it to be helpful yet, and it is worth being clear that what comes out of pretraining is not a chat assistant. It is a machine that continues text.

The pattern in that figure is the whole reason this technology arrived when it did. At small scale the output is grammatical nonsense. At medium scale it is locally sensible and globally incoherent, sentences fine, paragraph adrift. Somewhere past a threshold it becomes something you would not immediately identify as machine-made.

Nothing in the method changed between those points. The loop is identical. Abilities that were absent simply appear as the scale rises, which is why the field spent several years mostly building bigger things rather than cleverer ones.

Still, everything so far produces a text continuer, not an assistant. Feed our pretrained model a question and the honest thing for it to do is carry on in the same vein, which often means producing more questions.

A text continuer is not an assistant, and the gap is deliberately closed

This step is missing from most explanations, and its absence makes the whole thing seem more mysterious than it is. Between "predicts the next token" and "answers your question helpfully" there is a distinct phase, and it was invented because the raw thing was disappointing to use.

Give a purely pretrained model the input What is a lighthouse for? and the statistically natural continuation is not an answer. Text on the internet containing that sentence is often a list of quiz questions, or a forum post, or a worksheet. So it continues in kind: more questions, a heading, a page of exam paper. It is not malfunctioning. It is doing precisely what it was trained to do.

Fine-tuning is further training on a much smaller, deliberately chosen set of examples, which shifts the model's behaviour without rebuilding it. Where pretraining is trillions of tokens of whatever there was, fine-tuning might be tens of thousands of carefully written examples.

The first kind is instruction tuning: thousands of examples of a request followed by a good response to it. This alone transforms the experience, because it makes "a helpful answer" the statistically natural continuation of "a question".

The second kind uses human feedback. People are shown two of the model's responses to the same prompt and asked which is better. Those judgements train a second model to predict human preference, and that preference model is then used to nudge the first one toward responses people prefer. Bending a model toward what people actually want is generally called alignment.

Two things follow that are worth carrying around.

The model's personality, its willingness to help, its refusals and its habitual tone are not emergent properties of reading the internet. They were installed in this phase, by choices people made about which responses were better. Different choices produce a noticeably different assistant from the identical pretrained model.

And the phase is small and cheap relative to pretraining, often well under a percent of the total compute. Nearly all the capability comes from the enormous general phase; nearly all the behaviour you actually experience comes from the tiny one on the end.

It can only see what is in front of it, and that space has an edge

Everything the model considers when producing a token is inside its context. That space has a hard maximum, measured in tokens, called the context window. Ours holds 128,000 tokens, roughly three hundred pages.

Two facts about that window explain a great deal of everyday behaviour.

The first is statelessness: between one message and the next, the model remembers nothing. Nothing at all. It has no store of your conversation. What actually happens is that the entire conversation so far is re-sent with every message, so that the appearance of memory is produced by resending the past, not by retaining it.

The second follows immediately. Since the whole conversation is re-sent each time, a long conversation eats the window. Once the total exceeds 128,000 tokens, something has to go, and the usual something is the oldest part. From your side this looks like the assistant forgetting what you agreed at the start, and that is exactly what it is: those tokens are no longer in front of it.

This also disposes of a common worry and confirms another. The model is not learning from your conversation: nothing you type changes a single parameter. But everything you paste is in front of it for the rest of that conversation, and is sent again with every message you write.

That covers text end to end. But the same phrase, generative AI, also covers the systems that make pictures, and those work by a genuinely different mechanism. Accounts that say images work "in a similar way" are, unfortunately, wrong.

Pictures are not written left to right. They are uncovered from static.

An image model does not produce a picture pixel by pixel in reading order, and it does not predict the next pixel from the previous ones. It uses a method called diffusion, and the idea is odd enough to be worth taking slowly, because it is genuinely elegant.

Start with training, which runs backwards from what you would expect. Take a real photograph and add a little noise, meaning random speckle of the sort an old television showed with no signal. Add a little more. Keep going for a few hundred rounds until the image is indistinguishable from pure static.

Now train a model to undo one single step of that: given a slightly noisy image, predict what the slightly less noisy version looked like. This is an ordinary guess-measure-nudge problem, and the correct answer is known for free, because you added the noise yourself and kept the original.

Once that model works, run it in reverse to generate. Begin with a rectangle of pure random static, which contains no picture and never did. Ask the model to make it slightly less noisy. Then again. And again, perhaps fifty times. Denoising static that never held an image produces one, because at each step the model pushes the pixels toward what a plausible image would look like.

Your prompt steers it. The text is turned into numbers by the same kind of process from earlier, and those numbers are fed into every denoising step, so that at each stage the model is not asked for a plausible image but for a plausible image of a lighthouse at dusk.

Three things fall out of this that people ask about constantly.

It explains why image generation has a visible number of steps and why stopping early gives you something blurry and half-formed rather than a partial picture. It explains why the same prompt gives different images: the starting static is random, so a different starting point resolves somewhere else. And it explains why the whole image arrives at once, unlike text arriving word by word, since every part of the picture is being refined simultaneously.

Two mechanisms, then, under one name. Both learned from examples, both producing rather than choosing. And both with a specific, structural failure that follows from how they work.

It invents things because plausible and true are different targets

A hallucination is a confident, fluent, entirely fabricated statement: a citation to a paper that does not exist, a quoted price that was never charged, a plausible date for an event that happened elsewhere. It is the most-discussed weakness of these systems and it is usually explained as a bug or a gap in the training data. It is neither, and understanding why makes it far easier to work with.

Go back to the loss. The model was rewarded, trillions of times, for one thing: assigning high probability to whatever token actually came next in real text. Nothing in that objective mentions truth. Truth was never measured, never scored and never nudged toward. What was measured was whether the continuation looked like the sort of thing that follows.

Now consider a request for a source on lighthouse construction. Real text contains millions of citations, and they have a shape: an author, a year, a title with certain kinds of words, a journal name. The model has learned that shape extremely well. Producing something with exactly that shape is what its training optimised for. Whether the paper exists is a fact about the world, and the model has no mechanism that checks facts about the world at generation time.

So the output is not a lie, and it is not a malfunction. It is a very good answer to the question the machine was actually trained on, which was never the question you were asking.

This also explains the two most useful practical rules. Hallucination is worst for specifics that are highly patterned but individually arbitrary, which is precisely what citations, statistics, dates, prices and legal references are. And a model's confident tone carries no information about correctness, because fluency was the objective and confidence is just more fluency.

Hallucination gets the attention. Several other limits matter as much or more, and a couple of them are about the world rather than the machine.

Six limits, and only two of them are about the machine being wrong

Each of these follows from something already established, rather than being a caveat bolted on at the end.

Bias. The model absorbed the statistical tendencies of its training data, including the ones nobody wanted. If nurses are overwhelmingly described as women in the text it read, that association is now a tilt in the parameters, and it will show up in what it generates. This is not the model having views. It is the data's tendencies reproduced faithfully, which is the one thing the training loop is guaranteed to do.

The knowledge cutoff. Training finished on a date, and the parameters have not changed since. A model's grasp of the world is frozen at that moment, so it may confidently describe as current something long since replaced. Some systems patch this by searching and putting results into the context window, which is a genuinely different mechanism from knowing: the fact is in front of it, not in it.

Provenance. The training data came from somewhere, largely from the public internet, largely without the authors being asked. Whether that constitutes fair use is under active litigation in several countries, and reasonable people disagree sharply. It is not settled, and anyone telling you it obviously is, in either direction, is telling you their position rather than the law.

Cost and energy. Training runs consume power on an industrial scale, and every individual response costs a small amount of electricity too. Multiplied across hundreds of millions of daily users, running these systems is a meaningful and rising share of data-centre demand.

Synthetic media. The same capability produces convincing fake photographs, cloned voices and fabricated video. This is not a misuse of a flawed system; it is the intended capability pointed somewhere harmful, which makes it much harder to engineer away than a bug.

It does not understand in the way you do. Worth stating carefully, because both overclaiming and dismissal are common. The model has no body, no continuous experience, no stake in anything and no way to check a claim against the world. It also demonstrably does something more than recite, since it handles genuinely novel combinations. What to call that is contested, and the honest position is that we do not have good language for it yet.

Every number on this page describes one illustrative model: 12 billion parameters, 4 trillion training tokens, a 100,000-token vocabulary, 4,096-number embeddings and a 128,000-token context window. These are plausible mid-range figures chosen so the page can compute rather than assert. They are not the specifications of any particular product, and real systems vary by more than an order of magnitude in every one of them.

The whole journey, from a pile of text to the words on your screen

Every piece is now in place. Here is the entire path in order, with the panel beside these steps filling in as you go.

Notice where the line falls. Everything above inference happened once, months ago, at enormous cost, and is identical for everybody. Everything below happens in a second, costs a fraction of a penny, and is yours alone. People often picture the model consulting or learning while it answers. It is doing neither. It is a fixed set of numbers being multiplied.

A guess, weighted, rolled, and appended

There is no list of answers, no retrieval, and nothing being consulted. There is a machine corrected trillions of times until its guess about what comes next stopped being wrong so often, and a die rolled against that guess, once per word.

Now build a request and watch it behave

Everything adjustable at once: what you ask for, how the die is weighted, and how much room the conversation has. Three presets stand for real situations where the right settings genuinely differ.