runs once, front to back
and then, only sometimes
Both watch the room. Both decide something. Both act. The word that separates them is not smart, and it is not modern — it is whether the thing ever finds out it was wrong. Run the two halves above and the whole distinction is already visible.
Picture an old wall thermostat. The room hits 26°. You set it to 24°. It switches on the air conditioning. It watched the world, applied a rule, and did something about it.
Now picture a Nest on the same wall. It also watches, applies a rule, and switches the air conditioning on. From across the room, on any single afternoon, you could not tell them apart.
Here is the question that separates them, and it has nothing to do with how clever either one looks. Press the button and skip forward.
On day ten thousand, in a different house, in a different season, the thermostat does exactly what it did on day one. It has never once been wrong in a way it could notice, because it never made a guess and never checked one.
Any system that does something useful runs three steps, in order, once per question. Beginners skip past them and get confused later, so it is worth slowing down.
The first is observe — something comes in from the world. Notice how astonishingly narrow this always is. Pick a system and look at what it genuinely has, next to what it does not.
Whatever is not observed simply does not exist as far as the system is concerned. A surprising share of real-world failures trace back to exactly this line: the thing that actually mattered was never in the input.
Then comes analyze — the observation gets pushed through some internal rule. In a thermostat a human wrote that rule in plain words. In machine learning nobody wrote it; it was worked out from thousands of past examples, and it exists as a very large pile of numbers no one can read as a sentence.
And then infer. The word is deliberate. It does not mean know, or look up, or calculate. It means: reach the most reasonable conclusion available from incomplete evidence. A doctor looking at a fever infers an infection. She does not know it.
Nothing here is ever certain. Every extra clue reshuffles the ranking, and the top line is only ever the best available guess — which is exactly why these systems are sometimes confidently wrong. Being confidently wrong is not a bug in inference; it is the nature of inference. People do it constantly too.
Below the line the activity changes kind entirely. It does not produce an answer for anybody. It produces a better system.
The system commits to a guess before reality reveals the answer, in a form that can later be scored. Reality arrives and differs from the guess. Then the internal rule is adjusted so the gap would have been smaller — not a rewrite, a nudge. Thousands of small nudges is what training actually is.
Here it is with real numbers, and you can switch it off.
Ramesh must decide each morning how much milk to buy. Too little and he turns customers away; too much and it curdles by evening. He starts believing rain lifts demand by about 20%. It actually lifts it by far more.
The only thing that changes across days is the rule in Ramesh’s head, and it changes because he compared his guess against what actually happened. Switch off the counting and the gap stops shrinking — same stall, same tea, same effort, forever.
And the failure mode is just as plain. With no error signal he will make the same mistake for twenty years. No feedback, no learning — however much data you have. A great many stalled projects fail here and nowhere else: they observe constantly and reconcile never.
You are salting a pot of dal. You taste it, compare that against a memory of how it should taste, decide it needs half a teaspoon more, add it, and taste again. Still flat — your instinct was calibrated for a smaller pot. Adjust.
After twenty years of cooking, you stop tasting. You look at the pot and you know. Thousands of feedback cycles have compressed into an instinct that fires instantly and is almost always right.
Those are the same activity in two different phases, and the distinction matters more than any other on this page — because a chatbot is permanently stuck in the second one.
The tasting-and-adjusting phase is training. The fast, accurate, no-deliberation end state is a trained model in production. Hold on to this; it is what makes the chatbot section later make sense.
A batsman faces a fast bowler. He observes the run-up, the arm at the moment of release, the angle of the seam, the shine on the ball. He analyses it in roughly four hundred milliseconds, below conscious thought. He infers — good length, outside off, coming in — and plays the cover drive.
The shot itself is the prediction: I believe the ball will be here when my bat arrives. Step through it, and then watch what happens when he is wrong.
A great batsman is not someone with faster reflexes. Reflex speed is roughly the same across professionals. A great batsman is someone whose feedback loop has run more times, and more honestly.
Talent is a good starting model. Greatness is a good feedback loop.
A maps app tells you thirty-four minutes. It observed your start point, destination, time of day, day of week, live speed data from every other phone on that road, historic patterns, reported accidents. The trained model analysed it. The inference was “34 minutes”.
That number is a public, testable prediction. Take the trip.
You actually arrive in forty-one minutes. That seven-minute gap is the single most valuable thing in the whole interaction, and it is captured automatically, because your phone knows exactly when you arrived. Nobody had to fill in a form. Multiply by a billion journeys a day.
A sales rep gets a hundred inbound leads in a quarter. She observes company size, industry, job title, how the call went. Her internal rule is gut feel — this one sounded really enthusiastic. She marks thirty as hot and expects roughly half of them to close.
Six close. Predicted fifteen, actual six: an error of nine. Worse, the rule was actively harmful — three months of effort went to the wrong twenty-four people.
So she goes back through the data looking for what actually separated the six winners. Test each candidate signal yourself.
Enthusiasm turns out to be worthless — everyone is enthusiastic on a free call. But five of the six that closed had booked a demo within three days, and of the twenty-four hot leads that did not close, only three had. That behaviour, not the sentiment, is the signal.
Next quarter she rescores using it: twenty marked hot, twelve predicted, eleven actually close. Error of one, down from nine.
Notice too that the good rule was discovered from outcomes, not designed in a meeting.
Your card is declined at a restaurant. A large amount, a jewellery merchant, 11:47pm, a city you have never transacted in, card-not-present. The trained model infers a 94% chance of fraud, predicts, and blocks. You reply YES, IT WAS ME. The model was wrong, you are furious, and your dinner is ruined.
Your reply is a perfectly labelled training example, delivered free, within seconds. So the bank retrains. Here is what that actually bought — and cost.
Both columns are measured settings of the same system, not points on a dial — and there is no configuration that improves both at once. That is what a trade-off means.
The second model is better on customer experience and worse at catching fraud. Which one you ship is not a technical question. It is a business decision about the relative cost of an angry customer versus a fraudulent transaction, and it belongs to the people who own that cost, not to the data science team.
Watch a toddler learn what a dog is. She sees a Labrador in a park. Four legs, fur, tail, wet nose, makes a noise. An adult says “dog” and she files it away. Then the corrections start arriving.
By five she is essentially perfect at identifying dogs — including breeds she has never seen, from behind, in bad light, half hidden behind a car. And she cannot tell you the rule. Ask a five-year-old to define “dog” and you get nonsense.
The knowledge is real. It is reliable. It works on cases never encountered before. And it is not expressible in words. That is exactly what a trained model is: not a rulebook, but a capability learned from corrected examples, sitting in a form nobody can read back as a sentence.
This is why you cannot audit one of these systems by reading it. You audit it by testing it, on cases where you already know the right answer. It is also why “why did it reject my application?” has no easy answer, and why a model can be confidently and systematically wrong in a way that stays invisible until somebody deliberately looks.
You now have everything you need. There is one test: does it have the bottom half? Not “is it clever”, not “is it modern”, not “does it react to the world”. Does it ever find out it was wrong, and change?
If the traffic light and the chess computer caught you out, they were meant to. A system can respond to live sensor data, second by second, and still be pure automation — reacting is not learning. And a chess program from the 1990s was genuinely artificial intelligence while containing no learning whatsoever, which is the cleanest proof that these two words are not synonyms.
This is the part that changes how people use these tools, and almost every beginner misunderstanding traces back to getting it wrong. The bottom half already happened, once, and it is over. It ran across an enormous quantity of text, took months, cost a great deal of money, and then stopped. The result was frozen. That frozen result is the model.
Switch the board between training and chatting.
No prediction is being scored. No reconciliation is happening. Nothing is being recalibrated. The bottom half is switched off entirely while you chat — and everything that confuses people follows directly from that one fact.
| What people notice | Why |
|---|---|
| “I corrected it, and it apologised — but made the same mistake in a new conversation.” | Your correction changed the input for that conversation, not the model. Nothing was learned. |
| “It seems to remember what I said earlier.” | The whole conversation is re-fed as input every single turn. It is re-observing, not remembering. |
| “It doesn’t know about last month’s news.” | Learning stopped when training stopped. |
| “It said something false, with total confidence.” | Inference means best plausible answer from incomplete evidence. It is doing exactly what it was built to do. |
Picture someone who studied intensively for two years and has now walked into a closed-book exam. During the exam they cannot learn anything new. They can only apply what is already in their head, and where they are unsure they write the most plausible thing they can construct. That is a large language model answering you — and it is the dal cook in production, not the dal cook learning.
Which makes the two ways of working around frozen weights obvious. You can slide them a reference sheet during the exam — hand the model the right documents as part of the input. That changes nothing about the model, only what it observes. Or you can send them back for a short refresher course — re-run a small slice of the bottom half on your own examples. That one does change the model.
A company builds a résumé screener. It observes ten years of past résumés, each labelled with whether that person was hired. It finds the patterns that separated hired from not-hired, scores each new résumé, and retrains as new hiring decisions come in. Mechanically, the loop is working perfectly.
A well-known attempt along exactly these lines was abandoned after it was found to be downgrading résumés containing the word women’s — as in “women’s chess club captain” — and penalising graduates of two all-women colleges. The system was not broken. It had been shown ten years of tech hiring, and it faithfully learned the pattern that was actually in that data.
Then the sting: this system also runs a feedback loop on the world. Its decisions shape next year’s training data. Run it forward.
That is why the last line of the board says objective, and why choosing it is the most consequential decision anyone makes here.
One inference is not intelligence. A thousand inferences are not intelligence either. Intelligence is many inferences, integrated, and pointed at an objective.
A self-driving car makes thousands of separate inferences every second. Each one is a complete six-step system of its own, expert at exactly one narrow job, and none of them knows the others exist.
Each specialist is individually excellent and individually useless. Not one of them is driving.
When the reports point in different directions, nothing in the list resolves it. What resolves it is the objective — and specifically the fact that “safely” outranks “on time”. The car lifts off the accelerator and covers the brake.
That arbitration is the intelligence. Not the sensing, not the recognising, not the braking calculation.
There is a second lesson hiding in the chain, and it explains why this took a decade longer than everyone predicted. A chain of inferences is only as strong as its weakest link — you multiply, you do not add. If each of the five is 99% reliable, the combination is roughly 95%: one failure in twenty. In a car at speed, one in twenty is a catastrophe. The individual pieces got good quickly; integrating them to the standard the objective required did not.
This is also the cleanest description of an AI agent you will get: several models, each expert at one narrow inference, none aware of the others, plus a layer that holds the objective and decides which inference wins when they disagree. Swap the five capabilities and you have a business system rather than a car — classify the complaint, pull the customer’s history, judge how annoyed they are, draft a reply, decide whether a human must approve it, all pointed at “resolve it in one contact without giving away more than policy allows”. Same shape. Same arbitration problem.
One last pass over the board. Pick a word and watch which parts of it light up.
Two relationships people routinely get backwards. All machine learning is AI, but not all AI is machine learning — those 1990s chess programs were AI and contained no learning at all. And machine learning is how a system gets good at one inference; AI is many inferences pointed at an objective. That is the whole distinction, and it is sitting on the board: the six steps are machine learning, the closing line is AI.
It does not tell you whether any of this is thinking. Observing, processing, inferring and correcting against feedback is all mechanism. Whether that adds up to thinking is a genuinely open question and this page does not resolve it. What you can say confidently is narrower: it is not doing what you do when you think, and it does not need to in order to be extremely useful — or extremely wrong.
And the board flatters the field. Six tidy steps make training sound orderly. In practice most of the work sits upstream of the whole diagram — finding data with the answers attached, discovering that nobody recorded what happened next, arguing about which objective to aim at. The loop is the easy part to draw and the hard part to feed.