0%
temp °C
weekend

known
columns

How data becomes an equation

Supervised learning, clustering and reinforcement learning, worked by hand on eight days of a lemonade stand, then run live in your browser.

Figure 1. Eight days of trading. Click any row to hide its answer, and the fitted rule guesses it back.

Three columns on a napkin already contain a rule

Here are three days from a lemonade stand. Twenty two degrees, a Tuesday, fifty cups. Thirty degrees, a Saturday, one hundred and thirteen cups. Thirty three degrees, a Sunday, one hundred and nineteen cups.

Now answer this before reading on. Tomorrow is forecast at twenty six degrees and it falls on a Saturday. How many cups?

Whatever number you said, you did not pull it out of the air. You noticed that hotter days sell more, that weekends sell more, and that both effects stack. You did arithmetic on a pattern you extracted from three rows. Anything near a hundred is defensible, and the model trained further down this page settles on ninety four.

picture itEach day is an index card. Two facts are printed on the front, the columns you get to see, and one number is written on the back, the answer you want. A stack of cards with both sides filled in is the entire raw material of supervised learning.

The two front facts have a name worth learning now, because it appears everywhere: they are the features. The number on the back is the label. One card is a row. That is the whole vocabulary for the next few screens.

Figure 2. Click a row on the left to drop that day onto the grid on the right. Two are placed already.

Drop all eight on and the shape gives itself away. The points climb left to right, and the weekend days sit as a second band above the weekday ones, roughly parallel. Two effects, stacked, exactly as your gut said.

Seeing the band is not the same as being able to state it. A guess of "ninety-ish" cannot be checked, cannot be argued with, and cannot be handed to a machine that has never smelled a lemon.

A rule you can draw is a rule you can write down

Take a pencil to the grid and lay a straight edge through the middle of the points. You have just committed to a rule. Every straight line through that space says the same kind of thing: start from a base number of cups, then add so many cups for every degree above mild.

Written out, with mild fixed at twenty six degrees:

cups = w × (temp − 26) + b

Two numbers decide everything. The slope w is how many extra cups one degree buys. The intercept b is what a mild day sells on its own. Change the pair and you have a different rule; there are no other choices to make.

picture itA ruler pinned to the paper by two thumbtacks. One tack sets its height, the other sets its tilt. Fitting a model is nothing more exotic than sliding those two tacks until the ruler runs through the crowd of points.

Slide them yourself. The red sticks are the misses: for each day, the gap between the cups the line claims and the cups actually sold.

Figure 3. Drag either slider, or click anywhere on the grid to pull the line through that point. The bar under the plot is the average squared miss.

One number is doing the scoring: square every miss, take the mean. Squaring matters. It makes misses in both directions count the same, and it punishes one catastrophic day far more than five slightly-off days. That number has a name, the loss, and shrinking it is the only goal the machine is ever given.

The button labelled smallest error does not search. It solves a two-unknown equation and lands on w near 4.3, b near 79. That shortcut exists for straight lines and almost nothing else, which is why the rest of machine learning looks the way it does.

The line still ignores the weekend column entirely, so every Saturday sits stubbornly above it and every Tuesday below. Watch the red sticks: they are not random. Structured leftovers are the signature of a column you have not used yet.

So there are two numbers to find, a score that says how bad the current pair is, and no formula to reach for once the model gets any more complicated than a ruler. Something has to walk the tacks into place.

The misses point at their own fix

Stand at the bad line and look at the sticks. If nearly every point sits above the line, the line is too low, and you know to lift it without being told. If the points on the right sit above while the points on the left sit below, the line is too flat, and you know to tilt it.

That is the whole training algorithm. The direction of the misses is the instruction.

picture itFog on a hillside. You cannot see the valley floor, but you can feel which way the ground falls under your boots, and you can step that way. Step, feel, step again. Nobody needs a map of the valley to reach the bottom of it, only the slope underfoot.

Start anywhere. The weights begin at zero, so the first line is flat and far too low. Its average squared miss is over seven thousand cups. No knowledge, no problem: this is the honest starting point.

Measure every miss. For each of the eight days, subtract what the line claims from what actually sold. Eight numbers, all large and all in the same direction, which is already a message.

Turn the misses into a direction. Multiply each miss by the day's temperature and average them, and you get how much the slope should move. Average the plain misses and you get how much the height should move. That pair of numbers is the gradient.

Take a small step. Not the whole way. Move each weight a fraction of the distance the gradient suggests, then recompute from scratch. Five steps in, the line has visibly lifted and the loss is down from 7,379 to 4,344.

Repeat until it stops paying. Two hundred steps in, the line has settled into the crowd and the loss has dropped from seven thousand to single digits. Nothing here was solved. It was walked to.

One detail decides whether that walk works: how big a step to take. Too timid and two hundred steps leaves you halfway up the hill. Too bold and each step overshoots the bottom, lands higher than it started, and the next step overshoots further. The number that sets the step size is the learning rate, and it is the one dial worth turning by hand.

The model below uses both columns at last, so it has three numbers to find rather than two: cups per degree, cups for a weekend, and the base.

cups = 0.00 × (temp − 26) + 0.00 × weekend + 0.00

Figure 4. Set the rate, then run. Below about 0.005 the walk crawls; above about 0.05 it throws the weights off the chart and the loss climbs instead of falling.

Leave it at 0.03 and run twice. The loss lands near 6.2, and the equation reads four cups per degree, about thirty cups for being a weekend, sixty four as a base. Those three numbers are the model. Not the code, not the training loop, not the eight rows: three numbers in a text box, which you could copy onto a napkin and use for the rest of the summer.

Turn the rate to its maximum and run. The loss column shoots upward, the equation fills with implausible numbers, and the lines leave the plot. That failure is not exotic, it is the ordinary way training dies, and the fix is almost always the same dial.

Every model on this page trains the same way. What changes from here is not the walking. It is what the misses are measured against, and whether anyone bothered to write the answer column at all.

A pass or fail label is the same equation with a cut-off bolted on

Swap the napkin. Now each row is a student sitting an exam: hours revised, score on the last mock, and on the back of the card a single word, passed or failed. Two columns again, but the answer is no longer a quantity. It is one of two words.

Subtraction is meaningless here. There is no gap between "passed" and "failed" to square. So the arithmetic gets one extra move: keep the same weighted sum, then squash it into a number between zero and one and read that as a probability.

Figure 5. Drag the sum. Large and positive reads as almost certainly passed, large and negative as almost certainly failed, zero as a coin toss.

picture itThe weighted sum is a see-saw. Hours revised push one end down, a weak mock score pushes the other. The squash is just the rule for reading the see-saw out loud: lean far enough one way and you call it, sit near level and you admit you do not know.

Training changes by one line. Instead of squaring the miss, the loss asks how much confidence the model put on the wrong word, and punishes confident mistakes hardest. The walk downhill is identical, and it still ends with a handful of numbers.

Those numbers draw a line, but the line means something new. It is the boundary: on one side the model says passed, on the other failed, and it sits where the probability is exactly a half.

add a pass add a fail

Figure 6. Click anywhere in the plot to add a student of the chosen kind. The boundary retrains on every click. Try adding a fail deep in pass territory.

Put one stubborn failure at nine hours and a strong mock score, and watch the boundary tilt to accommodate it. One row moved the rule for everybody. With eighteen rows that is what a single odd case is worth; with eighteen thousand it would barely register, which is most of the practical argument for collecting more data.

The equation is still readable. As it stands, one more hour of revision moves the lean by about 4.5, and ten more points on the mock by about 9.2. You can disagree with a model you can read, which is exactly why regulators keep asking for models you can read.

Both napkins so far had something written on the back of the card. Somebody sat down and marked each student, each day. Take that person away and the arithmetic has nothing to be wrong about.

With the answer column gone, distance is the only teacher left

Eighteen customers of the same lemonade stand, two columns each: visits per month, and average spend per visit. No third column. Nobody has ever labelled these people, and nobody is going to.

Look at them plotted anyway and your eye does something the previous sections needed a loss function for. It finds groups. A knot of frequent small spenders, a knot of rare big spenders, a knot in between. You did not have a target to be right about; you used nothing but which points sit near which.

picture itDrop three drawing pins on the map, give every town to its nearest pin, then slide each pin to the middle of the towns it just collected. Repeat. The pins stop moving when no town wants to change hands, and where they stop is the answer.

That loop has two lines and no derivatives. Assign every point to the nearest pin. Move every pin to the average of its points. The pins are called centroids, the groups are clusters, and the score being minimised is the total squared distance from points to their own pin. Same shape of goal as before, measured against the data itself rather than against an answer.

2 groups 3 groups 4 groups

Figure 7. Press assign twice and watch the pins jump to the middle of their catch. Press scatter to start from a different random set of pins, sometimes with a different ending.

Press scatter a few times and read the score at the bottom. Most starts settle at 236, the three obvious knots. Roughly one start in four settles near 890 instead: two pins land inside the same knot and split it, while the third swallows the other two groups whole. The algorithm has no way to notice, because from where it now stands nothing is moving any more.

Ask for four groups and it will give you four, cheerfully, by slicing a real group in half. The method cannot tell you how many groups exist. It can only tell you the best arrangement for the number you demanded, which makes the choice of that number a judgement call dressed as a setting.

Nothing here was predicted and nothing was labelled. But at least the rows arrived. In the last setting, even that stops being true.

When no rows arrive, the machine has to go and collect its own

A delivery robot in a five by five stockroom. It starts at the top left, the crate is at the bottom right, and nobody has ever shown it a route. Each move costs one point. Reaching the crate pays ten. That is the entire specification, and it says nothing whatsoever about turning right.

So the robot flails. It moves at random, bumps walls, wanders, and eventually stumbles onto the crate. That single lucky arrival is the first real information in the system, and it gets used in a specific way: the square the robot was standing on when it stepped into the crate is now known to be worth something. Next time round, the square before that one inherits a share of it.

picture itScent on a trail. The crate smells strongest, and every square that leads to a good-smelling square picks up a little of the smell itself. After enough wandering the whole floor is a gradient the robot can simply walk up, and none of it was designed by anyone.

Two quantities do the bookkeeping. The value of a square is how much reward the robot expects to collect from there onward. The policy is the arrow it would follow: whichever neighbour looks worth the most.

episodes 0 · last run 0 steps

Figure 8. Press run and watch the shading spread backwards from the crate. Click any square to move the crate there, which wipes what the robot knew.

Press run once and the readout says the arrows do not reach the crate yet: the shading is a smear in the bottom corner and the rest of the floor is still blank. Press it again, forty episodes in, and the arrows have joined up into a staircase that walks from the start to the crate in eight steps, which is the shortest route the room allows. Every press after that changes nothing except the wandering, because the answer is already found.

Notice what never existed: a table of correct moves. The robot generated every row it learned from by acting, and the only ground truth in the building was ten points for a crate.

Move the crate and the knowledge is worthless in one click. Everything the robot learned was about the old goal, and there is no partial credit. Brittleness of exactly this kind is why so much reinforcement learning stays in simulators, where a wasted lifetime costs a few seconds of compute.

Three settings, three sources of instruction: a marked answer, the shape of the data, a number arriving late and rarely. Underneath, the machinery has been the same one every time.

All three are the same loop with a different judge

Line up what actually happened in each case and the differences shrink to one slot. Start with adjustable numbers. Produce something. Get told how bad it was. Nudge the numbers. Repeat.

Only the third box changes.

supervised clustering reinforcement

Figure 9. Switch settings. The layout never changes, which is the point: only the judge and the thing being adjusted swap out.

picture itOne kitchen, three critics. A recipe card with the right answer printed on it, a room of diners seated by taste, and a food writer who eats in silence and posts a score a week later. The cooking never changes, the feedback does, and everything difficult about each setting follows from how late and how thin its feedback is.

Written as equations, the family resemblance is closer still. Supervised regression predicts a weighted sum. Supervised classification squashes the same sum. Clustering compares a point to a handful of centre points, which is a weighted sum in disguise. The robot scores each move with a stored number per square, which for a bigger stockroom would itself be predicted by, of all things, a weighted sum.

That resemblance is also the trap. Four boxes and a nudge will happily fit numbers to anything, including patterns that do not exist.

The equation only knows the days it was shown

The lemonade model was fitted on days between eighteen and thirty five degrees. Ask it about a mild spring morning at eight degrees and it answers with confidence. Ask it about a heatwave at forty five and it answers with equal confidence. It has no idea it has left the room.

And it has never met a festival. Add one, a Saturday in a heatwave with three hundred cups sold, and watch what a single unusual row does to a rule fitted on eight.

picture itA tide table written from one week of measurements at one harbour. Inside that week it is accurate to the minute. Carry it to a different coast and it will still print times, in the same confident font, and every one of them will be wrong.

Figure 10. Drag the prediction point outside the shaded band and the model keeps answering. Add the festival and the whole line pivots under it.

Two failures, both ordinary. Outside the shaded band the model is extrapolating, and a straight line extended far enough will predict negative cups on a cold morning. Inside the band, one extreme row drags the fit toward itself because squared error punishes large misses so hard that the model would rather be slightly wrong about seven days than badly wrong about one.

Neither failure shows up in the loss. The number on screen stays small and keeps falling while the rule quietly becomes useless outside its band. Every serious pipeline holds back rows the model never trains on, precisely because a loss computed on the training rows cannot see this.

The same holds for the other two. Clustering will hand you groups in data that has none. The robot will learn a perfect route to a crate that has moved.

Three settings, one habit: guess with numbers, measure how wrong the guess was, nudge the numbers, and keep the ones that survive contact with days you have not seen.