Classification, regression and deep learning are the same four motions

One commuter logged twenty-two mornings: what time they left, whether it was raining, how long the trip took. Three different models are about to argue over that table. Each one trains, gets tested on mornings it has never seen, gets picked apart miss by miss, and gets scored. The output changes. The loop does not.

22 mornings · 2 features · 20 numbers · every figure computes its own arithmetic

how wrong the network is about each morning, right nowstarting up
nothing here is pre-recorded: twenty numbers are being adjusted in your browser, played slowly at first so you can watch the shape arrive

Guess how long tomorrow’s trip takes

Tomorrow this person leaves at 08:00. It is not raining. How many minutes will the trip take?

Do not skip this. Move the marker to a number and press the button. You need to have committed to something before the next paragraph makes any sense.

your guess against what actually happenedplace your guess
Drag, then reveal. The rust bar is the only thing this page is about. It does not care whether you guessed high or low.

Whatever you said, there is now a distance between your number and the recorded one. That distance is the miss. Every idea on this page is arithmetic performed on a pile of misses.

Look at what you actually did to produce your guess. You were handed two facts, the departure time and the weather, and you were asked for a third. In the language everybody uses for this: the two facts you were given are features, the answer you were asked for is the label, and one morning with both of them filled in is a row. Twenty-two rows is the whole of what any model on this page gets to know.

picture it

A market trader who has watched the same stall for a month. Ask what a crate of tomatoes goes for on a wet Tuesday and you get an answer, fast, with no arithmetic. Ask them how they did it and they cannot tell you, because the answer is not stored anywhere as a rule. It is stored as having been there.

A model is that trader, forced to write the rule down. Not smarter. Not more experienced. Just obliged to commit the guess to something you can inspect, argue with, and check.

One guess is a party trick. The moment you want an answer for a morning nobody has lived through yet, you need something written down that will produce a number for any departure time you hand it.

A rule is two numbers you are allowed to change

Here is the smallest thing that deserves to be called a model. Pick a journey time for a 07:30 departure, then add a fixed amount of minutes for every hour later than that you leave.

minutes = base + slope × (hour − 7.5)
leave at 07:30 → base
leave at 08:30 → base + slope
leave at 06:30 → base − slope

That is it. Two numbers and a promise to apply them to anybody. Numbers like these, the ones the rule owns and is allowed to change, are its parameters. Everything else is fixed by you before training starts: the shape of the rule, which features go into it, what comes out.

Both of these parameters mean something you can say out loud, which is deliberate and will not survive past the middle of this page. base is how long a 07:30 trip takes. slope is the price in minutes of every extra hour in bed.

For now the twelve mornings below are the whole log. The later shift had not started yet. Drag both parameters and watch every miss at once.

twelve mornings, one straight rule, twelve missestotal miss: 0.0 min
Two knobs, one number to make small. The readout is the average miss across all twelve mornings. See if you can get it under 4 minutes by hand before you press the button.

Something worth noticing while you were dragging: nobody told you which way to move. You moved a knob, the total went up or down, you kept the direction that helped. Reduce that habit to arithmetic and you have the whole of training.

picture it

Tuning an old radio with two dials, eyes shut, with somebody in the room telling you only whether the hiss got louder or quieter. You never need to know where the station is. You only need to know which way is better, and to keep going that way until it stops helping.

That is genuinely all the information a training run has. Not the answer. Just which way is downhill from here.

The button lands on a base of 47.9 minutes and a slope of 19.2, leaving an average miss of 2.97 minutes. Both numbers say something believable: a 07:30 trip runs about forty-eight minutes, and every hour later you leave costs you another nineteen.

For a rule this small the arithmetic can jump straight to those two numbers in one shot, no searching required. That stops being possible almost immediately. Anything with more than a couple of parameters has to walk.

Training is one nudge, repeated until the total stops falling

Set both parameters badly on purpose. A base of 32 minutes, a slope of 3. The rule now claims that leaving an hour later costs you three minutes, which is nonsense, and its average miss is 13.6 minutes.

What follows is the entire loop, at the speed of one morning at a time. Scroll it.

one nudge, in fullbase 32.0 · slope 3.0
Click any step to jump to it. The two cells at the bottom are the parameters. Everything else on the page is built out of cells like those.

Two numbers, both wrong

Twelve mornings sit where they sit. The rule is the line, and the line is just those two numbers drawn.

Pick one morning and apply the rule

The 07:45 departure. Feed 7.75 in: 32 + 3 × 0.25 gives 32.8 minutes. That morning actually took 49.

Measure the miss

32.8 minus 49 is 16.2 minutes short. One row, one number. The sign matters here even though most scores throw it away later: short means the rule needs to come up.

Nudge both numbers a little

Push base up by 0.65 and slope up by 0.16. The slope moves less because this morning is close to 07:30, so the slope had almost nothing to do with the miss. Blame lands where blame belongs.

Now do that two hundred times

Every morning, over and over, each one tugging the two numbers a little. The ghost lines are where the rule used to be. It settles at base 47.9, slope 19.2 and stops moving.

the same run, under your controlstep 0 · average miss 13.62 min
Press one nudge repeatedly, then run 200. Push nudge size past about 50 and the run throws itself past the answer instead of settling on it.

Three words for what you just watched, because they are the words everybody else uses. One pass of pick-a-row, measure, nudge is a step. The single number being made small, the total miss over every row, is the loss. How far each nudge moves the parameters is the learning rate, and it is the one setting on this page that you choose rather than train.

picture it

Walking downhill in fog thick enough that you can see your boots and nothing else. You cannot see the valley. You can feel which way the ground tilts, so you take a step that way, and then you feel again.

Stride length is the learning rate. Tiny steps and you are out there all night. Enormous steps and you cross the valley floor on every stride, land on the far slope, and bounce back and forth across the bottom without ever standing in it. Nothing about the walk requires knowing where the valley is.

Training is not a search for the answer. It is repeatedly asking which way is better and going a short distance that way.

The rule now has an average miss of 2.97 minutes across the twelve mornings it was trained on. That number is worth almost nothing, and the next figure is about why.

A score on rows it has already seen is the flattering one

Every miss in the figure above came from a morning the rule was allowed to study. It saw the answer, adjusted itself, and was then congratulated for being close. That is not a measurement of anything.

The fix is to hide some rows before training starts and never let them into the loop. Those rows are held back. They are the only mornings the rule can be honestly quizzed on, because they are the only ones it cannot have absorbed.

Click mornings to hold them back. The rule refits instantly on whatever is left, and the two scores appear side by side.

you choose what it is allowed to studystudying all 12 · nothing held back
Click a morning to hollow it out. Hollow means held back: it is scored, never studied. The line only ever touches the filled ones.

Now the honest part, and it is not the tidy result you were probably expecting. There are 495 ways to hold four of these twelve mornings back. Work through all of them and the held-back score is the worse of the two in 363, which is about three times in four. The typical gap is three quarters of a minute.

So on this table, holding rows back barely catches anything. That is not a reason to stop doing it. It is a fact about this particular pairing of a very small rule with a very small log: two parameters cannot memorise twelve mornings even if they want to, so there is not much memorisation to catch.

picture it

Handing a student the exam paper a week early and then being delighted by their marks. What you measured was your own generosity.

But if the paper has two questions and the student has to answer them with one sentence they wrote in advance, seeing the paper early does not help them much. The trap only bites when the model has enough room to store answers, and by the end of this page there is a model with twenty numbers and four rows, which has room for nothing else.

Hold that thought. The two scores you have been reading are both averages, and an average is a decision about what to ignore.

Evaluating is looking at all the misses; scoring is throwing them away

Here are the twelve misses the trained rule leaves, drawn as a comb rather than collapsed into one number. Click one to see the arithmetic behind it.

every miss the rule leaves, one per morningclick a tooth
Click a tooth for its arithmetic. Switch to the wet mornings and the whole comb sits on one side of the line, which is the rule quietly telling you about a feature it was never given.

Nothing in that comb is random. All four wet mornings sit above the line and the rule is short on every one of them, by 2.7 to 5.5 minutes. The rule has no idea rain exists. It cannot, because nobody put rain into it. So the information turns up as a pattern in the leftovers, which is where information you failed to supply always turns up.

Reading the leftovers like that is evaluating. Squashing the comb into one number is scoring, and scoring is what you do when you have two rules and have to pick one.

The trouble is that there is more than one way to squash. Take the average of the misses and you get one answer. Take the worst single miss and you get another. Those are different questions, and they crown different rules.

three ways to score, three different winnersscoring by: average miss
Switch the scoring rule and the winning line moves. Then add the freak morning and watch the three choices fly apart.

On the twelve ordinary mornings the disagreement is small: the line that minimises the worst miss gets it down from 5.4 to 5.0 minutes, and pays for that by letting the average rise from 2.95 to 3.33. Half a minute either way. Twelve mornings that nearly lie on a line do not give three scoring rules much to argue about.

Now add the morning the road was closed, a 07:00 departure that took 78 minutes. The line chosen to protect the worst case swings up to meet it, and its average miss goes to 17.8 minutes against the 5.7 of the line that ignores the freak entirely. One extra row, and the three scores are no longer nearly the same opinion. They are three different models.

picture it

Two ways to fine a late train. A euro a minute, so a ten minute delay costs ten. Or a euro per minute squared, so the same delay costs a hundred.

Neither is the correct fine. They are different opinions about whether one disaster is worse than several annoyances, and the choice belongs to whoever is running the railway, not to the arithmetic.

A score is not a measurement of the model. It is a measurement of the model against something you decided to care about.

That is all four motions: train, hold rows back and test, evaluate the leftovers, score. So far the answer coming out has always been a quantity. Change the answer to a choice and every one of those four motions has to be rebuilt.

When the answer is a choice, the model still hands you a number

New question about the same twenty-two mornings. Not how long the trip took. Just: did this person get in after 09:00? Yes or no.

Twelve of the twenty-two were late. A yes-or-no answer like that is a class, and a model that picks between classes is doing classification. The word covers spam or not spam, tumour or not tumour, this-of-nine-digits, fraud or fine.

Here is the part that surprises people, and it is the single most useful thing to know about classification. The model does not output a class. It cannot. Nudging parameters requires a miss you can measure, and there is no such thing as being a little bit wrong about the word yes. So the model outputs a number between 0 and 1, a probability, and something else turns that into a decision afterwards.

That something else is a single number you choose: the threshold. Drag it.

a probability for each morning, and a line you place across itthreshold 0.50
Drag the threshold. A cell holding L was genuinely late. Cells the threshold gets wrong turn rust. The four counts on the right are every way one morning can turn out.

Four mornings are stuck at a probability of 0.509 and there is nothing wrong with that. All four left at 08:00. The model has only been given the departure time, so as far as it can see those four mornings are the same morning. Two of them ran late. A coin flip is the honest answer, and the model says so.

Watch what the threshold buys and costs. At 0.50 the rule catches all twelve late mornings and raises two false alarms, mornings it flagged that were fine. Push it to 0.60 and the false alarms vanish, but three genuinely late mornings are now missed. Both settings score twenty and nineteen out of twenty-two, which is to say they are nearly identical if all you look at is accuracy, and completely different if you are the one catching the train.

picture it

A smoke alarm with a sensitivity dial. Turn it right up and it screams when you make toast, but it will never fail to notice a fire. Turn it down and it stops bothering you, and one night it stops bothering you about something that matters.

There is no setting that is simply correct. There is only a decision about which of the two mistakes you would rather make, and the arithmetic cannot make that decision for you because it does not know what is on the other side of the wall.

So classification runs the same four motions with two things swapped: a probability comes out instead of a quantity, and the miss is counted rather than measured. The model still trains by being nudged. It is still held back from rows it will be quizzed on. And it can still only see what you gave it, which is why four mornings collapsed into one.

Give it a second feature and the rule becomes a line across a map

The log has a column the classifier has not been shown: whether it was raining. Put it in.

Now each morning has two coordinates, departure time and wet or dry, so it has a position on a map. The rule stops being a point on a scale and becomes a line drawn across that map, with late on one side and fine on the other. It is called the decision boundary, and it is still nothing but a small set of parameters, trained by exactly the loop from earlier.

two features, one boundary, twenty-two mornings22 of 22 on the right side
click any morning to flip its answer and retrain
Click a morning to change what actually happened, then watch the boundary move to accommodate you. Drop the rain column and the boundary has to stand upright, because there is no second direction left to lean in.

With both features the boundary crosses the dry row at 08:06 and the wet row at 07:42, and it gets all twenty-two mornings right. Read the tilt: on a wet morning the model wants you out of the door twenty-four minutes earlier for the same verdict.

Drop the rain column and the line has to stand straight up, because with one feature there is only one direction. The four 08:00 mornings go back to being indistinguishable and two of them go back to being wrong. Nothing about the model changed. It just went blind in one eye.

picture it

Sorting apples on a conveyor by size alone, and being told off for letting bruised ones through. No amount of adjusting the size gate fixes it. The gate is not wrong, it is looking at the wrong thing, and the fix is a second camera rather than a better setting.

Clicking a morning to flip its answer is worth doing a few times, because it shows something the arithmetic will not say out loud. Flip one row near the boundary and the line shrugs and moves slightly. Flip one at the far end and the line barely notices. The rows that decide where the boundary sits are the ones it nearly gets wrong, and everything comfortably on the correct side contributes almost nothing.

Both models so far have been a straight thing: a straight line through a cloud, a straight boundary across a map. Straight has been enough because we have been looking at half a morning.

A straight rule cannot have a rush hour

The shift changed. Ten more mornings went into the log, all of them leaving after 08:00, and here is the whole day at once.

Before you look at the figure, predict what happens to the rule. It learned that every hour later costs nineteen minutes. It has never seen a departure after 08:00. Ask it about 10:30.

the rule from the early shift, meeting the rest of the daytrained on 12 · average miss on the new 10: 48.5 min
Switch between the three. The rule predicts 105 minutes for a 10:30 departure. That morning took 22.

It says 105 minutes. The trip took 22. It is wrong by an hour and twenty minutes, and it is wrong with total confidence, because nothing in the loop that produced it could tell it that the world stops behaving like a line somewhere past 08:00.

Now the second button, which is the one that matters. Refit a straight line on all twenty-two mornings, with nothing held back and nothing hidden. The best straight line available says: ignore the departure time entirely and predict about 40 minutes, every single morning. Its slope is minus 1.3 minutes per hour, which rounds to flat. Ignoring the departure time completely and always saying 40 would score 11.86; the best straight line manages 11.68. That is the entire value the feature adds once a line is all you have.

That is not a training failure. Training worked perfectly. It found the best straight line there is. The problem is that the shape you chose before training started cannot hold the thing you are trying to describe, and no amount of nudging fixes the shape.

picture it

Being asked to trace the outline of a hill using a metal ruler. You can slide the ruler, tilt it, put it anywhere you like. You will never get the top.

Deep learning is not a cleverer way to place the ruler. It is being handed a hinge.

The traffic has two regimes and one turning point: it gets worse until about 08:15 and then gets better for the rest of the morning. A straight rule has no way to say the word "until". So the next question is the smallest possible one. What is the cheapest thing you can add to a straight rule to let it bend once?

One unit multiplies, adds, then refuses to go below zero

Take a straight line and do one thing to it. Whenever it would go negative, make it zero instead.

a straight line   →   w × hour + b
a unit         →   the larger of ( 0 , w × hour + b )

That is the whole modification. It is called clipping at zero, the thing doing it is a unit, and the numbers w and b are its weight and its bias. Weight is the same idea as slope: how much this unit cares about the feature going in.

The clip is what buys you the corner. A straight line is straight everywhere. A clipped line is flat, then straight, with a hinge at the hour where the inside of the bracket crosses zero. One hinge is not much. Three of them, added together with their own weights, can make almost any shape you can draw without lifting the pen.

So build the rush hour by hand. Three hinges, six numbers, and a target: the best straight line on this data leaves an average miss of 11.68 minutes. Get under four.

three hinges, added up, against twenty-two real morningsaverage miss 11.68 min
Pick a hinge, then place it and set what it does. Hinge 1 should climb from the start; hinge 2 has to turn the whole thing over around 08:00, so it needs a bigger number and a negative one.

A setting that works: hinge 1 climbing 18.5 minutes an hour from 06:00, hinge 2 at 08:00 pulling back 36 an hour, hinge 3 at 09:45 adding 11.5 to flatten the tail. Average miss 2.85 minutes, against 11.68 for the best straight line. Seven numbers, one clip, four times better.

Notice what hinge 2 has to do. To turn a line that is climbing at 18.5 into one that is falling, it has to be twice as strong and pointed the other way, so the sum comes out at minus 17.5. Nothing in the model knows that. It is just what the arithmetic requires, and it is the kind of thing training discovers and nobody would guess.

picture it

A folding ruler, the wooden kind with brass hinges. Straight out it is exactly as useless as a metal ruler for tracing a hill. Break it at two places and it traces the hill.

Every hinge you add is one more corner the shape is allowed to have. That is the entire currency deep learning is buying with all those parameters: permission to bend, in more places, in more directions than you can draw.

You just did by hand what training does by nudging, on a model with one feature and one layer of hinges. Stack a second layer of hinges on top of the first and the hinges start bending each other, which is where the word deep comes from and where hand-setting stops being possible.

Twenty numbers, two hidden layers, and every single number on screen

Here is the network the rust bars at the top of this page belong to. Two features go in. Three units, then two units, then one answer. Every number it owns is visible, all twenty of them, and the whole of a prediction is four multiplications and an addition repeated six times.

The two rows of units in the middle are the hidden layers. Hidden is a dull word for a dull reason: nothing outside the network ever reads them. You put in a departure time and get back minutes. What happens in between happens.

one prediction, all the way through08:00, dry
Click a step to jump to it. Filled cells are the twenty numbers the network owns, sitting under the unit that owns them. Outlined cells are numbers it just worked out.

Two numbers go in

An 08:00 dry departure. The clock time is rescaled to sit near zero, because units work best on small numbers, and dry is 0.

The first layer: three units, each one a hinge

Each unit multiplies both inputs by its own weights, adds its bias, clips at zero. Three units, three numbers out. A unit whose bracket came out negative sends zero, which means it is contributing nothing at all to this particular morning.

The second layer works on the first layer’s answers

Same operation, but the inputs are no longer clock times. They are three numbers the first layer invented. This is the step that makes the thing deep: a hinge whose input is already hinged.

One number comes out

The last unit does not clip, because journey times are allowed to be any size. Multiply, add, done: 54.7 minutes. That morning took 55.

Now every morning at once

Run all twenty-two through and the curve appears. It climbs to a peak of 55.6 minutes at 08:03 and comes back down, and it draws a second copy of itself above the first for the wet mornings: four minutes higher at 07:00, eight at 09:00. Nobody told it that rain costs more in traffic than on an empty road. It was handed rain as one of two inputs and that fell out.

That trip through the network is a forward pass, and the intermediate numbers the layers produce are activations. Both words describe something you can now point at.

picture it

A workshop with two benches. The first bench takes the raw stock and cuts it into three rough parts. The second bench takes those three parts and assembles two sub-assemblies. A final worker screws them together.

Nobody at the second bench has seen the raw stock. They work only on what the first bench handed them, which is why the numbers in the middle of a network stop having names you can say. They are not departure times any more. They are whatever turned out to be useful.

Twenty numbers all had to be right at once for that pass to land on 54.7. Not one of them was set by a person. Which leaves the obvious question: when the answer comes out wrong, which of the twenty do you blame?

Backward: the miss is divided among the numbers that caused it

The straight rule had it easy. Two parameters, so a miss of 16 minutes short meant push both up, and how hard depended on how big the feature was.

Twenty parameters, arranged in layers, is the same idea carried out with bookkeeping. Start at the answer with the miss in your hand. Hand each unit that fed the answer a share of the blame, in proportion to how much it contributed. Those units then hand shares of their own blame back to the units that fed them. Keep going until every number has a share.

Two rules do all the work, and both are things you would guess. A number that had more influence gets more of the blame. And a unit that was clipped to zero on this morning gets none, because it did not affect the answer, so it cannot be at fault.

The trip back is the backward pass. Each number's share is its gradient: which way to move it, and how much.

who is to blame for this morning being wrongclick a number
Click any number to see its share. Rust means this number is being pushed down, teal means up, and the bar length is how hard. Grey means a clipped unit had no say in this morning.
picture it

A restaurant order comes back wrong. The manager does not shout at everybody equally. She works backwards: the plate came from the pass, so how much of this is plating, how much is the sauce station, how much is whoever ordered the wrong fish on Tuesday. Each person gets a share sized by how much of the dish went through their hands.

And the station that was closed all evening gets nothing, because it did not touch this plate. That is exactly the rule for a unit that clipped to zero. No contribution, no blame, no adjustment on this morning.

Do that for every morning, add up the shares, move every number a small distance in its own direction, and that is one step. Twenty steps in, the curve already has a peak in it. A hundred steps in, the peak is in roughly the right place at roughly the right height. The remaining two thousand four hundred steps are spent arguing about five minutes of accuracy.

The slider below is the only knob in this whole page that changes what the argument is about. Everything else has been an inspector. Drag it from left to right and you are watching the same twenty numbers at every point in their training, with three readouts moving together.

the same network at every point in its trainingstep 0 · average miss 14.22 min
Drag through the training run. The peak appears inside the first hundred steps. Everything after that is the curve inching towards the dots.

The shape of that run is not what most descriptions of training suggest. Average miss starts at 21 minutes and is down to 6.5 by step 100. Then 5.5 at step 200, 4.3 at 400, 2.5 at 700, 1.9 at 1000, 1.5 at 1500, 1.3 at 2000, and 1.26 at 2500, where it is still creeping downwards. The shape arrives almost immediately; the rest of the run is haggling. Stop at step 100 and you already have a model that knows there is a rush hour, and one that is five minutes worse than the model you get for twenty-five times the electricity.

Nothing in the loop understood traffic. Twenty numbers were pushed downhill until the shape they drew happened to be a rush hour.

Twenty numbers, two layers, an average miss of 1.26 minutes on a curve where the best straight rule could only manage 11.68. So the four motions still hold. What changed?

The same loop, three times over

Three models have now been trained on one log. Put them side by side and run a morning through each. The four stations are the same four stations in the same order. What differs is what comes out of the last one and what you are allowed to call a miss.

one morning, three models, four stations eachpick a model
Switch model, then send a morning through. The route is identical every time. Only the shape of the box in the middle and the label on the miss change.

Laid out like that, the differences are smaller than the vocabulary suggests. Regression and classification are not two kinds of learning. They are two kinds of answer, and the answer type decides how you are permitted to measure being wrong: a distance in minutes if the answer is a quantity, a count of four outcomes if the answer is a choice.

Deep learning is not a third kind of answer at all. The network in the last two figures did regression: minutes out, distance measured. Put a probability on the end instead and the same twenty numbers do classification, with a threshold on top exactly as before. What the layers buy is not a new task. It is a shape flexible enough to hold whatever is actually in the data, at the cost of every number in it becoming uninterpretable.

picture it

Three tradespeople with the same four-step method: look at the job, try something, see how far off it is, adjust. A tailor works in centimetres, an electrician in yes-there-is-continuity-or-no, a cabinetmaker in a curve that has to match a wall nobody built straight.

The method is not what makes them different. The material is.

Everything up to here has been the loop working. The last thing worth knowing is the shape of it failing, and the failure is not subtle.

Twenty numbers and four mornings is memorising, not learning

Earlier, holding rows back from the two-parameter rule caught almost nothing: the gap between the studied score and the held-back score was three quarters of a minute. Two parameters have nowhere to hide twelve answers.

Twenty parameters are a different animal. Choose how many mornings the network is allowed to study, and watch both scores at once.

how much of the log it gets to seestudying 22 · nothing held back
Start with four. The curve passes through all four almost exactly. Then look where it goes between them.

Four mornings: average miss on those four, 0.00. The curve goes through all four exactly. Average miss on the eighteen it was never shown, 8.3 minutes. Six mornings: 0.00 again, and 3.2 on the rest. Eight: 0.13 and 4.9. Twelve: 0.44 and 2.3. All twenty-two: 1.26, with nothing left to hold back.

The held-back score does not fall neatly as rows are added: six mornings scored 3.2 and eight scored 4.9. More data made it look worse. That is not an error in the arithmetic, it is what a held-back score built from fourteen rows does, and it is the same wobble that made the split in the earlier figure look pointless. A score measured on a handful of rows is itself a guess.

The studied score says the model is perfect at every one of those settings. It is a lie at every one of them, and the size of the lie is the entire reason anybody holds rows back. This has a name, overfitting, and the picture of it is the curve at four mornings: a shape that goes exactly through the four dots it was shown and does something arbitrary and confident everywhere in between.

picture it

A student who has memorised four past exam papers word for word. On those four papers they are flawless. Hand them a fifth and they answer the question they memorised, at length, with total confidence.

You cannot tell the difference between that student and one who understands the subject by looking at their marks on the four papers. You can only tell by using a paper they have not seen, which is why the held-back rows are not a formality.

Two other things this page has been quietly showing you, both worth saying out loud.

The starting numbers matter, and not a little. All twenty parameters begin at small random values, and from some starting points the network never finds the peak at all. It settles into predicting a nearly flat 40 minutes and stays there, average miss above 5 minutes, however long you leave it running. Trying sixty different starting points on this data, eighteen of them end up stuck like that: closer to three in ten than one in ten.

The reason is the one place the clip at zero bites back. A unit whose bracket comes out negative for every morning outputs zero for every morning, so it contributes nothing, so it receives no share of the blame, so it never moves again. It is dead, and nothing in the loop can revive it. The response in practice is not cleverness. It is running the whole thing from several starting points and keeping the one that worked, which is exactly what every figure here does: eight starting points, and the one with the lowest miss on the rows it was allowed to study survives.

And the network still cannot see anything you did not give it. It was handed a clock time and a yes-or-no about rain. It knows nothing about school holidays, roadworks, or the morning the road was closed. Ask it about a Sunday and it will answer, in minutes, to one decimal place, with no way of telling you that it has never seen one.

What none of this promises. Twenty-two mornings is far too small a log to trust for anything. The gaps between the studied and held-back scores here are measured on a handful of rows, so they wobble. Every number on this page is real arithmetic on real rows, but the rows are a toy, chosen so that all the arithmetic fits on a screen and you can check it. A production model does exactly what you have just watched, with millions of rows and billions of parameters, and the four motions do not change at all.

Train, test, evaluate, score. Then argue about the score.

The three words on the tin describe what comes out of the box: a quantity, a choice, or a quantity produced by something bendy enough to be worth the trouble. They do not describe three different things happening inside. Inside there is one loop, and it is a loop of being wrong on purpose, in small measured amounts, until being wrong gets boring.

The parts worth carrying away are the parts that were decisions rather than arithmetic. You chose the shape of the rule before training began, and no amount of training escapes a shape too rigid to hold the data. You chose which rows the model was allowed to study, and the score on the others is the only one that means anything. You chose what counts as a miss, and that choice crowned a different winner. The arithmetic was never in charge of any of that.

everything unlocked at onceready
Now break it. One unit per layer cannot bend at all. Four rows and eight units per layer memorises. Press different starting numbers a few times on a narrow network and watch it fail to learn about two times in five.

One thing the sandbox will not let you do is win. There is no setting that makes the held-back score match the studied one, because the held-back rows are mornings this log does not contain, and no arrangement of numbers can recover something nobody wrote down.