One commuter logged twenty-two mornings: what time they left, whether it was raining, how long the trip took. Three different models are about to argue over that table. Each one trains, gets tested on mornings it has never seen, gets picked apart miss by miss, and gets scored. The output changes. The loop does not.
Tomorrow this person leaves at 08:00. It is not raining. How many minutes will the trip take?
Do not skip this. Move the marker to a number and press the button. You need to have committed to something before the next paragraph makes any sense.
Whatever you said, there is now a distance between your number and the recorded one. That distance is the miss. Every idea on this page is arithmetic performed on a pile of misses.
Look at what you actually did to produce your guess. You were handed two facts, the departure time and the weather, and you were asked for a third. In the language everybody uses for this: the two facts you were given are features, the answer you were asked for is the label, and one morning with both of them filled in is a row. Twenty-two rows is the whole of what any model on this page gets to know.
A market trader who has watched the same stall for a month. Ask what a crate of tomatoes goes for on a wet Tuesday and you get an answer, fast, with no arithmetic. Ask them how they did it and they cannot tell you, because the answer is not stored anywhere as a rule. It is stored as having been there.
A model is that trader, forced to write the rule down. Not smarter. Not more experienced. Just obliged to commit the guess to something you can inspect, argue with, and check.
One guess is a party trick. The moment you want an answer for a morning nobody has lived through yet, you need something written down that will produce a number for any departure time you hand it.
Here is the smallest thing that deserves to be called a model. Pick a journey time for a 07:30 departure, then add a fixed amount of minutes for every hour later than that you leave.
That is it. Two numbers and a promise to apply them to anybody. Numbers like these, the ones the rule owns and is allowed to change, are its parameters. Everything else is fixed by you before training starts: the shape of the rule, which features go into it, what comes out.
Both of these parameters mean something you can say out loud, which is deliberate and will not survive past the middle of this page. base is how long a 07:30 trip takes. slope is the price in minutes of every extra hour in bed.
For now the twelve mornings below are the whole log. The later shift had not started yet. Drag both parameters and watch every miss at once.
Something worth noticing while you were dragging: nobody told you which way to move. You moved a knob, the total went up or down, you kept the direction that helped. Reduce that habit to arithmetic and you have the whole of training.
Tuning an old radio with two dials, eyes shut, with somebody in the room telling you only whether the hiss got louder or quieter. You never need to know where the station is. You only need to know which way is better, and to keep going that way until it stops helping.
That is genuinely all the information a training run has. Not the answer. Just which way is downhill from here.
The button lands on a base of 47.9 minutes and a slope of 19.2, leaving an average miss of 2.97 minutes. Both numbers say something believable: a 07:30 trip runs about forty-eight minutes, and every hour later you leave costs you another nineteen.
For a rule this small the arithmetic can jump straight to those two numbers in one shot, no searching required. That stops being possible almost immediately. Anything with more than a couple of parameters has to walk.
Set both parameters badly on purpose. A base of 32 minutes, a slope of 3. The rule now claims that leaving an hour later costs you three minutes, which is nonsense, and its average miss is 13.6 minutes.
What follows is the entire loop, at the speed of one morning at a time. Scroll it.
Twelve mornings sit where they sit. The rule is the line, and the line is just those two numbers drawn.
The 07:45 departure. Feed 7.75 in: 32 + 3 × 0.25 gives 32.8 minutes. That morning actually took 49.
32.8 minus 49 is 16.2 minutes short. One row, one number. The sign matters here even though most scores throw it away later: short means the rule needs to come up.
Push base up by 0.65 and slope up by 0.16. The slope moves less because this morning is close to 07:30, so the slope had almost nothing to do with the miss. Blame lands where blame belongs.
Every morning, over and over, each one tugging the two numbers a little. The ghost lines are where the rule used to be. It settles at base 47.9, slope 19.2 and stops moving.
Three words for what you just watched, because they are the words everybody else uses. One pass of pick-a-row, measure, nudge is a step. The single number being made small, the total miss over every row, is the loss. How far each nudge moves the parameters is the learning rate, and it is the one setting on this page that you choose rather than train.
Walking downhill in fog thick enough that you can see your boots and nothing else. You cannot see the valley. You can feel which way the ground tilts, so you take a step that way, and then you feel again.
Stride length is the learning rate. Tiny steps and you are out there all night. Enormous steps and you cross the valley floor on every stride, land on the far slope, and bounce back and forth across the bottom without ever standing in it. Nothing about the walk requires knowing where the valley is.
Training is not a search for the answer. It is repeatedly asking which way is better and going a short distance that way.
The rule now has an average miss of 2.97 minutes across the twelve mornings it was trained on. That number is worth almost nothing, and the next figure is about why.
Every miss in the figure above came from a morning the rule was allowed to study. It saw the answer, adjusted itself, and was then congratulated for being close. That is not a measurement of anything.
The fix is to hide some rows before training starts and never let them into the loop. Those rows are held back. They are the only mornings the rule can be honestly quizzed on, because they are the only ones it cannot have absorbed.
Click mornings to hold them back. The rule refits instantly on whatever is left, and the two scores appear side by side.
Now the honest part, and it is not the tidy result you were probably expecting. There are 495 ways to hold four of these twelve mornings back. Work through all of them and the held-back score is the worse of the two in 363, which is about three times in four. The typical gap is three quarters of a minute.
So on this table, holding rows back barely catches anything. That is not a reason to stop doing it. It is a fact about this particular pairing of a very small rule with a very small log: two parameters cannot memorise twelve mornings even if they want to, so there is not much memorisation to catch.
Handing a student the exam paper a week early and then being delighted by their marks. What you measured was your own generosity.
But if the paper has two questions and the student has to answer them with one sentence they wrote in advance, seeing the paper early does not help them much. The trap only bites when the model has enough room to store answers, and by the end of this page there is a model with twenty numbers and four rows, which has room for nothing else.
Hold that thought. The two scores you have been reading are both averages, and an average is a decision about what to ignore.
Here are the twelve misses the trained rule leaves, drawn as a comb rather than collapsed into one number. Click one to see the arithmetic behind it.
Nothing in that comb is random. All four wet mornings sit above the line and the rule is short on every one of them, by 2.7 to 5.5 minutes. The rule has no idea rain exists. It cannot, because nobody put rain into it. So the information turns up as a pattern in the leftovers, which is where information you failed to supply always turns up.
Reading the leftovers like that is evaluating. Squashing the comb into one number is scoring, and scoring is what you do when you have two rules and have to pick one.
The trouble is that there is more than one way to squash. Take the average of the misses and you get one answer. Take the worst single miss and you get another. Those are different questions, and they crown different rules.
On the twelve ordinary mornings the disagreement is small: the line that minimises the worst miss gets it down from 5.4 to 5.0 minutes, and pays for that by letting the average rise from 2.95 to 3.33. Half a minute either way. Twelve mornings that nearly lie on a line do not give three scoring rules much to argue about.
Now add the morning the road was closed, a 07:00 departure that took 78 minutes. The line chosen to protect the worst case swings up to meet it, and its average miss goes to 17.8 minutes against the 5.7 of the line that ignores the freak entirely. One extra row, and the three scores are no longer nearly the same opinion. They are three different models.
Two ways to fine a late train. A euro a minute, so a ten minute delay costs ten. Or a euro per minute squared, so the same delay costs a hundred.
Neither is the correct fine. They are different opinions about whether one disaster is worse than several annoyances, and the choice belongs to whoever is running the railway, not to the arithmetic.
A score is not a measurement of the model. It is a measurement of the model against something you decided to care about.
That is all four motions: train, hold rows back and test, evaluate the leftovers, score. So far the answer coming out has always been a quantity. Change the answer to a choice and every one of those four motions has to be rebuilt.
New question about the same twenty-two mornings. Not how long the trip took. Just: did this person get in after 09:00? Yes or no.
Twelve of the twenty-two were late. A yes-or-no answer like that is a class, and a model that picks between classes is doing classification. The word covers spam or not spam, tumour or not tumour, this-of-nine-digits, fraud or fine.
Here is the part that surprises people, and it is the single most useful thing to know about classification. The model does not output a class. It cannot. Nudging parameters requires a miss you can measure, and there is no such thing as being a little bit wrong about the word yes. So the model outputs a number between 0 and 1, a probability, and something else turns that into a decision afterwards.
That something else is a single number you choose: the threshold. Drag it.
L was genuinely late. Cells the threshold gets wrong turn rust. The four counts on the right are every way one morning can turn out.Four mornings are stuck at a probability of 0.509 and there is nothing wrong with that. All four left at 08:00. The model has only been given the departure time, so as far as it can see those four mornings are the same morning. Two of them ran late. A coin flip is the honest answer, and the model says so.
Watch what the threshold buys and costs. At 0.50 the rule catches all twelve late mornings and raises two false alarms, mornings it flagged that were fine. Push it to 0.60 and the false alarms vanish, but three genuinely late mornings are now missed. Both settings score twenty and nineteen out of twenty-two, which is to say they are nearly identical if all you look at is accuracy, and completely different if you are the one catching the train.
A smoke alarm with a sensitivity dial. Turn it right up and it screams when you make toast, but it will never fail to notice a fire. Turn it down and it stops bothering you, and one night it stops bothering you about something that matters.
There is no setting that is simply correct. There is only a decision about which of the two mistakes you would rather make, and the arithmetic cannot make that decision for you because it does not know what is on the other side of the wall.
So classification runs the same four motions with two things swapped: a probability comes out instead of a quantity, and the miss is counted rather than measured. The model still trains by being nudged. It is still held back from rows it will be quizzed on. And it can still only see what you gave it, which is why four mornings collapsed into one.
The log has a column the classifier has not been shown: whether it was raining. Put it in.
Now each morning has two coordinates, departure time and wet or dry, so it has a position on a map. The rule stops being a point on a scale and becomes a line drawn across that map, with late on one side and fine on the other. It is called the decision boundary, and it is still nothing but a small set of parameters, trained by exactly the loop from earlier.
With both features the boundary crosses the dry row at 08:06 and the wet row at 07:42, and it gets all twenty-two mornings right. Read the tilt: on a wet morning the model wants you out of the door twenty-four minutes earlier for the same verdict.
Drop the rain column and the line has to stand straight up, because with one feature there is only one direction. The four 08:00 mornings go back to being indistinguishable and two of them go back to being wrong. Nothing about the model changed. It just went blind in one eye.
Sorting apples on a conveyor by size alone, and being told off for letting bruised ones through. No amount of adjusting the size gate fixes it. The gate is not wrong, it is looking at the wrong thing, and the fix is a second camera rather than a better setting.
Clicking a morning to flip its answer is worth doing a few times, because it shows something the arithmetic will not say out loud. Flip one row near the boundary and the line shrugs and moves slightly. Flip one at the far end and the line barely notices. The rows that decide where the boundary sits are the ones it nearly gets wrong, and everything comfortably on the correct side contributes almost nothing.
Both models so far have been a straight thing: a straight line through a cloud, a straight boundary across a map. Straight has been enough because we have been looking at half a morning.
The shift changed. Ten more mornings went into the log, all of them leaving after 08:00, and here is the whole day at once.
Before you look at the figure, predict what happens to the rule. It learned that every hour later costs nineteen minutes. It has never seen a departure after 08:00. Ask it about 10:30.
It says 105 minutes. The trip took 22. It is wrong by an hour and twenty minutes, and it is wrong with total confidence, because nothing in the loop that produced it could tell it that the world stops behaving like a line somewhere past 08:00.
Now the second button, which is the one that matters. Refit a straight line on all twenty-two mornings, with nothing held back and nothing hidden. The best straight line available says: ignore the departure time entirely and predict about 40 minutes, every single morning. Its slope is minus 1.3 minutes per hour, which rounds to flat. Ignoring the departure time completely and always saying 40 would score 11.86; the best straight line manages 11.68. That is the entire value the feature adds once a line is all you have.
That is not a training failure. Training worked perfectly. It found the best straight line there is. The problem is that the shape you chose before training started cannot hold the thing you are trying to describe, and no amount of nudging fixes the shape.
Being asked to trace the outline of a hill using a metal ruler. You can slide the ruler, tilt it, put it anywhere you like. You will never get the top.
Deep learning is not a cleverer way to place the ruler. It is being handed a hinge.
The traffic has two regimes and one turning point: it gets worse until about 08:15 and then gets better for the rest of the morning. A straight rule has no way to say the word "until". So the next question is the smallest possible one. What is the cheapest thing you can add to a straight rule to let it bend once?
Take a straight line and do one thing to it. Whenever it would go negative, make it zero instead.
That is the whole modification. It is called clipping at zero, the thing doing it is a unit, and the numbers w and b are its weight and its bias. Weight is the same idea as slope: how much this unit cares about the feature going in.
The clip is what buys you the corner. A straight line is straight everywhere. A clipped line is flat, then straight, with a hinge at the hour where the inside of the bracket crosses zero. One hinge is not much. Three of them, added together with their own weights, can make almost any shape you can draw without lifting the pen.
So build the rush hour by hand. Three hinges, six numbers, and a target: the best straight line on this data leaves an average miss of 11.68 minutes. Get under four.
A setting that works: hinge 1 climbing 18.5 minutes an hour from 06:00, hinge 2 at 08:00 pulling back 36 an hour, hinge 3 at 09:45 adding 11.5 to flatten the tail. Average miss 2.85 minutes, against 11.68 for the best straight line. Seven numbers, one clip, four times better.
Notice what hinge 2 has to do. To turn a line that is climbing at 18.5 into one that is falling, it has to be twice as strong and pointed the other way, so the sum comes out at minus 17.5. Nothing in the model knows that. It is just what the arithmetic requires, and it is the kind of thing training discovers and nobody would guess.
A folding ruler, the wooden kind with brass hinges. Straight out it is exactly as useless as a metal ruler for tracing a hill. Break it at two places and it traces the hill.
Every hinge you add is one more corner the shape is allowed to have. That is the entire currency deep learning is buying with all those parameters: permission to bend, in more places, in more directions than you can draw.
You just did by hand what training does by nudging, on a model with one feature and one layer of hinges. Stack a second layer of hinges on top of the first and the hinges start bending each other, which is where the word deep comes from and where hand-setting stops being possible.
Here is the network the rust bars at the top of this page belong to. Two features go in. Three units, then two units, then one answer. Every number it owns is visible, all twenty of them, and the whole of a prediction is four multiplications and an addition repeated six times.
The two rows of units in the middle are the hidden layers. Hidden is a dull word for a dull reason: nothing outside the network ever reads them. You put in a departure time and get back minutes. What happens in between happens.
An 08:00 dry departure. The clock time is rescaled to sit near zero, because units work best on small numbers, and dry is 0.
Each unit multiplies both inputs by its own weights, adds its bias, clips at zero. Three units, three numbers out. A unit whose bracket came out negative sends zero, which means it is contributing nothing at all to this particular morning.
Same operation, but the inputs are no longer clock times. They are three numbers the first layer invented. This is the step that makes the thing deep: a hinge whose input is already hinged.
The last unit does not clip, because journey times are allowed to be any size. Multiply, add, done: 54.7 minutes. That morning took 55.
Run all twenty-two through and the curve appears. It climbs to a peak of 55.6 minutes at 08:03 and comes back down, and it draws a second copy of itself above the first for the wet mornings: four minutes higher at 07:00, eight at 09:00. Nobody told it that rain costs more in traffic than on an empty road. It was handed rain as one of two inputs and that fell out.
That trip through the network is a forward pass, and the intermediate numbers the layers produce are activations. Both words describe something you can now point at.
A workshop with two benches. The first bench takes the raw stock and cuts it into three rough parts. The second bench takes those three parts and assembles two sub-assemblies. A final worker screws them together.
Nobody at the second bench has seen the raw stock. They work only on what the first bench handed them, which is why the numbers in the middle of a network stop having names you can say. They are not departure times any more. They are whatever turned out to be useful.
Twenty numbers all had to be right at once for that pass to land on 54.7. Not one of them was set by a person. Which leaves the obvious question: when the answer comes out wrong, which of the twenty do you blame?
The straight rule had it easy. Two parameters, so a miss of 16 minutes short meant push both up, and how hard depended on how big the feature was.
Twenty parameters, arranged in layers, is the same idea carried out with bookkeeping. Start at the answer with the miss in your hand. Hand each unit that fed the answer a share of the blame, in proportion to how much it contributed. Those units then hand shares of their own blame back to the units that fed them. Keep going until every number has a share.
Two rules do all the work, and both are things you would guess. A number that had more influence gets more of the blame. And a unit that was clipped to zero on this morning gets none, because it did not affect the answer, so it cannot be at fault.
The trip back is the backward pass. Each number's share is its gradient: which way to move it, and how much.
A restaurant order comes back wrong. The manager does not shout at everybody equally. She works backwards: the plate came from the pass, so how much of this is plating, how much is the sauce station, how much is whoever ordered the wrong fish on Tuesday. Each person gets a share sized by how much of the dish went through their hands.
And the station that was closed all evening gets nothing, because it did not touch this plate. That is exactly the rule for a unit that clipped to zero. No contribution, no blame, no adjustment on this morning.
Do that for every morning, add up the shares, move every number a small distance in its own direction, and that is one step. Twenty steps in, the curve already has a peak in it. A hundred steps in, the peak is in roughly the right place at roughly the right height. The remaining two thousand four hundred steps are spent arguing about five minutes of accuracy.
The slider below is the only knob in this whole page that changes what the argument is about. Everything else has been an inspector. Drag it from left to right and you are watching the same twenty numbers at every point in their training, with three readouts moving together.
The shape of that run is not what most descriptions of training suggest. Average miss starts at 21 minutes and is down to 6.5 by step 100. Then 5.5 at step 200, 4.3 at 400, 2.5 at 700, 1.9 at 1000, 1.5 at 1500, 1.3 at 2000, and 1.26 at 2500, where it is still creeping downwards. The shape arrives almost immediately; the rest of the run is haggling. Stop at step 100 and you already have a model that knows there is a rush hour, and one that is five minutes worse than the model you get for twenty-five times the electricity.
Nothing in the loop understood traffic. Twenty numbers were pushed downhill until the shape they drew happened to be a rush hour.
Twenty numbers, two layers, an average miss of 1.26 minutes on a curve where the best straight rule could only manage 11.68. So the four motions still hold. What changed?
Three models have now been trained on one log. Put them side by side and run a morning through each. The four stations are the same four stations in the same order. What differs is what comes out of the last one and what you are allowed to call a miss.
Laid out like that, the differences are smaller than the vocabulary suggests. Regression and classification are not two kinds of learning. They are two kinds of answer, and the answer type decides how you are permitted to measure being wrong: a distance in minutes if the answer is a quantity, a count of four outcomes if the answer is a choice.
Deep learning is not a third kind of answer at all. The network in the last two figures did regression: minutes out, distance measured. Put a probability on the end instead and the same twenty numbers do classification, with a threshold on top exactly as before. What the layers buy is not a new task. It is a shape flexible enough to hold whatever is actually in the data, at the cost of every number in it becoming uninterpretable.
Three tradespeople with the same four-step method: look at the job, try something, see how far off it is, adjust. A tailor works in centimetres, an electrician in yes-there-is-continuity-or-no, a cabinetmaker in a curve that has to match a wall nobody built straight.
The method is not what makes them different. The material is.
Everything up to here has been the loop working. The last thing worth knowing is the shape of it failing, and the failure is not subtle.
Earlier, holding rows back from the two-parameter rule caught almost nothing: the gap between the studied score and the held-back score was three quarters of a minute. Two parameters have nowhere to hide twelve answers.
Twenty parameters are a different animal. Choose how many mornings the network is allowed to study, and watch both scores at once.
Four mornings: average miss on those four, 0.00. The curve goes through all four exactly. Average miss on the eighteen it was never shown, 8.3 minutes. Six mornings: 0.00 again, and 3.2 on the rest. Eight: 0.13 and 4.9. Twelve: 0.44 and 2.3. All twenty-two: 1.26, with nothing left to hold back.
The held-back score does not fall neatly as rows are added: six mornings scored 3.2 and eight scored 4.9. More data made it look worse. That is not an error in the arithmetic, it is what a held-back score built from fourteen rows does, and it is the same wobble that made the split in the earlier figure look pointless. A score measured on a handful of rows is itself a guess.
The studied score says the model is perfect at every one of those settings. It is a lie at every one of them, and the size of the lie is the entire reason anybody holds rows back. This has a name, overfitting, and the picture of it is the curve at four mornings: a shape that goes exactly through the four dots it was shown and does something arbitrary and confident everywhere in between.
A student who has memorised four past exam papers word for word. On those four papers they are flawless. Hand them a fifth and they answer the question they memorised, at length, with total confidence.
You cannot tell the difference between that student and one who understands the subject by looking at their marks on the four papers. You can only tell by using a paper they have not seen, which is why the held-back rows are not a formality.
Two other things this page has been quietly showing you, both worth saying out loud.
The starting numbers matter, and not a little. All twenty parameters begin at small random values, and from some starting points the network never finds the peak at all. It settles into predicting a nearly flat 40 minutes and stays there, average miss above 5 minutes, however long you leave it running. Trying sixty different starting points on this data, eighteen of them end up stuck like that: closer to three in ten than one in ten.
The reason is the one place the clip at zero bites back. A unit whose bracket comes out negative for every morning outputs zero for every morning, so it contributes nothing, so it receives no share of the blame, so it never moves again. It is dead, and nothing in the loop can revive it. The response in practice is not cleverness. It is running the whole thing from several starting points and keeping the one that worked, which is exactly what every figure here does: eight starting points, and the one with the lowest miss on the rows it was allowed to study survives.
And the network still cannot see anything you did not give it. It was handed a clock time and a yes-or-no about rain. It knows nothing about school holidays, roadworks, or the morning the road was closed. Ask it about a Sunday and it will answer, in minutes, to one decimal place, with no way of telling you that it has never seen one.
The three words on the tin describe what comes out of the box: a quantity, a choice, or a quantity produced by something bendy enough to be worth the trouble. They do not describe three different things happening inside. Inside there is one loop, and it is a loop of being wrong on purpose, in small measured amounts, until being wrong gets boring.
The parts worth carrying away are the parts that were decisions rather than arithmetic. You chose the shape of the rule before training began, and no amount of training escapes a shape too rigid to hold the data. You chose which rows the model was allowed to study, and the score on the others is the only one that means anything. You chose what counts as a miss, and that choice crowned a different winner. The arithmetic was never in charge of any of that.
One thing the sandbox will not let you do is win. There is no setting that makes the held-back score match the studied one, because the held-back rows are mornings this log does not contain, and no arrangement of numbers can recover something nobody wrote down.
The usual textbook form is minutes = start + slope × hour, where start is the value at hour zero. Fit that to the early shift and it lands on minus 95.9, which claims a midnight departure takes minus ninety-six minutes. The line is identical; only the label is silly.
There is a practical reason to centre it, beyond readability. Gradient descent on the uncentred form is badly behaved: the two parameters have wildly different sensitivities, so any nudge size small enough to be safe for one is uselessly small for the other, and the run crawls for thousands of steps. Subtracting 7.5 from the hour before it goes in makes both parameters comparably sensitive and the same run converges in about twenty-five steps. Every serious model does some version of this to its inputs, which is also why the network rescales the clock time to sit near zero before the first layer sees it.
Fit the late-or-not classifier with no restraint and it drives its weight up to about 24, which makes the probability curve a cliff: every morning comes out at 0.000, 0.500 or 1.000, and the threshold slider has exactly one place where it does anything.
The fit on this page adds a small penalty on the size of the weights, which stops them running away and leaves the weight at 1.36. The probabilities then spread from 0.145 to 0.940. Nothing about the data changed. The model was told to prefer a gentler opinion, and a gentler opinion is more useful because it distinguishes a 06:00 departure from a 07:00 one instead of calling both of them certain. The technique is called regularisation and this is the mildest possible taste of it.
Four rows in the log leave at 08:00. Two were dry and arrived at 08:55 and 08:58; two were wet and arrived at 09:01 and 09:03. Given only the clock time, those four rows are one row with two contradictory answers, and no rule of any complexity can separate them. The single-feature classifier gives all four a probability of 0.509, which is the correct behaviour and looks like a failure.
This is the most common way a model looks stupid while behaving perfectly. Before reaching for more parameters, it is worth asking whether the rows the model gets wrong are rows that are genuinely distinguishable in the columns you supplied. Twice on this page the answer was no, and both times the fix was a column rather than a model.
The run this page keeps starts from the fifth of its eight starting points. Afterwards the first layer’s three units have weights on the clock of 0.95, minus 1.93 and minus 0.67, and weights on rain of 0.75, minus 0.07 and 0.45. So unit two is almost purely a clock unit running backwards and barely notices the weather, while units one and three carry the rain between them. Nobody designed that division of labour. It fell out of the starting numbers and the walk downhill.
The second layer weights those three by minus 0.43, minus 1.22, 1.97 for its first unit and minus 0.97, minus 0.17, 1.80 for its second, and the output takes 1.26 of the first and minus 1.04 of the second. Both second-layer units are genuinely doing work here, which is somewhat lucky: from other starting points one of them ends up contributing almost nothing and the network does the same job with four fewer numbers. You cannot know which numbers were unnecessary until after you have trained them, which is a large part of why models ship bigger than they need to be.
The hand-built model earlier added three hinges together directly. That is a network with one hidden layer, and one hidden layer of enough units can already approximate any smooth curve, which is a genuine and famous result. So why stack a second layer?
Because "enough units" can mean an enormous number. A second layer takes hinges built from the input and bends them again, so the shapes available grow much faster per parameter added. On a one-feature curve like this morning’s traffic the difference barely matters, and the two-layer network here is showing you the mechanism rather than earning its keep. On an image, where a useful feature is something like "an edge, at any position, at any angle", building it out of one layer of hinges is hopeless and building it out of six is routine.
Three things every real project needs and this page does not mention. The rows here were split once, by hand, into studied and held back; in practice you rotate the split several times and average, because a single split of a small log is mostly luck. The network here sees all its training rows on every step, which stops being possible above a few thousand rows, so real training uses small random handfuls at a time. And nothing here uses a third pile of rows, kept back from even the held-back scoring, which is what you need once you start choosing between models using the held-back score, because at that point the held-back rows have quietly started influencing the model too.