Ten rows, two inputs, one answer, and a network small enough to print. Every weight is on the page, every multiplication can be checked on paper, and the error that trains it can be watched walking backwards through the layers one edge at a time.
Ten rooms. For each one, the temperature, the humidity, and how comfortable people said it was out of ten.
Comfort is not a straight line in either input, and that is the whole reason this page exists. Too cold is bad. Too hot is bad. Somewhere around 23 degrees is lovely. The same for humidity. A rule of the form "comfort goes up by so much per degree" is wrong at both ends no matter what number you choose.
Press the button and the best straight line through both features will be drawn. It is the best one that exists, and it is still bad.
Fitting a plank to the inside of a bowl. You can angle the plank, you can slide it, and it will still cross the curve in two places and miss everywhere else. No amount of choosing a better plank fixes a curve. You need something that can bend.
A neural network is the cheapest way to build something that bends. It is made of a part so simple you can compute it in your head.
That is the entire operation. Take the inputs, multiply each by its own weight, add a bias, then push the result through a function that flattens the extremes.
Here is one neuron doing it, on room seven: 27 degrees, 65 percent humidity. The inputs are scaled to sit near 0 and 1 first, which is bookkeeping rather than mathematics.
Drag the weights below and watch the output move. The squash is what stops the neuron being a straight line, and it is the only nonlinear thing in the entire network.
A volume knob with a limiter. Turn it up and the sound gets louder, but only to a point; past that the amplifier flattens everything into the same loudness. That flattening is exactly what makes a stack of neurons able to bend, and exactly what makes them stop learning when you push them too far.
Stack those neurons. Two inputs feed two neurons, those feed two more, and those feed one output that is not squashed because comfort is a quantity rather than a yes or no.
Count the numbers. Four weights and two biases into the first layer, four and two into the second, two and one into the output. Fifteen. That is the whole model, and every one of them is printed on the diagram below.
Click any edge to see its weight, or any neuron to see what it computes.
A mixing desk with fifteen faders and no other controls. Learning is somebody nudging faders, listening, and nudging again. Nothing is added, nothing is rewired, and at the end the desk looks identical to the one you started with apart from where the faders sit.
Those fifteen numbers start as guesses. To improve them you first have to find out how wrong they are, which means pushing a row all the way through.
Room seven again: 27 degrees, 65 percent, actually rated 8.3. Press play and watch the numbers travel. Each neuron waits for its inputs, multiplies, adds its bias, squashes, and passes the result on.
Nothing here is learned or clever. It is fifteen multiplications and some additions, in a fixed order.
A bucket chain at a fire. Nobody in the line knows where the water came from or where it is going; each person takes what arrives, does one thing to it, and hands it on. The answer at the end is the whole chain's doing, and no single link contains it.
Do that for every row, square each miss, take the average. This is the same mean squared error from any regression, and it is what "training" is trying to make small.
The important thing about the loss is not the formula, it is what it depends on. Every one of those fifteen weights affects it. Change one and the loss changes. That makes the loss a function of fifteen numbers, and finding its lowest point is the entire job.
Standing on a hillside in fog with fifteen dials, each of which tilts the ground a little. You cannot see the valley, but you can feel which way each dial makes the ground fall. Turn them all slightly downhill, then feel again. That is the whole of training.
Feeling all fifteen slopes without walking the hill fifteen times is what backpropagation is for.
The model said 0.899, the room was 0.830. That gap is the only thing the network has to learn from, and it starts at the output and moves left, splitting at every neuron.
At each step the question is the same: how much did this weight contribute to the error? A weight that fed a large signal into a big mistake gets a large share. A weight whose neuron was saturated gets almost nothing, because a flat curve passes no blame.
Press play. The dashed tokens carry blame backwards, and every weight lights up with the amount assigned to it.
A project goes wrong and the blame is apportioned backwards down the chain of command. Everyone's share depends on how much they contributed and how much authority they had, and someone who was asleep at the time gets none. The derivative of the squash is exactly that "how awake was this neuron" factor.
Note what backpropagation is not: it is not searching, guessing, or trying weights to see what happens. It is one pass of the chain rule from calculus, and it produces all fifteen slopes for roughly the cost of one forward pass. That efficiency is the only reason any of this is practical.
Forward, backward, nudge, repeat. That loop is training, and it has exactly one setting that matters here: how far to move each weight along its slope.
The loop below is real. It runs the same fifteen weights, the same ten rooms, and the same arithmetic you just checked by hand. Set the rate and press train.
Walking downhill in fog again. Tiny steps and you are still on the hillside at nightfall. Enormous steps and you leap clean across the valley to the opposite slope, then leap back. Nothing about the hill changed; only your stride.
Watch the loss curve rather than the number. A curve that falls fast and flattens is healthy. One that falls and then jitters sideways is a rate slightly too high. One that goes flat immediately at a bad value is saturation, and no amount of patience fixes it.
After training, the four hidden neurons are not mysterious. Each one is a squashed tilt across the temperature and humidity plane, and the output is a weighted sum of those tilts. Bending happens where they overlap.
Click a hidden neuron to see the surface it responds to. Nobody designed these. They fell out of fifteen numbers being nudged downhill three hundred times.
Making a curved shape out of stiff card by folding several flat pieces and overlapping them. No individual piece bends. The shape does. That is a hidden layer, and adding a second one lets you fold the folds.
Now the honest part. Ten rows and fifteen weights is close to one and a half parameters per observation, which is a ratio no serious model would accept. This network can very nearly memorise the table, and its 0.0031 loss on the same ten rooms it trained on is not evidence that it would work on the eleventh room.
Everything on this page is the mechanism, checked to the digit. None of it is a claim that the model is good. Establishing that would take a held-back set, and with ten rows there is nothing to hold back.
Fifteen numbers turn two inputs into one output by multiplying, adding and squashing. A loss says how wrong the answers are. The chain rule walks that wrongness backwards and hands each of the fifteen a share of the blame. Each moves a little against its share. Repeat three hundred times and a set of arbitrary numbers has become the shape of a comfortable room. Every model you have heard of is this, with more numbers.
Comfort is a quantity that can be any number, so the last step is left linear. Squash the output with tanh and the model could never predict outside minus one to one. For classification you would put a sigmoid there instead, precisely because you do want the answer trapped between 0 and 1.
Tanh is smooth, centred on zero, and its derivative is a tidy 1 minus a squared, which is why the arithmetic on this page stays checkable. Modern networks mostly use ReLU, which is max(0, z): faster and it does not saturate at the positive end, which matters enormously once you have fifty layers rather than two.
They were chosen to be round numbers so the worked example is readable. Real initialisation draws small random values with a variance that depends on the layer width, because starting every weight identical would make every neuron in a layer compute the same thing forever, and starting them large saturates the squash before training begins.
This page computes the gradient over all ten rows before each update, which is full batch gradient descent. Real training almost always uses small random batches: noisier steps, far cheaper per step, and the noise itself helps escape poor spots. The chain rule is identical either way.
A huge step lands the weights somewhere z is large, where tanh is flat. The derivative 1 minus a squared is then almost zero, so the blame passed backwards is almost zero, so the weights barely move again. The network is not stuck at a minimum, it is stuck on a plateau of its own making, and this is the small version of the vanishing gradient problem that made deep networks hard for years.
A single hidden layer with enough units can approximate this surface perfectly well; the universal approximation theorem says so. Two layers is used on this page because the backward pass through two layers shows the chain rule chaining, which is the thing worth seeing. Depth earns its keep on problems far larger than ten rooms.