One slider, three different models, one accuracy

A classifier does not hand you a verdict. It hands you a probability, and somebody has to decide where to cut. Move that cut and the same model catches every spam, or catches one third of it, while accuracy sits at 0.87 the whole time and tells you nothing.

15 emails · 3 of them spam · one threshold you choose

every email, sorted by how suspicious the model finds itdrag the line
everything right of the line gets flagged

The model never says spam. It says 0.72.

Email B arrives. Four links, 58 percent of the subject line in capitals, sender not in your address book. You want to know one thing: is it spam?

The model will not tell you. What it produces is a probability: 0.72. Not a decision, a degree of suspicion. Every classifier you have used, including the one filtering your inbox right now, works this way, and the label you eventually see was manufactured afterwards by comparing that number to a cut-off somebody chose.

Click any email below to see what the model actually produced for it.

figure 1 · what the model hands youclick an email
Click a row. The truth column is what actually happened, and the model never sees it. It is only there so you can mark the homework.
picture it

A smoke alarm that reported "0.72 of the way to a fire" instead of screaming. Useless as a thing to hang in a kitchen, and far more honest. Somebody still has to decide how much smoke is enough to wake the house, and that decision is not the alarm's to make.

Three of these fifteen are genuinely spam. The model has never been told which, and its job was only to sort them by suspicion. It has done that rather well: the three real spam sit at 0.91, 0.72 and 0.34.

That 0.34 is going to cause trouble.

A threshold is a line you choose, not a fact you discover

To turn 0.72 into an action you need a rule: flag anything at or above some number. That number is the threshold, and nothing in the data tells you what it should be.

0.5 feels natural because it sounds like "more likely than not". It is a convention, not a law, and on this inbox it is not even the best choice.

Drag the line and watch the labels flip. Notice that the probabilities never move. Only the verdicts do.

figure 2 · the same scores, different verdictsthreshold 0.50
Drag it. At 0.30 you flag five emails. At 0.80 you flag one. Same model, same fifteen scores, two different products.
picture it

Airport security deciding how loudly the metal detector has to beep before they pull someone aside. Turn the dial up and queues move but knives get through. Turn it down and nobody gets through but everybody is searched. The detector is unchanged; the policy is the whole argument.

So a single model is really a family of models, one per threshold. To compare them you have to count what each one gets right and wrong, and there are exactly four ways that can go.

Four things can happen to one email, and only two of them are fine

Take email D. The model scored it 0.66 and it is a perfectly real email from a colleague. Set the threshold at 0.50 and watch what that combination produces.

figure 3 · the four outcomesemail D
Each step changes the drawing. Click a step to jump to it.

What the model said

0.66, which is above the line at 0.50, so the filter flags it. That is the prediction, and it is all the filter knows.

What was actually true

Email D is real. It is your colleague asking about Thursday. The model has never seen this column and never will.

Cross the two and you get a name

Predicted spam, actually real. That is a false positive: a false alarm. Your colleague's message is now in the junk folder and you will find it in a week.

The opposite mistake has its own name

Email C scored 0.34 and really is spam. Below the line, so it is allowed through: a false negative. Spam in your inbox. Annoying rather than damaging, which is exactly the asymmetry that decides your threshold.

And the two ways to be right

True positive: flagged, and it really was spam. True negative: allowed, and it really was real. Every one of the fifteen emails lands in exactly one of these four boxes, and every metric on this page is arithmetic on those four counts.

picture it

A jury. Convicting the guilty and acquitting the innocent are both correct. Convicting the innocent and acquitting the guilty are both errors, and no legal system on earth treats them as equally bad. Which mistake you would rather make is a value judgement that arithmetic cannot settle for you.

The confusion matrix is those four counts in a box

Sort all fifteen emails into the four outcomes and write the totals in a two by two grid. That grid is the confusion matrix, and despite the name it is the least confusing object in machine learning: it is a tally.

Every single metric that follows is built from these four numbers and nothing else. Click any cell to see which emails are in it.

figure 4 · the tally, at threshold 0.50click a cell
Click a cell, then move the threshold. Watch emails migrate between cells. The four counts always add to fifteen, whatever you do.
at threshold 0.50

            actually spam   actually real
flagged        TP 2            FP 1
allowed        FN 1            TN 11

2 + 1 + 1 + 11 = 15 emails
picture it

A lost property office with four shelves: returned to the right owner, returned to the wrong owner, still on the shelf although someone is looking for it, and still on the shelf because nobody wants it. You cannot judge the office without counting all four shelves, and most people only ever look at the first.

Accuracy is the first number you see and the easiest to fool

Accuracy is the fraction you got right. Add the diagonal, divide by everything.

accuracy = (TP + TN) / (TP + FP + FN + TN)

= ( 2 + 11 ) / 15
= 13 / 15 = 0.87

Eighty seven percent. That sounds like a working spam filter. Now press the button below, which replaces the model with a rule so stupid it does not look at the email at all: never flag anything.

figure 5 · accuracy against a rule that does nothingthe model at 0.50
Compare the three. Doing nothing scores 0.80 and catches no spam whatsoever. That gap, 0.87 against 0.80, is the entire value the model adds by this measure.

Twelve of the fifteen emails are real, so a rule that flags nothing is right twelve times out of fifteen: 0.80. It has never caught a single piece of spam and never will, and it beats plenty of real models on accuracy alone.

picture it

A doctor who tells every patient they are healthy. In a room where 99 people in 100 are healthy, that doctor is 99 percent accurate and will miss every single case of the disease. Accuracy rewards them for the crowd, not for the job.

The number that matters here is the base rate: how common the thing you are hunting actually is. When 20 percent of your emails are spam, 0.80 is the floor, not an achievement. When 1 percent are, the floor is 0.99, and accuracy has stopped carrying information at all.

Always read accuracy next to the base rate. On its own it is a number about your data, not about your model.

One slider, three different models, one accuracy

This is the figure the whole page exists for. The slider is the threshold. The matrix refills as emails migrate between cells, and all four metrics recompute underneath.

Stop at 0.30, then 0.50, then 0.80. Accuracy reads 0.87 at all three. Everything else is different.

figure 6 · the threshold, livethreshold 0.50
Press 0.30, 0.50 and 0.80 in turn. The emails fly to their new cells. Accuracy holds at 0.87 through all three while recall goes 1.00, 0.67, 0.33.
threshold   TP  FP  FN  TN   accuracy  precision  recall
   0.30       3   2   0  10     0.87      0.60    1.00
   0.50       2   1   1  11     0.87      0.67    0.67
   0.80       1   0   2  12     0.87      1.00    0.33

same model, same fifteen emails, same accuracy, three products
picture it

Three security guards with identical eyesight and different instructions. One waves everybody through, one stops anything that beeps, one strip-searches the nervous. They will all report the same proportion of correct decisions and you would not hire them for the same job.

Accuracy cannot see the difference because it counts all four cells equally. The two numbers that can see it are the two that ignore a different cell each.

Precision and recall are two different fears

Both are fractions of the true positives. They differ only in what they divide by, and that difference is the whole argument.

precision = TP / (TP + FP)   of everything I flagged, how much was really spam
= 2 / (2 + 1) = 0.67   a third of my flags were false alarms

recall     = TP / (TP + FN)   of all the spam there was, how much did I catch
= 2 / (2 + 1) = 0.67   a third of the spam got through

They happen to be equal at 0.50 on this inbox, which is a coincidence of small numbers. Move the threshold and they pull apart immediately, always in opposite directions.

figure 7 · the two curves that crossthreshold 0.50
Sweep the threshold. Precision climbs as you get fussier; recall falls. You are not tuning quality, you are choosing which mistake to make.
picture it

Fishing with a net. Precision is how much of your catch is fish rather than boots. Recall is how much of the river's fish you actually caught. A finer net raises one and lowers the other, and no net does both.

Which one you care about is a question about consequences, not statistics. A spam filter that loses a job offer has failed worse than one that leaves you three adverts, so inbox filters run at high precision on purpose. A cancer screen makes the opposite call, because a missed tumour costs more than a second test.

F1 is what you report when you are forced to pick one number

You cannot maximise precision and recall together, but you can score their balance. F1 is their harmonic mean, which is a deliberately unforgiving average: it sits close to the smaller of the two, so it collapses whenever either one collapses.

F1 = 2 × (precision × recall) / (precision + recall)

at 0.50  = 2 × (0.67 × 0.67) / (0.67 + 0.67) = 0.67
at 0.80  = 2 × (1.00 × 0.33) / (1.00 + 0.33) = 0.50  perfect precision cannot rescue it
at 0.72  = 2 × (1.00 × 0.67) / (1.00 + 0.67) = 0.80  the best this model can do

a plain average of 1.00 and 0.33 would have said 0.67. The harmonic mean says 0.50.
figure 8 · F1 across every thresholdpeak 0.80 at 0.72
Press sweep. The marker walks every threshold from 0 to 1 and F1 traces out underneath it. The peak is at 0.72, not at 0.50.
picture it

Grading a restaurant on its worst course. An arithmetic mean lets a superb dessert hide a raw chicken; a harmonic mean does not. That is exactly why F1 is preferred to a plain average of precision and recall.

F1 is a convenience, not a truth. It weights the two fears equally, and your business almost never does.

The ROC curve is every threshold at once

Every metric so far judged one threshold. But the threshold is a policy you can change on a Tuesday, and the model underneath does not change with it. To judge the model rather than the policy, plot what happens at all of them.

TPR = TP / (TP + FN)   the recall, again, under a second name
FPR = FP / (FP + TN)   of the real emails, how many did I wrongly flag

at 0.50  TPR = 2/3 = 0.67   FPR = 1/12 = 0.08

Plot FPR across and TPR up, once per threshold, and join the dots. Press sweep and watch the curve draw itself as the threshold falls.

figure 9 · the ROC curve, drawn by sweeping the thresholdAUC 0.94
Sweep it. The dot walks the curve as the threshold falls from 1 to 0. Up and left is good. The diagonal is a coin flip.

The area under that curve is 0.944, and it has a meaning far more concrete than "area":

take one real spam and one real email, at random
the model scores the spam higher 34 times out of 36
34 / 36 = 0.944 = the AUC

0.5 means it ranks no better than a coin. 1.0 means it never gets a pair wrong.
picture it

Not "how many did it get right", but "does it put the guilty higher up the list than the innocent". A model can rank beautifully and still be useless until someone picks a cut-off, which is why AUC and accuracy so often disagree.

What the numbers have to clear before anyone should believe them

There is no universal threshold for "good", but there are floors, and most reported results quietly fail one of them. Compare any classifier against these four before believing it.

figure 10 · this model against the floors it has to clearat threshold 0.50
Switch the comparison. Each one answers a different version of "compared to what", and a model has to survive all three.

Accuracy above the base rate. Here the base rate is 0.80, so 0.87 is a gain of seven points, not a triumph. If accuracy is not comfortably above the majority-class rate, the model has learned nothing.

AUC well above 0.5. 0.5 is a coin flip. 0.944 says the ranking is genuinely good, which is a different claim from "the decisions are good".

Precision and recall quoted together, at a stated threshold. Either alone is meaningless, and a metric without its threshold is not a result.

The confusion matrix printed in full. Four numbers, not one. Everything anybody can compute is in there, and nothing can be hidden by an average.

picture it

Nobody accepts "our surgeon has a 90 percent success rate" without asking what the operations were. The confusion matrix is the case list, and every summary metric is a press release written from it.

The honest report for this model is one sentence: at a threshold of 0.72 it catches two of the three spam with no false alarms, its ranking is right 94 percent of the time, and the third spam looks exactly like an ordinary email to it.

The model was never the decision

It produced fifteen numbers between 0 and 1 and stopped. Everything after that, the flag, the folder, the four counts, the seven metrics and the argument about which one to put in the deck, came from a threshold that a person chose and could change this afternoon.