A classifier does not hand you a verdict. It hands you a probability, and somebody has to decide where to cut. Move that cut and the same model catches every spam, or catches one third of it, while accuracy sits at 0.87 the whole time and tells you nothing.
Email B arrives. Four links, 58 percent of the subject line in capitals, sender not in your address book. You want to know one thing: is it spam?
The model will not tell you. What it produces is a probability: 0.72. Not a decision, a degree of suspicion. Every classifier you have used, including the one filtering your inbox right now, works this way, and the label you eventually see was manufactured afterwards by comparing that number to a cut-off somebody chose.
Click any email below to see what the model actually produced for it.
A smoke alarm that reported "0.72 of the way to a fire" instead of screaming. Useless as a thing to hang in a kitchen, and far more honest. Somebody still has to decide how much smoke is enough to wake the house, and that decision is not the alarm's to make.
Three of these fifteen are genuinely spam. The model has never been told which, and its job was only to sort them by suspicion. It has done that rather well: the three real spam sit at 0.91, 0.72 and 0.34.
That 0.34 is going to cause trouble.
To turn 0.72 into an action you need a rule: flag anything at or above some number. That number is the threshold, and nothing in the data tells you what it should be.
0.5 feels natural because it sounds like "more likely than not". It is a convention, not a law, and on this inbox it is not even the best choice.
Drag the line and watch the labels flip. Notice that the probabilities never move. Only the verdicts do.
Airport security deciding how loudly the metal detector has to beep before they pull someone aside. Turn the dial up and queues move but knives get through. Turn it down and nobody gets through but everybody is searched. The detector is unchanged; the policy is the whole argument.
So a single model is really a family of models, one per threshold. To compare them you have to count what each one gets right and wrong, and there are exactly four ways that can go.
Take email D. The model scored it 0.66 and it is a perfectly real email from a colleague. Set the threshold at 0.50 and watch what that combination produces.
0.66, which is above the line at 0.50, so the filter flags it. That is the prediction, and it is all the filter knows.
Email D is real. It is your colleague asking about Thursday. The model has never seen this column and never will.
Predicted spam, actually real. That is a false positive: a false alarm. Your colleague's message is now in the junk folder and you will find it in a week.
Email C scored 0.34 and really is spam. Below the line, so it is allowed through: a false negative. Spam in your inbox. Annoying rather than damaging, which is exactly the asymmetry that decides your threshold.
True positive: flagged, and it really was spam. True negative: allowed, and it really was real. Every one of the fifteen emails lands in exactly one of these four boxes, and every metric on this page is arithmetic on those four counts.
A jury. Convicting the guilty and acquitting the innocent are both correct. Convicting the innocent and acquitting the guilty are both errors, and no legal system on earth treats them as equally bad. Which mistake you would rather make is a value judgement that arithmetic cannot settle for you.
Sort all fifteen emails into the four outcomes and write the totals in a two by two grid. That grid is the confusion matrix, and despite the name it is the least confusing object in machine learning: it is a tally.
Every single metric that follows is built from these four numbers and nothing else. Click any cell to see which emails are in it.
A lost property office with four shelves: returned to the right owner, returned to the wrong owner, still on the shelf although someone is looking for it, and still on the shelf because nobody wants it. You cannot judge the office without counting all four shelves, and most people only ever look at the first.
Accuracy is the fraction you got right. Add the diagonal, divide by everything.
Eighty seven percent. That sounds like a working spam filter. Now press the button below, which replaces the model with a rule so stupid it does not look at the email at all: never flag anything.
Twelve of the fifteen emails are real, so a rule that flags nothing is right twelve times out of fifteen: 0.80. It has never caught a single piece of spam and never will, and it beats plenty of real models on accuracy alone.
A doctor who tells every patient they are healthy. In a room where 99 people in 100 are healthy, that doctor is 99 percent accurate and will miss every single case of the disease. Accuracy rewards them for the crowd, not for the job.
The number that matters here is the base rate: how common the thing you are hunting actually is. When 20 percent of your emails are spam, 0.80 is the floor, not an achievement. When 1 percent are, the floor is 0.99, and accuracy has stopped carrying information at all.
This is the figure the whole page exists for. The slider is the threshold. The matrix refills as emails migrate between cells, and all four metrics recompute underneath.
Stop at 0.30, then 0.50, then 0.80. Accuracy reads 0.87 at all three. Everything else is different.
Three security guards with identical eyesight and different instructions. One waves everybody through, one stops anything that beeps, one strip-searches the nervous. They will all report the same proportion of correct decisions and you would not hire them for the same job.
Accuracy cannot see the difference because it counts all four cells equally. The two numbers that can see it are the two that ignore a different cell each.
Both are fractions of the true positives. They differ only in what they divide by, and that difference is the whole argument.
They happen to be equal at 0.50 on this inbox, which is a coincidence of small numbers. Move the threshold and they pull apart immediately, always in opposite directions.
Fishing with a net. Precision is how much of your catch is fish rather than boots. Recall is how much of the river's fish you actually caught. A finer net raises one and lowers the other, and no net does both.
Which one you care about is a question about consequences, not statistics. A spam filter that loses a job offer has failed worse than one that leaves you three adverts, so inbox filters run at high precision on purpose. A cancer screen makes the opposite call, because a missed tumour costs more than a second test.
You cannot maximise precision and recall together, but you can score their balance. F1 is their harmonic mean, which is a deliberately unforgiving average: it sits close to the smaller of the two, so it collapses whenever either one collapses.
Grading a restaurant on its worst course. An arithmetic mean lets a superb dessert hide a raw chicken; a harmonic mean does not. That is exactly why F1 is preferred to a plain average of precision and recall.
F1 is a convenience, not a truth. It weights the two fears equally, and your business almost never does.
Every metric so far judged one threshold. But the threshold is a policy you can change on a Tuesday, and the model underneath does not change with it. To judge the model rather than the policy, plot what happens at all of them.
Plot FPR across and TPR up, once per threshold, and join the dots. Press sweep and watch the curve draw itself as the threshold falls.
The area under that curve is 0.944, and it has a meaning far more concrete than "area":
Not "how many did it get right", but "does it put the guilty higher up the list than the innocent". A model can rank beautifully and still be useless until someone picks a cut-off, which is why AUC and accuracy so often disagree.
There is no universal threshold for "good", but there are floors, and most reported results quietly fail one of them. Compare any classifier against these four before believing it.
Accuracy above the base rate. Here the base rate is 0.80, so 0.87 is a gain of seven points, not a triumph. If accuracy is not comfortably above the majority-class rate, the model has learned nothing.
AUC well above 0.5. 0.5 is a coin flip. 0.944 says the ranking is genuinely good, which is a different claim from "the decisions are good".
Precision and recall quoted together, at a stated threshold. Either alone is meaningless, and a metric without its threshold is not a result.
The confusion matrix printed in full. Four numbers, not one. Everything anybody can compute is in there, and nothing can be hidden by an average.
Nobody accepts "our surgeon has a 90 percent success rate" without asking what the operations were. The confusion matrix is the case list, and every summary metric is a press release written from it.
The honest report for this model is one sentence: at a threshold of 0.72 it catches two of the three spam with no false alarms, its ranking is right 94 percent of the time, and the third spam looks exactly like an ordinary email to it.
It produced fifteen numbers between 0 and 1 and stopped. Everything after that, the flag, the folder, the four counts, the seven metrics and the argument about which one to put in the deck, came from a threshold that a person chose and could change this afternoon.
Precision is TP divided by TP plus FP. Flag nothing and both are zero, so the fraction is 0/0 and simply has no value. Libraries usually report 0 and warn you. It is worth knowing because a suspiciously perfect precision often means almost nothing was flagged at all.
ROC uses FPR, whose denominator is the number of true negatives. When negatives vastly outnumber positives, a large absolute number of false alarms still divides down to a tiny FPR, and the ROC curve looks flattering. A precision-recall curve has no such denominator and stays honest under heavy imbalance, which is why fraud and disease work usually quotes it.
With three positives, recall can only ever be 0, 0.33, 0.67 or 1. One email changing its mind moves it by 33 points. Everything here is arithmetically exact and statistically meaningless: the point is the machinery, not the verdict. Real evaluation needs enough positives that the metrics have somewhere to move.
This page treats the scores as given. In practice they come from a fitted model, and many models produce numbers that rank well without being genuine probabilities: a score of 0.9 does not necessarily mean nine times in ten. Checking that is called calibration, and it is a separate exercise from everything here.
If a false negative costs you 1 and a false positive costs you 10, the threshold that minimises expected cost is not the one that maximises F1. Write both costs down, sweep the threshold, and pick the minimum of the total. That calculation is usually more defensible than any named metric.