37 standalone questions on when a rule beats a model — the checklist, the three worked examples, and the layered designs real systems use.
Audience: absolute beginners. No maths, no code, no prior exposure assumed.
Source: the single most commercially useful line on the session-one whiteboard, and the three worked examples written under it.
How to use this: every question stands alone. Pick an option, then read the answer. The sections move from the test, through the three worked examples, to the layered designs that real systems actually use.
A · The line, and why it needs saying
Q1The principle sounds obvious, yet it is violated constantly and expensively by people who are not stupid. Why?
Answer: (2)There is no conference talk in an IF statement, no press release, no board slide. Meanwhile "we deployed AI" is a sentence that funds careers. The pull is always toward the model, and the pull has nothing to do with whether the model is the right tool.
Q2Which sentence best captures the principle?
Answer: (2)Not a prohibition on AI. A statement about where each tool earns its cost. A rule commits in advance to exactly what will happen; a model offers a probability.
B · What "deterministic" actually means
Q3Two questions settle whether something is deterministic. Which pair?
Answer: (2)Two yeses means deterministic — same input, same answer, every time, and you can say in advance what it will be. Write the rule.
Q4Which of these is genuinely deterministic?
Answer: (3)A comparison against a threshold somebody chose in a meeting. The other three are all predictions about something not yet known or not directly stated — they have no rule, and pretending otherwise produces a rule that is reliably wrong.
Q5"Should we approve this loan?" Deterministic or not?
Answer: (3)This is the interesting middle, and it is where the real arguments happen. Almost every real business decision splits like this, which is why the layered designs later in this bank matter more than the either/or framing.
C · Why a rule wins when one exists
Q6A model reaches 99% accuracy on tax calculation. What is the verdict?
Answer: (3)Tax is defined by law. The rule already exists, is exact, and is free to run. A 99% model here means one wrong calculation in a hundred, forever, with no way to know which one — in exchange for nothing.
Q7Which difference between rules and models do people most consistently underweight?
Answer: (3)A broken rule throws an error and somebody notices within minutes. A degraded model just gets quietly worse for months. That is far more dangerous than an outage, because an outage is at least noticed.
Q8You need to change a business threshold. What does that cost under each approach?
Answer: (2)And note the asymmetry in testing too: with a rule you can check every case, whereas with a model you test on samples and hope. These differences compound over the life of a system.
D · Using a language model as a calculator
Q9Someone pastes a column of numbers into a chatbot and asks for the total. What is the harm?
Answer: (2)The last part is the real damage. A wrong SUM does not happen. A wrong chatbot total looks exactly like a right one. The model is not adding — it is producing the most plausible-looking answer, which is precisely what inference means.
Q1017 × 24. A calculator returns 408 and will return 408 forever. A very clever friend says "about 400, maybe 410" — and on a bad day says 408 with total confidence when the answer was something else. Which do you use for an invoice?
Answer: (2)Nobody hesitates when it is put this way. Now notice: asking a language model to do arithmetic is asking the clever friend. It is the most common live violation of the principle you will see in a room.
Q11What do all these have in common — adding a column, sorting a list, validating an email address, calculating GST, looking up an order, counting rows, redacting card numbers?
Answer: (2)Inference is for when the answer does not exist yet and must be guessed at from evidence. When the answer already exists, guessing at it is strictly worse than fetching it — slower, costlier, and unverifiable.
Q12You need to strip card numbers out of a document. Rule or model?
Answer: (2)Redaction is a job where partial success is failure. A pattern rule is exact, testable on every case, and free. This is the same category as tax: the requirement is not "usually right."
E · The checklist
Q13The decision checklist puts three questions before any consideration of a model: can a person write the rule in a sentence or two? Is the rule fixed by law, contract or policy? Must you explain every individual decision? Why do these come first?
Answer: (2)Most bad AI projects fail because somebody started at question 5 and never asked questions 1 to 3. The ordering is the whole design of the list: permission first, capability second.
Q14Which of these is a genuine reason to reach for a model?
Answer: (3)Adversaries defeat fixed rules by design — this is the spam filter that blocked "lottery" until spammers typed "l0ttery." The other three legitimate triggers are: the right answer keeps changing, there are more signals than a person can hold at once, and the input is messy (photos, speech, free text).
Q15You work down the checklist and reach the bottom without a single yes. What should you do?
Answer: (2)The checklist is designed so that a model has to earn its place. Reaching the bottom with no yes means nothing about the problem requires a guess — so do not pay for one.
F · The three from the board
Q16Telephone number validation. Rule, model, or both?
Answer: (1)An Indian mobile number is ten digits starting 6, 7, 8 or 9, after stripping any +91, 0 or spaces. One line, and exactly right on every number that has ever existed or ever will. Question 1 of the checklist is a yes, and you stop there.
Q17Suppose you built a model for phone-number validation instead. What would you have achieved?
Answer: (2)You would have spent real money to make a solved problem worse and unexplainable. This is the clearest illustration in the whole topic of what the principle is protecting you from.
Q18Three questions sit next to each other: is this number correctly formatted? Is it real and active? Is it a spam caller? What are the three answers?
Answer: (3)Formatting is deterministic. "Real and active" is neither a rule nor a model — it is a lookup, a network query whose answer is a fact somebody already holds. "Spam caller" changes constantly and has adversaries. Validation is not verification is not judgement, and most arguments about whether something needs AI are really arguments about which of these three is being asked.
Q19Identifying a fraudulent UPI transaction. Rule, model, or both?
Answer: (3)This is the most important of the three, because it is what real systems actually look like. Not "rules or model" but rules, then model, then rules.
Q20In that fraud design, why not use rules alone?
Answer: (2)A published boundary is an invitation. This is the spam-filter problem with money attached.
Q21And why not use a model alone?
Answer: (2)Paying for a prediction where the answer is already certain is waste, and "the model scored it 0.91" is not an answer a customer or a regulator accepts. Both objections are handled by putting rules on either side of the model.
Q22Predicting consumer churn. Someone proposes the rule "hasn't logged in for 30 days." What is wrong with it?
Answer: (2)Which is what failing the checklist looks like in practice. Then question 4 is a yes (what drives churn shifts when a competitor changes price) and question 5 is a yes (tenure, tickets, usage, billing, payment failures, app opens, NPS, contract date — more than a person can hold at once).
Q23For that churn model: what should decide the action taken on a customer scoring 0.86?
Answer: (2)A model tells you what is likely. A rule tells you what to do about it. Most well-built AI systems are a thin model wrapped in thick rules — and the rules are where your company's actual judgement lives. Nobody would accept a model deciding on its own how much money to hand out.
G · Finding the boundary
Q24Two tasks from the same business. A: check a mobile number is ten digits starting 6–9. B: decide whether that number belongs to a spam caller. Which is which?
Answer: (2)Format is fixed forever. Who is a nuisance changes weekly, and they are actively trying not to be caught. Same phone number, two completely different questions.
Q25A: read the handwritten amount off a photo of a cheque. B: check that the amount in words matches the amount in figures. Which is which?
Answer: (2)People expect the rule first and the clever bit last. Here the messy job genuinely needs a model, and then a dead-simple rule checks its work — comparing "forty-two thousand" against "42,000" is trivial and catches the model's mistakes for free.
Q26What is the general pattern behind that reversal?
Answer: (2)The model does what only a model can. The rule catches it when it slips. Transcribing a call and then stripping the 12-digit numbers out of the transcript is the same shape.
Q27A: give 10% off to every customer in the Gold tier. B: decide which customers should be in the Gold tier. Why do these feel like one question and behave like two?
Answer: (2)Whenever someone says a decision needs AI, ask which half they mean. Usually it is this split, and usually the policy half was never in question.
Q28A: escalate any support ticket untouched for 24 hours. B: decide how urgent a ticket is from what the customer wrote. Which is which?
Answer: (1)A clock is deterministic. Reading distress in free text is not — it is messy input with no writable rule, which is checklist question 6.
H · Layered designs
Q29In a layered design, what does Layer 1 do?
Answer: (2)Frozen account, expired card, two cities in ten minutes, above the daily limit — decided outright, instantly, for free. Layer 2 is the genuine judgement. Layer 3 is the rules that sit on the model's output before anyone acts on it.
Q30Why does putting hard rules first make the model better as well as cheaper?
Answer: (2)Layer 1 handles most of the volume perfectly, in microseconds, at no cost. A pure-model design pays inference cost on every single transaction, including the ones where the answer was never in doubt — and asks the model to be good at cases that needed no judgement at all.
Q31Which of these belongs in Layer 3 — rules on the model's output?
Answer: (3)Layer 3 is where the governance lives, and it is the layer groups always argue about. It is also the layer that makes the system defensible: a reason code exists because a regulator will ask.
Q32Two customer records: Rajesh Kumar, 9876543210, Andheri and Rajesh Kr., 98765 43210, Andheri (W). What belongs in Layer 1?
Answer: (2)Then Layer 2 handles the genuine near-misses, and Layer 3 says: never auto-merge two records holding different PANs, no matter how similar the names.
I · The honest counter-argument
Q33State the strongest argument against the principle.
Answer: (2)Say this yourself before someone else does, or the principle sounds dogmatic. The refined version is: rules are better than models where a rule is genuinely available — and "available" means it will still be correct next year.
Q34Three warning signs tell you a rule has stopped being available. Which is not one of them?
Answer: (4)Age alone means nothing — plenty of rules written decades ago are still exactly right, because the thing they describe has not changed. The other three all mean the same thing: you have a machine learning problem that you are currently solving by hand.
Q35A company adopts the rule "reject every applicant under 24." It is deterministic, explainable and cheap. What is wrong with it?
Answer: (2)The mirror-image failure is equally real: forcing a rule onto something inherently uncertain. The principle is not "rules are always better." It is that rules win where a rule genuinely exists — and "under 24" is not a rule about repayment, it is a proxy someone invented.
Q36Someone insists the product needs AI. What is the most productive thing to ask?
Answer: (2)Do not argue about AI; ask what job it is doing. Related openers: "Smarter than what — what does the current rule get wrong?" and, for "it'll handle the edge cases," "name three. If we can name them, they're rules."
Q37What is the fair, non-cynical statement of the whole principle?
Answer: (2)And that second place — where nobody can write the rule — is where all your competitive advantage actually is. The point is not to keep AI out. It is to stop wasting it on solved problems.