Decide whether AI belongs in the feature at all

Consider a model that predicts which invoices will be paid late. It is not a hard build: the history is in the ledger, the target is clean, the accuracy is respectable. Then nobody uses it, because the collections team already knew which customers pay late — they just did not have authority to do anything about it.

The model was never the problem. The problem was that prediction was not the bottleneck.

That failure was knowable in advance, in an hour, before anybody wrote a prompt. Four questions do it, and none of them is about the model.

Is the output a judgment or a lookup?

If the answer exists somewhere and just needs finding, you want a query, not a model. “What did this customer order last March” is a lookup. “Is this customer about to churn” is a judgment. Models are for judgment. People reach for them for lookups constantly, and then pay per token for something a WHERE clause did for free.

The test that settles it: would two careful people, given the same source material, produce the same answer? If yes it is a lookup, and the variation a model adds is not insight — it is error.

Mixed features are fine when you know which half is which. A box that answers “how much did we ship to the northeast in Q3” is a model translating the question and a database producing the number: the model is the interface, never the source of it.

Can you tolerate being wrong?

Not “will the model be wrong” — it will. The question is what happens next. Drafting a reply someone reads before sending: fine. Posting the reply: not fine. If a wrong answer is unrecoverable, either add a human step or drop the feature.

The two directions of error rarely cost the same. A flagged invoice that turns out to be fine costs a minute of attention; a missed one costs the money. Write both down: the feature you design for one is not the one you design for the other.

Then there is silence. If a wrong answer produces no signal — nobody complains, nothing fails, no number moves — you will not find out for months, and the discovery event is usually an audit. How much review is a later decision; here you need only the binary: is there an output this feature could produce that you could not walk back?

That is the opening. The rest of the lesson — the remaining sections, the exercise and the check-your-understanding questions — is behind the gate below.