When a model should not decide
Almost every useful argument about putting a model into a workflow gets had in the wrong units. Somebody quotes an accuracy figure, somebody else says it is not high enough yet, and the conversation stalls on a number that was never going to settle it.
Accuracy tells you how often the model is right. It tells you nothing at all about what happens the rest of the time, and that is the part the business lives or dies on.
The three questions that actually decide it
- What does a wrong answer cost, and who pays it? A wrong product recommendation costs a click. A wrong sanctions screening costs a licence.
- Is a wrong answer visible before it becomes expensive? Some errors announce themselves within the hour. Others sit quietly in a file for nine months and arrive as a regulatory finding.
- Can the decision be defended afterwards? Somebody will eventually ask why this file was treated this way, and “the model said so” is not an answer that survives the question.
Answer those three and the accuracy figure usually stops being the argument. It tells you where in the workflow a model belongs, rather than whether it belongs at all.
95% is not a grade, it is a defect rate
A model right nineteen times in twenty sounds like a strong result, and in a workflow where the twentieth is caught, corrected and forgotten, it is. In a workflow where the twentieth becomes a filing, a payment or a position, the same figure means something else entirely: one defect every twenty files, each costing more than the nineteen it saved.
The same accuracy is excellent or unacceptable depending entirely on what sits downstream of it. That is a property of the workflow, not of the model.
The move that usually works
Stop asking whether the model can make the decision, and start asking what it is allowed to decide on its own. Most workflows of this kind split cleanly into a large, dull majority where the answer is obvious and a small, hard remainder where it is not. The value is in handling the first at speed and routing the second to a person, with the reasoning and the sources attached to it.
That design has a property the all-or-nothing version does not: it degrades gracefully. A model that is worse than hoped makes the queue longer. A model that is better than hoped makes it shorter. Neither outcome produces a defect that nobody sees.
Build the evidence trail before you need it
In these markets the thing that eventually gets audited is not the model, it is the decision: what was in front of the system, what it concluded, which version concluded it, who reviewed it and when. A trail retro-fitted after launch is incomplete for everything that happened before — which is precisely the period anyone will want to look at.
Illustrative example: a first-pass review model that clears 70% of files outright and escalates the rest removes most of the work without ever being the last word on a file. A model that clears 95% and is wrong on one in twenty of those is a better model and a worse product.