No bank expects its investigators to be right every time. It builds quality assurance, escalation, and case review to catch any mistakes they make.

When AI enters the same workflow, the conversation can shift to proving it will never be wrong, and recent research suggests that approach has its limits.

In September, a study of 202 US senior AI decision-makers at publicly traded companies with $1B or more in revenue found that 98% now have formal AI governance policies [1]. 

Of these companies, 47% said their organisation had previously ignored its governance process for urgent deployments, and among those using Agentic AI, 49% had not yet updated their framework for its specific risks [1].

While the survey is US-only and cross-sector, it raises a question every institution using AI in financial crime compliance should confront: if the processes designed to prove AI is safe can be set aside under pressure, what protects the institution when an AI decision turns out to be wrong?

The answer cannot be a promise that AI will never make mistakes. It has to be an operating model designed so that mistakes are detectable, containable, explainable, and correctable. 

That is the real test of compliance AI.

Proving AI Is Safe Is Not the Same as Controlling It in Production

Pre-deployment governance asks whether AI is safe enough to switch on. That question matters, but it is answered once, before go-live, and the figures show it is sometimes skipped altogether when the pressure is on. 

What protects an institution day to day is something else: the controls that catch an incorrect decision in live operation and allow it to be fixed.

This is already how financial institutions manage their investigators. Nobody tries to prove in advance that an investigator will never make a mistake. Instead, quality assurance, escalation, and case review catch errors as they happen. 

Compliance AI needs the same thing: not proof that it will never be wrong, but controls that catch it when it is.

"The data says it all. Organisations spend an inordinate amount of time on governance to prove AI is safe, rather than building the production controls that identify and fix errors rapidly and robustly.”

Ben Rayner
Global Head of Account Management
Silent Eight

Why Accuracy Alone Doesn't Make Compliance AI Production-Ready

Accuracy matters, but a headline accuracy figure is an average, and an institution's exposure rarely sits within the average case.

Risk concentrates at the edges: incomplete or transliterated names, poor-quality data, unfamiliar typologies. A controlled test on prepared data says little about how a system behaves when it meets those cases in production.

An equally important test is how a model behaves when its confidence is low or the evidence is incomplete. When a case sits close to a decision threshold, does the system force a call one way or the other, or does it recognise its uncertainty and route the case to an investigator?

"The issue with accuracy, as important as it is, is that it's a number. Can it be 100%, always and forever? If not, it's an average over a period of time. What matters is understanding where the model is weakest and what it does there."

Ben Rayner

Agentic AI Raises the Stakes of Every Error

The question becomes more pressing as AI moves from recommending decisions to acting on them.

Agentic AI changes the governance challenge because greater autonomy creates new obligations to evidence, explain and control end-to-end decision loops. Banks need governed and auditable processes around AI Agents, with defined accountability, escalation, evidence requirements, human-machine boundaries, monitoring and ongoing assurance [2].

Set that against the 49% of Agentic AI users that have not yet updated their governance frameworks, and the gap is clear. An Agent that closes an alert incorrectly may repeat that decision across similar cases, potentially acting on flawed evidence or reasoning before the error is identified.

As autonomy increases, so does the importance of defining where an Agent must stop, escalate, or seek human approval. Detection, containment, and override cannot be retrofitted; they have to be designed in.

"What happens when the model gets it wrong? Every AI gets things wrong sometimes. The question is whether the system is designed to surface those cases, explain them, and allow them to be corrected in a governed way."

Ben Rayner

What Evidence Should Exist When an AI Decision Goes Wrong?

When an investigator, auditor, or regulator challenges a decision, the institution has to reconstruct it.

This is where explainable AI proves its worth, not as an abstract principle, but as an operational capability. Policies, thresholds, and models all change over time, so the record has to show what was true at the moment the decision was made, not how the system works today.

Without that, an institution can describe its AI in general terms but cannot evidence why it acted as it did in a specific case.

"You need to know what data the model saw, what it concluded, why it concluded that in plain language, who reviewed or overrode it, and when. You need to be able to recreate the event, not just provide a log file."

Ben Rayner

Treat an Exception as Feedback, Not as a Kill Switch 

What separates institutions that handle exceptions well is acceptance. They recognise from the outset that errors will happen, build processes to identify and manage them, and use each one as feedback to improve the model.

That does not mean every error can be resolved without interrupting automation. Some failures warrant an immediate pause, escalation, or temporary restriction. The difference is whether those interventions are governed, proportionate, and designed to support a safe return to operation.

Without that process, institutions risk treating every exception as a reason to disable functionality altogether, returning to the same manual workloads without addressing the underlying cause or learning from what went wrong.

Responding to Failure Is The Real Test

For compliance leaders evaluating AI, this changes the conversation with vendors. Rather than enquiring about how often a system is right, ask about cases it got wrong: how were they detected, did the system flag its own uncertainty, was the decision explained, who overrode it and how the lesson fed back into the model.

The strongest evidence of production readiness is an imperfect case that can be fully reconstructed. Compliance AI earns trust not by promising perfect decisions, but by proving that mistakes can be detected, explained, and corrected.

Silent Eight's Iris 7 applies Agentic AI to financial crime compliance decisions, with explainability built into each one. 

Every outcome can be traced to the data considered, the policy applied, and the reasoning behind it in plain language, so compliance teams can review, challenge, and override decisions within a governed process.