Expertise

Blog

What Your Screening Dashboard Isn’t Measuring

Alert volume is easy to count, which makes it tempting to treat as proof that screening works. False-positive reduction matters, but it does not show whether relevant risks are detected, whether the right records are screened, or whether decisions can be defended. To know whether screening has improved, institutions need to link efficiency measures to evidence about coverage, detection, and decision quality.

Why can alert reduction give an incomplete picture?

Alert volume is an operational measure. It tells a team how many cases its screening process produced, but not why that number changed or whether the change improved risk control.

A fall in alerts may mean irrelevant matches are being resolved more effectively. It may also follow a change in thresholds, data, lists, or screening scope. Reported without those details, alert reduction can reward a smaller queue without showing whether screening still catches relevant cases.

Institutions should keep measuring alert reduction, but as one part of performance, with a clear view of the evidence behind the result.

What does the FCA’s testing show about the limits of headline metrics?

The UK Financial Conduct Authority’s (FCA) 2026 review shows why outcomes need testing across different matching conditions. In its testing, 90% of alerts raised for exact name matches correctly identified the sanctioned party. For names with slight spelling variations, the figure was 75%. Both results describe specific test conditions, not overall system accuracy [1].

The FCA also found problems with titles, long names, non-Latin scripts, and vessel identifiers. For metric design, this means a single average can hide weaker performance on the variations that are harder to match. An efficiency claim should sit beside testing that shows how detection performs across relevant name types and risk scenarios.

In a separate case, a data-feed failure left a set of transactions unscreened for an extended period. The FCA also found errors or omissions in sanctions lists supplied by third-party vendors. A metric that counts alerts only after records enter the screening process would show neither.

Alert counts need to be read alongside evidence about matching performance and the process that produces them.

How can screening scope make a metric misleading?

A screening system can only assess the information it receives. If required records, list updates, or ownership details are missing, a lower alert count may reflect incomplete inputs rather than more efficient decisions.

In its final report on restrictive-measures guidelines, the European Banking Authority (EBA) said supervisors had found outdated or incorrect lists, weak calibration, incomplete screening scope, and poor-quality customer or beneficial-owner data. Supervisors also told the EBA that technical screening alone was not enough to ensure compliance [2].

Alert performance therefore depends on what reached the system. To interpret a fall in alerts, institutions should also track the share of required records screened, failed or bypassed feeds, list-update timing, missing fields, and ownership coverage.

These measures show whether an alert-volume result applies to the full population the control is meant to screen.

Why is missed risk difficult to measure?

A missed-risk rate can show whether screening detects relevant cases, but only if the institution has a credible way to find the cases the system did not flag. Production outcomes are often incomplete evidence, because teams usually investigate only the alerts the system generates.

Independently labelled test cases, retrospective reviews, and external intelligence can all strengthen this evidence. Institutions should explain how cases were selected and what risks the test set represents. Without a sound denominator, a precise-looking missed-risk figure can create false confidence.

Research shows why paired measures matter. A September 2025 Federal Reserve Board staff working paper found that, under its test conditions, large language models cut false positives by an average of 92% and increased detection by 11% compared with the strongest fuzzy-matching baseline. They were also over four orders of magnitude slower on average, so the researchers proposed a tiered approach for uncertain cases [3].

Detection and efficiency can improve together. Only well-defined testing can show whether they did.

Which metrics should institutions assess together?

No single measure shows whether screening has become more efficient and remains effective. Institutions should link each operational result to evidence about the relevant part of the control:

  1. Alert reduction and detection testing: Did the system cut unnecessary alerts while holding performance on representative, independently labelled cases?

  2. Automation or solve rate and quality review: How often do sampled decisions hold up under review, and how often are they overturned?

  3. Handling time and case timeliness: Did faster decisions reduce backlogs and support timely escalation, or only speed up closure?

  4. Precision and screening coverage: Are fewer alerts a sign of better matching, and were all required records and lists included?

  5. Decision consistency and evidence quality: Do similar cases get similar outcomes, with a record of the data, policy, and rationale behind each one?

Pairing metrics this way links workload changes to control outcomes. The US Office of Foreign Assets Control’s sanctions compliance guidance recommends comprehensive, independent, and objective testing, including checks that screening appropriately flags transactions for review [4]. The Wolfsberg Group’s screening guidance supports risk-relevant metrics, documented rationale, and independent, risk-based testing [5]. Neither sets out a universal scorecard.

A useful dashboard connects efficiency claims to tested coverage, detection, and decision quality. Adding more metrics will not do that on its own.

What should a screening performance claim show?

A screening performance claim should define its metric and denominator, identify the population and period measured, explain what changed, and state how detection and coverage were validated. It should also show the operational result, such as the effect on workload, backlogs, or escalation timeliness.

False-positive reduction can free analysts for cases that need more judgement and reduce unnecessary friction for legitimate customers. It becomes a meaningful performance result when institutions can show that screening scope stayed intact, relevant risks were still detected, and decisions remained reviewable.

A lower alert count is an efficiency result. Paired evidence shows whether screening got better.

Key takeaways

  • Alert volume measures operational output, not screening effectiveness.

  • Alert reduction means more when paired with detection testing and with coverage and data-quality measures.

  • Missed-risk rates need credible ground truth and a clear denominator.

  • An efficiency claim should state what changed and how the institution validated the result.

  • The aim is to cut unnecessary work while keeping screening effective and explainable.

Share article

Related Posts

Related Posts

Discover how AI is Revolutionising Compliance and Risk Adjudication

Download our latest collateral to stay ahead.


Discover how AI is Revolutionising Compliance and Risk Adjudication

Download our latest collateral to stay ahead.


Discover how AI is Revolutionising Compliance and Risk Adjudication

Download our latest collateral to stay ahead.