aml-monitoring

How it is built

Architecture overview

Anti-money-laundering (AML) transaction monitoring is the system through which a bank or payment company watches its own customers' activity for patterns that suggest laundering, and turns what it finds into cases that a human investigator can close, escalate or report. In the United States a bank that decides activity is suspicious files a Suspicious Activity Report with FinCEN (source: FinCEN, filing information); other jurisdictions have their own reporting offices, and industry bodies such as the Wolfsberg Group publish guidance on the risk-based approach. Laundering is rare but missing it is costly, so the system alerts on far more than is truly illicit, and each alert costs investigator time. Real labels are scarce and biased: only what was alerted gets examined, and the UN estimate of 2 to 5 percent of global GDP laundered each year (cited by Altman et al., 2023) coexists with the fact that many laundering transactions in real data are never detected. The system therefore combines explicit rules that a compliance officer can read and defend, statistics over time windows that capture velocity and change of behaviour, graph analysis that links accounts into networks, and machine learning that ranks alerts so investigators see the likeliest first. The one idea that makes it work: detection is a pipeline over an ordered, replayable event log, and every alert, rule version, model version and human decision is recorded as evidence, so that any decision can be explained and any rule change can be tested on history before it goes live.

Parts:

• Ingestion — gets transactions and customer changes from the core in order — Change data capture, Partitioned commit log
• Enrichment and identity — attaches customer risk, resolves who is who — Entity resolution, fuzzy-string-matching (no page yet)
• Real-time detection — rules and velocity checks at transaction time — Rete rules engine, Sliding-window counters, Windowed stream aggregation
• Name and list screening — checks parties and payment text against sanctions and watchlists — Aho-Corasick multi-pattern matching, fuzzy-string-matching (no page yet)
• Network analysis — links accounts through shared attributes and money paths — Union-find, BFS and DFS, graph-community-detection (no page yet)
• Model scoring — ranks alerts and finds unusual behaviour — Isolation forest, Gradient-boosted decision trees (GBDT)
• Evaluation and tuning — measures whether rules and models are worth their alerts — Evaluation metrics
• Case management and evidence — alert queue, decisions, reports, audit — Tamper-evident audit log

 core banking ──CDC──► commit log (partitioned by customer) ──────────────────────────────┐
 payments, cards                │                                                         │
 KYC changes                    ├─► enrich + entity resolution ─► customer profile store    │ replay
                                │                                     │                   │ for backtests
                                ├─► real-time path: rules (Rete) + window counters ──► alerts
                                ├─► screening: names, narratives vs lists ───────────► alerts
                                └─► batch/near-line: graph links, communities, ML ───► alerts
                                                                                          │
                                       alert store ◄───── dedupe, merge, score ◄──────────┘
                                           │
                                investigator queue ──► case ──► decision: close / escalate / report
                                           │                                   │
                                           └──────► audit log (signed) ◄───────┘
                                       outcomes ──► labels ──► model training and rule tuning

How the parts fit together

Ingestion

Monitoring must see every transaction, in order per customer, and must be able to see it again. Change data capture reads the core system's own commit log into row-change events with a resume position, so monitoring needs no hooks in the posting path and cannot miss a committed transaction. Events land in a Partitioned commit log keyed by customer (or by account owner), because every detection that follows is per-customer state, and a key hash keeps one customer's events ordered while consumers scale out by partition. The log is retained for months: it is what lets a new rule be run against last year's traffic and what lets a crashed consumer resume exactly where it stopped.

The alternative is a nightly batch over the data warehouse: simpler, but a day late and unable to hold an instant payment. Many programs run both: real time for velocity and screening, batch for expensive graph and model scenarios.

Enrichment and identity

An alert is only useful if it concerns the right entity. The same person appears as a customer in two systems, a counterparty typed three different ways, a beneficial owner behind a company. Entity resolution decides which records describe the same real-world party: block records to cut the pair space, score pairs with a probabilistic model, and cluster the matches transitively. The output is a stable party id that every later stage uses, so "this customer sent money to that one" and "these two accounts have the same owner" can be answered at all.

Name matching is approximate: transliteration, nicknames, word order, company suffixes. The block fuzzy-string-matching (no page yet) is what a screening and entity-resolution engine rests on. The enrichment stage also attaches the customer's risk rating, expected activity from onboarding (declared income, business type, countries), and peer group, because most scenarios compare actual behaviour with those expectations.

Real-time detection

Rules. A scenario such as "cash deposits just under a reporting threshold, repeated within a week" or "payment to a high-risk country from a new account" is a rule over facts. A bank has hundreds of them, owned by compliance, changed often and applied to millions of facts per hour. A Rete rules engine caches partial matches in a shared network so each new transaction only does the work it affects, instead of re-evaluating every rule against every fact. A regulator can read a rule, and compliance can change its threshold without a deploy. The open product Jube describes a rule builder that compliance staff can edit (source: jube-home/aml-fraud-transaction-monitoring README); Marble describes transaction monitoring tailored to the business with all data visible to the user (source: checkmarble/marble README).

Velocity. Many scenarios are counts or sums over a recent window: number of transactions to new payees in the last hour, total cash in the last seven days. Sliding-window counters keep a ring of sub-bucket counters per key, which bounds memory per customer and makes reads constant-time. They fit short windows on the hot path.

Aggregates over event time. Longer or cross-entity comparisons ("this customer's monthly outgoing total against their own previous twelve months") need event-time windows with watermarks so late events land in the right window. Windowed stream aggregation groups events per key and window, keeps one partial aggregate per slice and emits a window when the watermark passes its end. The result is written to the customer profile store as features, which both the rules and the models read.

The Rete network and counters keep state, so they sit behind the log: each partition's consumer owns its customers' state and checkpoints its log offset together with the state. Recovery means restoring the checkpoint and replaying from the offset.

Name and list screening

Separate from behaviour, every party and sometimes every payment narrative is checked against sanctions and watchlists. Two problems appear. The first is scanning free text, such as a payment reference or a trade-finance document, for any of tens of thousands of listed names, aliases, vessels or place names. Aho-Corasick multi-pattern matching finds all occurrences of many fixed patterns in one pass, with time independent of the number of patterns, which is the right tool for narrative scanning. The second is party-to-list matching, where names are not identical: this is approximate matching and scoring, with the cutoff a policy choice.

Moov's open Watchman is a worked example: it downloads OFAC and other lists, indexes them in memory, and scores each customer or counterparty from 0 to 1 with a field-level matcher; a matching passport, IMO number, SWIFT/BIC or crypto address scores 1.0 and the cutoff (minMatch) is configured by policy, with debug output for field-level scores (source: moov-io/watchman README). The README reports that on 472,477 labelled records Jaro-Winkler at a minMatch of 0.80 had precision of 0.99; treat such figures as the project's own benchmark on its own data, not a guarantee for yours. Lower cutoffs raise recall and alert volume together.

Network analysis

Individual transactions rarely prove laundering; the shape of a network does. Three structural questions, each a classic graph block.

• Who is linked through shared attributes? Accounts sharing a phone number, address, device or beneficial owner form clusters. Union-find merges the sets as links arrive and answers same-set queries in effectively constant time, so a streaming job can maintain "linked group" ids cheaply.
• Where did the money go? Layering moves funds through hops to hide the origin; fan-out then fan-in is a common shape, and money returning to its origin through intermediaries is a cycle. BFS and DFS give hop distances from a flagged account, reachable-set size within N hops, and cycle detection on the transaction graph.
• Which groups transact mostly among themselves? Tight communities of accounts whose money rarely leaves the group are a typical review target. Block: graph-community-detection (no page yet).

Graph jobs run near-line, on a rolling window of the log, because they scan far more than one event. The output is features and alerts: "two hops from a previously reported account" is a model feature and, in some programs, an alert.

Model scoring

Two model types fit different data.

Unsupervised. Most customers behave like their peers, and a few do not. Isolation forest scores each record by how few random splits it takes to isolate it, so unusual feature vectors (a dormant account suddenly active, a retail account moving company volumes) get high scores without any labels. It finds the unexpected, not necessarily the illicit.

Supervised. Past investigator outcomes become labels (alert closed as false positive, case escalated, report filed). A GBDT model on the profile features ranks new alerts so that the likeliest are seen first, and can suppress the lowest-risk alerts under controlled review. Tabular boosted trees are the sensible default: in the original paper that released the Elliptic bitcoin dataset, a Random Forest outperformed the graph convolutional network that the authors had set out to test (Weber et al., 2019). Two cautions. Labels exist only for what was alerted, so the model inherits the rules' blind spots; keep a reviewed random sample of non-alerted activity to measure what is missing. And the model must be explainable to a regulator: monotonic constraints, feature attributions, reviewed versions.

Real data is not shareable, so Altman et al. (2023) release a synthetic generator and datasets with complete ground-truth labels for comparing models.

Evaluation and tuning

A rule or a model earns its place by what it adds. Evaluation metrics matter more than usual because positives are rare: accuracy is meaningless, and ROC AUC flatters a model on a tiny positive rate. What teams track is precision at the alert budget (of the N alerts investigators can handle per day, how many were worth escalating), recall on known positives, and average precision. Rule changes go through backtesting on the replayed log and a shadow period where the new rule alerts silently, with the results compared to the current rule on both volume and yield.

Case management and evidence

Alerts from all paths flow to a store, which merges duplicates (the same customer from five rules becomes one case with five reasons), applies the model score and assigns a queue. An investigator reviews the case with the customer profile and network view, and records a decision with reasons. Defensibility depends on records: rule and model version, what the investigator saw, the decision, who approved a report. A Tamper-evident audit log signs Merkle roots of those records in batches, so a reviewer or regulator can verify that an entry existed at the time and that history was not rewritten. Decisions feed back as labels and as tuning inputs.

Data and control flow

A transaction through the real-time path

1. The core commits a payment; change data capture writes an event to the commit log, partitioned by customer.
2. The partition's consumer enriches it: party ids from entity resolution, customer risk rating, peer group, the counterparty's country.
3. Counters are updated (payments in the last hour, new payees in the last day) and the transaction and its features are asserted as facts in the Rete network.
4. Rules that match produce alert candidates with reasons; screening of the counterparty name and the payment narrative runs in parallel and may add candidates.
5. The state and the log offset are checkpointed together; the candidates are written to the alert store. Time goes here: a few state reads and writes per event, normally well inside the payment's own latency budget, so a high-risk hit can hold a payment before release.

A weekly investigation cycle

1. A near-line job rebuilds linked-group ids with union-find over the last weeks of shared-attribute links, and runs hop and cycle queries from flagged accounts.
2. The model job joins profile features with graph features and scores every active customer with isolation forest and the supervised model.
3. Alert merging combines the new candidates with the existing queue; scores order it.
4. Investigators work the queue; closure reasons are recorded and signed into the audit log.
5. Analytics compute precision at the daily alert budget per rule and per model version, and propose threshold changes, which are backtested on the log and shadowed before activation.

Scaling

1. Alert volume, not compute, is the first limit: investigators are the scarce resource. Levers: better rule thresholds and segmentation by peer group, alert merging per customer, model ranking with a suppression policy under review, and closing the label loop.
2. State size per partition (counters, windows, profiles) grows with customers. Levers: more partitions, compact counters, keeping long-window aggregates in the profile store instead of in memory.
3. Rule count raises per-event cost. Levers: the Rete network's sharing of conditions, rule groups by customer segment, and removing rules whose yield is zero.
4. Graph jobs are the heaviest. Levers: restrict to a rolling window, incremental union-find rather than full recompute, limit hop depth, and run on a separate cluster off the log.
5. Screening volume rises with every payment and every list update. Levers: partitioned in-memory indexes by name token, a cheap prefilter before expensive scoring, and re-screening only when a list entry or a party changes.

Failure modes

Real implementations

The alternative: a rules-only system (a vendor scenario library with fixed thresholds) is easy to explain and audit and is where most programs start. It produces high alert volume and cannot find patterns nobody wrote down. Adding scoring and graph features does not replace the rules; the rules remain the part a regulator reads, and models decide what to look at first.

Sources

• FinCEN, Bank Secrecy Act filing information (suspicious activity reporting in the United States).
• The Wolfsberg Group, wolfsberg-group.org: industry guidance, including on the risk-based approach.
• M. Weber et al., Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics, 2019: the Elliptic data set and the comparison in which Random Forest outperformed the GCN.
• E. Altman et al., Realistic Synthetic Financial Transactions for Anti-Money Laundering Models, 2023: synthetic generator and datasets; the UN estimate quoted in its abstract.
• Moov Watchman README: github.com/moov-io/watchman (lists, scoring, minMatch, reported benchmark).
• Marble and Jube READMEs: checkmarble/marble, jube-home/aml-fraud-transaction-monitoring.
• OFAC, Sanctions List Service: the published lists that screening engines load.