ach

How it is built

Architecture overview

The Automated Clearing House is a deferred net settlement system for bulk payments: payrolls, bill payments, direct debits and account-to-account transfers. Banks do not send one payment at a time. They send files, each holding batches of credit and debit entries. The operator (FedACH or The Clearing House's EPN in the US, EBA Clearing's STEP2 for SEPA in Europe) validates each file, sorts all entries by receiving bank, delivers them in processing windows, and settles only the net position of every participant in central-bank money. A bank that sent a million entries and received nearly as many settles one number. Cheap clearing of huge volume is the point; the price is latency (hours to days) and a weaker kind of finality, because an entry can be returned after it has settled.

Two systems share the name and must not be confused. The operator is a closed, high-assurance batch engine run by a central bank or bank consortium. The participant side (originating bank, payment processor, payroll provider) builds files, tracks every entry through its lifecycle, and reconciles against what the operator reports. Most engineers touch the second one, so this page covers both, in that order of detail.

Parts:

• File intake — receive, authenticate and validate files in the fixed Nacha record format (ISO 20022 messages in SEPA) — fixed-width-record-parsing (no page yet), iso-20022-messages (no page yet), Mutual TLS, Ed25519 signatures, CRC32C checksums
• Window scheduler — decides which cycle an entry belongs to, when cut-offs and settlements happen — cron-scheduling (no page yet), business-day-calendars (no page yet), calendar-date-arithmetic (no page yet)
• Sorter and distributor — groups entries by receiving institution, builds outgoing files — External merge sort, k-way-merge (no page yet), pdqsort, Buffer pools and zero-alloc I/O
• Netting and settlement — computes net positions and posts them — multilateral-netting (no page yet), hash-aggregation (no page yet), real-time-gross-settlement (no page yet), Double-entry ledger, gapless-sequence-numbers (no page yet), Money representation
• Entry lifecycle (participant side) — each entry moves from created to settled or returned — Payment state machine, Hierarchical state machine, Idempotency keys, Event sourcing, Exponential backoff with jitter, Optimistic concurrency control
• Batch runner — long jobs that restart without doubling work — batch-checkpoint-restart (no page yet), Write-ahead log, fsync and direct I/O, Leases, Fencing tokens
• Reconciliation — compares operator reports, bank statements and the internal ledger — Reconciliation, Hash join
• Signing and keys — authenticate senders, protect transport keys — Hardware security modules, aes-key-wrap (no page yet), dual-control-approval (no page yet), Secure memory zeroization, csprng (no page yet)

 originator ──entries──► ODFI (originating bank) ── file (batches of entries) ──► operator (FedACH / EPN / STEP2)
                              ▲                                                     │ validate, dedupe, assign window
                              │ returns, NOCs                                       ▼
                              │                                          sort entries by RDFI (external merge sort)
                       entry lifecycle                                              │
                       (state machine)                                  ┌───────────┴───────────┐
                              ▲                                         ▼                       ▼
                              │                                  files to each RDFI      net positions per participant
                       reconciliation ◄── statements ──┐         (receiving bank)        (multilateral netting)
                              ▲                        │                │                       │
                              │                        │                ▼                       ▼
                      internal ledger ◄─────────── RDFI posts to accounts      settlement at central bank / settlement bank
                                                       └── exceptions ──► return entries in a later window

The shape is a pipeline over files: ingest, validate, sort, net, settle, deliver, with controls (counts, hash totals, dollar totals) checked at every hand-off. A file that does not add up to its own control record is rejected whole; a number that cannot be explained is worse than a late payment.

How the parts fit together

File intake

An ACH file is fixed-width text: records of 94 characters, in a strict sequence of file header, batch header, entry details with optional addenda, batch control and file control. The records carry the routing numbers of originating and receiving banks, an account number, a transaction code (credit or debit, checking or savings) and the amount in integer cents (Money representation). Parsing by offset (fixed-width-record-parsing, no page yet) is simple and fast; the work is the rules around it. Moov's open-source library documents the record sequence and field rules (see Sources).

Control totals do the integrity work. Each batch control record repeats the entry count, the sum of debits and credits, and an entry hash, the sum of the receiving banks' eight-digit routing numbers (check digit left out), keeping the ten low-order digits. A parser that recomputes these catches truncation, a dropped line or a corrupted field. The same job on the transport layer uses a checksum such as CRC32C for blocks, but the payment-level totals are the ones that count. Trace numbers (an originating-bank prefix plus a sequence) make each entry unique across the network, which is what duplicate detection and returns depend on.

Files arrive over authenticated channels: mutual TLS between institution and operator, so each connection carries a verified, rotatable identity, and in some set-ups signatures over the file itself so the content can be proven later. In the SEPA world the same intake validates ISO 20022 messages (iso-20022-messages, no page yet) instead of the fixed record format; the structure is XML but the principle of "validate fully, reject whole, report why" is the same.

Window scheduler

Entries are not processed on arrival. The operator publishes several processing windows per business day, each with a cut-off for submission and a settlement time. An entry's effective date decides which window it joins, and weekends and holidays move dates. This is a calendar problem, not a clock problem: business-day-calendars (no page yet) with adjustment rules, calendar-date-arithmetic (no page yet) for time zones and month ends, and cron-scheduling (no page yet) with catch-up for a window that started late. Bugs here are quiet and expensive: a payroll that misses a window arrives a day late with no error.

Sorter and distributor

The operator receives entries from many originators, but each receiving bank needs only its own, in one file. This is a group-by on the receiving routing number over data that exceeds memory, so it is an external merge sort: sort memory-sized runs (with pdqsort in memory), then merge them with k-way-merge (no page yet), writing sequentially. Buffer pools keep aligned I/O buffers recycled instead of allocated per record, which matters because the job is bound by sequential disk and network throughput, not by CPU. The contract with the next stage: output is ordered by receiving bank and, within it, by a stable key, so a re-run produces byte-identical files.

Netting and settlement

multilateral-netting (no page yet) is the reason ACH is cheap. While the sorter streams entries it accumulates, per participant and per window, total credits and total debits by hash-aggregation (no page yet): a hash table keyed by participant, a counter pair per key. The net position of a bank is credits received minus debits sent plus the reverse, in integer cents. The sum of all net positions is zero, which is the self-check: if it is not, something was dropped.

Settlement then moves the net amounts between accounts at the central bank or a settlement bank, posted as a double-entry ledger transaction whose postings sum to zero; journal entries get gapless-sequence-numbers (no page yet) so auditors can prove nothing was removed. The alternative is real-time-gross-settlement (no page yet): settle each payment individually against prefunded balances. RTGS needs liquidity for every payment; netting needs liquidity only for the net, at the price of credit exposure until settlement. Real-time retail rails such as FedNow are built on the gross model.

Entry lifecycle (participant side)

Each entry lives through a persisted payment state machine: created, batched, submitted, accepted by the operator, settled, and then, rarely, returned. Hierarchical states fit the real structure: "submitted" holds "awaiting acknowledgement" and "acknowledged". Transitions are events in an append-only log (Event sourcing), which also gives the audit trail a regulator asks for.

Submitting twice is the worst mistake, since the second copy debits customers again. So the file submission and every entry carry an idempotency key (the file's identity, the trace number), the state machine refuses a second submit, and retries after a timeout use exponential backoff with jitter and first ask the operator whether the file was received. Optimistic concurrency guards the state rows when two workers see the same acknowledgement.

Returns

Debits are the risky direction. A receiving bank can reject an entry for a reason such as insufficient funds, closed account or no such account, and a consumer can dispute an unauthorized debit for a much longer period than a business payment. A return is a new entry in a later window that references the original trace number, so it runs through the whole pipeline again and must be matched to its original (by trace number, then amount and date). The lifecycle therefore has a long tail: "settled" is not "final". Funds-availability, hold and limit decisions are a business-risk layer on top of this tail.

Batch runner

A processing window is a deadline-bound batch job: millions of entries, hours of runtime, and a partner who is waiting at the other end. The pattern is batch-checkpoint-restart (no page yet): split the job into steps with persisted progress, make every step idempotent, and on restart resume from the last checkpoint, never from the start. The checkpoint record is a write-ahead log entry written before the step's effect and flushed with the right fsync discipline, so after a crash the runner knows whether the file was already sent. A lease with a fencing token ensures only one runner owns the job, so a node that was merely slow cannot overwrite the successor's work. Atomic counters shared across threads (entry counts, running totals) rely on the rules of memory ordering.

Reconciliation

Three views of the same day must agree: the operator's reports (what was accepted, settled, returned), the bank statement from the central or settlement bank, and the internal ledger. Reconciliation matches them by reference, then amount and date, and surfaces breaks. For high volume the match is a hash join between statement lines and ledger entries on the trace or reference number. Names on returns and exceptions are free text and often mangled, so the matching of those uses fuzzy-string-matching (no page yet) as a last resort after exact keys fail.

Signing and keys

Keys for transport, file signing and operator authentication sit in HSMs; the application asks to sign or encrypt and never holds the key. Changes to routing-sensitive configuration use dual-control-approval (no page yet): one person proposes, another approves. Secrets that must pass through process memory are erased with zeroization, keys are wrapped for storage (aes-key-wrap, no page yet), and random values come from the operating system's generator (csprng, no page yet).

Data and control flow

Entry from origination to settlement:

1. The originator's instruction reaches its bank (the ODFI). The bank creates an entry with a trace number, amount in cents, and the receiver's routing and account number, in state created.
2. A window job collects entries by effective date into batches by company and type, writes a file, computes control totals, and moves entries to batched. The file is staged, not yet sent.
3. The job submits the file over the authenticated channel before the cut-off; the entries become submitted.
4. The operator validates format and control totals, checks for a duplicate file, and either acknowledges or rejects the file whole. Acceptance moves entries to accepted.
5. At the window, the operator sorts entries by receiving bank, accumulates the credits and debits per participant, builds the per-bank output files and the net positions.
6. At the settlement time, net positions are posted to settlement accounts. Entries are settled in the operator's view. The receiving banks get their files and post to customer accounts.
7. The originating bank's reconciliation matches the operator's settlement report against its ledger. The time goes mostly to sort and I/O at the operator, and to the waiting between cut-off and settlement for the participant.

A return: the receiving bank builds a return entry that references the original trace number and a reason code, and sends it in its own file in the next available window. The originating bank receives it, matches it to the original, moves that entry to returned and books a reversing posting in the ledger. The customer is notified and any hold or limit decision follows from the reason.

A restart after a failure: the runner starts, finds the checkpoint of the last completed step in its log, checks with the operator whether the file for that window was received (by file identity), and continues from the first step not recorded as done. If the file was received it does not send it again.

Scaling

The order in which limits bite:

1. Window deadlines. Files must be built, signed and delivered before the cut-off. Lever: build files incrementally during the day (small batches, merged at the end), and start the final job early with a hard stop for late entries that roll to the next window.
2. Sort and I/O throughput. The operator's sort over all entries is sequential-I/O bound. Levers: external merge sort with large runs and buffer reuse, partitioning by receiving-bank prefix across machines, fast local disks, and avoiding per-record allocation.
3. Validation CPU. Parsing, checksum and hash-total computation are linear passes. Levers: parse by fixed offset without allocation, vectorized sums (SIMD), parallel workers per file.
4. Reconciliation volume. Statement and ledger rows grow with every payment. Levers: hash joins on keys, incremental matching of new data, sharded ledgers.
5. Latency. Batch ACH cannot be made instant by tuning. Lever: more windows per day (same-day processing), or route urgent payments to a real-time rail and keep ACH for bulk.

Failure modes

Real implementations

There is no open-source ACH operator; the operator side is described by its rules, and the open-source work sits on the participant side. Rules for the US network are published by Nacha as the Nacha Operating Rules.

Sources

• Nacha, Nacha Operating Rules and Guidelines: formats, return reasons and time frames, same-day processing. rules
• Nacha, ACH Network volume and value statistics. stats
• Federal Reserve Financial Services, ACH services (FedACH, FedACH SameDay). FedACH · Federal Reserve overview
• The Clearing House, Electronic Payments Network (EPN). EPN
• EBA Clearing, STEP2. STEP2
• Moov, ACH file structure (record sequence, control records, hash totals). docs · code