marketplace-payouts

How it is built

Architecture overview

A marketplace takes one payment from a buyer and owes money to several parties: the seller or driver or host, the platform's own fee, sometimes a courier, a tax authority or a referrer. The system that does this has three jobs. It accepts and settles the buyer's payment through a payment processor. It splits the amount into exact shares and records who is owed what. And it pays out each party to a bank account or card on a schedule, after handling refunds, disputes and holds that can reverse earlier shares. The money moves through real banks that are slow, can fail, and settle on business days, so the heart of the design is bookkeeping that is correct under retries and crashes.

The idea that makes it work is a double-entry ledger with every business event as an idempotent, replayable posting. Amounts are integers in minor units; every posting sums to zero per currency; balances are derived from entries, never overwritten. A payment, a split, a refund, a chargeback and a payout are each a small state machine whose transitions write ledger entries in the same transaction as the state change, and whose external effects (call the processor, send a bank transfer, notify the seller) are triggered from the same transaction through an outbox. If anything crashes, retrying the step either finds the work done (same idempotency key) or finishes it.

Parts:

• Money model — integer amounts, currency exponents, exact splits and rounding — Money representation, largest-remainder-rounding (no page yet), currency-cross-rate-triangulation (no page yet)
• Ledger — accounts per party, append-only entries, balances as sums — Double-entry ledger, Event sourcing, Optimistic concurrency control
• Payment lifecycle — authorise, capture, refund, dispute as persisted states — Payment state machine, Hierarchical state machine, Idempotency keys
• Event plumbing — publish state changes reliably, consume processor events — Transactional outbox, Change data capture, Event deduplication window, Webhook delivery, Incremental sync cursors
• Payout engine — schedules, holds, retries, bank files — Durable workflow execution, Durable job queue, Hierarchical timing wheel, business-day-calendars (no page yet), iso-20022-messages (no page yet), multilateral-netting (no page yet)
• Reconciliation — match ledger against processor and bank statements — Reconciliation, Hash join, hash-aggregation (no page yet)
• Protection — secrets, bank details, partner connections, abuse limits — Envelope encryption, AEAD (AES-GCM, ChaCha20-Poly1305), Mutual TLS, Sliding-window counters, Circuit breaker
• Storage and recovery — durable database and rewind — Write-ahead log, Point-in-time recovery, Primary-backup replication, Leases, Fencing tokens
• Verification — find ordering bugs before money does — Deterministic simulation testing, Coverage-guided fuzzing
• Operations — keeps the services running and updatable without losing money events — Process supervision, Graceful draining, hdr-histogram (no page yet)

 buyer ► checkout ► payment service ──► PSP / acquirer      ◄── webhooks (signed, retried)
                         │ state machine, idempotency key            │
                         ▼                                           ▼
                 DB transaction: state row + ledger entries + outbox row
                         │ outbox relay / CDC (at least once)
                         ▼
        ┌──── event stream ────┬───────────────┬──────────────┐
        ▼                      ▼               ▼              ▼
   split engine           payout engine     notifier      analytics
   (shares → ledger)   (schedule, holds,    (sellers)
                        bank instruction)
                            │ bank file / API ──► bank rails ──► statement
                            ▼                                      │
                 reconciliation: ledger ⟷ PSP report ⟷ bank statement ► exceptions queue

How the parts fit together

Money model: integers and exact splits

All amounts are integers in minor units with an explicit currency exponent, as in Money representation: cents for USD, no decimals for JPY, three decimals for some dinar currencies. Arithmetic is done in a wider integer and rounded once with a named mode. A split is the first place this matters. Taking 12.5 percent of 1,003 minor units and 87.5 percent of the rest gives fractions; shares must sum to exactly the total. largest-remainder-rounding (no page yet) gives each party the floor of its exact share and hands the leftover units, one each, to the largest fractional remainders, with a deterministic tie-break (party id) so a replay gives the same answer. The platform's fee is computed as an explicit share, never as "whatever is left", so rounding is attributed. Cross-currency payouts add currency-cross-rate-triangulation (no page yet): rates quoted against a base currency are inverted and combined with stated precision, and the applied rate is stored on the posting.

Ledger: the source of truth for who is owed what

A Double-entry ledger is an append-only journal of transactions whose postings sum to zero per currency. Each party has accounts (pending, available, reserve), the platform has fee and clearing accounts, and the PSP has a receivable account that mirrors what the processor holds. A buyer payment posts: debit PSP receivable, credit seller pending, credit platform fee. Money cannot appear or vanish because an unbalanced transaction is rejected. This is Event sourcing in a restricted and well-understood shape: immutable events, state as a fold, and a correction is a new entry that reverses the old one, never an edit. Two operations racing on the same account (a payout and a refund) are serialised with Optimistic concurrency control: read the account version, compute, write with an expected-version check, and retry on conflict. For very high rates a dedicated ledger engine does the same with batching.

Platforms that use a processor's marketplace product mirror this. Stripe Connect documents that the platform account and each connected account hold their own balances with pending and available amounts, that a platform can also hold a Connect reserved balance used to cover negative balances of connected accounts, and that moving funds between platform and connected balances is not retried automatically on failure (source: Stripe docs, Connect account balances). The caller's own ledger therefore has to record the attempt and own the retry.

Payment lifecycle: unknown is a state

Payment state machine persists each payment's state (created, authorised, captured, voided, refunded, disputed). Its key rule is that a timeout is not a failure but an unknown, resolved by re-sending the same request with the same key or by querying the processor. Idempotency keys is the contract: the client generates a unique key per logical operation, the server stores a request fingerprint and the final response, and a retry replays the first result. Stripe's API documents the same behaviour: it saves the status code and body of the first request made with a given key and returns them on retries, keys are up to 255 characters, and for API v1 keys may be pruned after at least 24 hours (source: Stripe API reference, idempotent requests). The same fingerprint check catches a key reused with different parameters. A Hierarchical state machine keeps the transition table small: a shared parent state ("settled") holds the reactions common to captured, partially refunded and disputed children, and a child overrides only what differs.

Event plumbing: state and messages must not diverge

After a capture commits, other services (split, notifier, payout) must hear about it. Writing to the database and then publishing to a queue can lose or duplicate the message across a crash. The Transactional outbox writes the event as a row in the same local transaction as the state change; a relay publishes it at least once afterwards. The relay is either a poller or Change data capture, which reads the database's own commit log so events come out in commit order with a resume position. Consumers see duplicates, so they apply Event deduplication window or make handling idempotent by event id. Incremental sync cursors lets a consumer, or a seller-facing API, ask "everything after this token" and be cheap regardless of history. Inbound, processors call back by Webhook delivery: signed POSTs, 2xx means done, retries with jitter, and endpoints that must be idempotent; Stripe documents signature verification of the raw body against an endpoint secret (source: Stripe docs, webhooks). Webhooks may arrive out of order or twice, so handlers compare against the stored state and ignore stale transitions.

Payout engine: long, slow, and must not run twice

A payout is a workflow: check the seller's available balance and holds, apply the schedule, create the bank instruction, wait for the bank, mark paid or reverse. It spans days, so Durable workflow execution fits: workflow code is logged and replayed after crashes, activities (call bank API, post ledger entry) run at least once and are idempotent, and timers are durable. The engine underneath is a Durable job queue with leases, backoff and a dead-letter path for payouts that keep failing (wrong account, closed account) and need a human. Scheduling thousands of "pay this seller on Friday" timers uses a Hierarchical timing wheel for O(1) insert and cancel. Dates come from business-day-calendars (no page yet): holidays, the "following" or "modified following" adjustment rule, and settlement cycles (a transfer started Friday evening arrives Monday or Tuesday). Schedules are a product feature as well: Stripe documents automatic payouts to connected accounts configured by interval (for example weekly on named days), a minimum balance per currency, and a settlement delay in days (source: Stripe docs, manage payout schedule).

For bank instructions between institutions, iso-20022-messages (no page yet) covers the XML message families (the graph names pacs, pain, camt, sese). When many parties owe each other, multilateral-netting (no page yet) collapses obligations into one net amount per participant per currency before anything moves, which cuts bank fees and exposure.

Only one payout run may be active per account. A leader that pauses (a long GC, a network split) can wake up and write after a new leader took over; Leases gives time-bounded ownership and Fencing tokens lets the database reject the old holder's late writes.

Reconciliation: trust but verify

The ledger says what should have happened; the PSP report and the bank statement say what did. Reconciliation matches records by reference first, then by amount and a date window; unmatched or mismatched items become exceptions for a person. Matching millions of rows is a Hash join on the reference key; summaries by day and currency use hash-aggregation (no page yet). Run it daily; an exception queue that is never empty is itself a finding.

Protection

Bank account numbers and identity documents are encrypted per record: Envelope encryption wraps a data key under a KMS key, so rotating the key re-wraps small values and deleting a key shreds data; the cipher is an AEAD (AES-GCM, ChaCha20-Poly1305) that also detects tampering. Connections to partners that offer client certificates use Mutual TLS. Velocity limits (payout attempts per seller per hour, onboarding attempts per IP) use Sliding-window counters. A Circuit breaker in front of each PSP and bank client stops piling timeouts onto a failing dependency and lets traffic route to a second processor.

Storage and recovery

The ledger database is the system. Its durability rests on the Write-ahead log: fsync before acknowledging. Primary-backup replication with a synchronous standby limits what a crash can lose, at the cost of write latency; the alternatives are quorum replication and consensus. Point-in-time recovery rebuilds state as of an instant, which is how a bad deploy that posted wrong entries is investigated, though corrections are still made by new entries and not by rewinding history.

Verification

Bugs here are interleavings: a webhook arriving between two steps of a payout, a crash after the bank call but before the ledger write. Deterministic simulation testing runs the services in one thread with seeded scheduling and injected faults (the graph lists fault injection as a nested technique), so every failure replays from its seed. Coverage-guided fuzzing targets the parsers of partner files and webhook bodies.

Operations

Process supervision keeps the API, the processor-event consumers, the outbox publisher and the payout workers running and restarts them when they exit; a restarted worker resumes from persisted state, never from memory. Graceful draining is how they are deployed: a node stops accepting new payment requests and new webhook deliveries, finishes in-flight postings and bank calls (or leaves them in a persisted, resumable state), and tells the load balancer to send traffic elsewhere, so a deploy cannot cut a payout between the bank call and the ledger write. hdr-histogram (no page yet) records latency of the payment and refund paths and the delay between a processor event arriving and its ledger posting as percentiles; a growing tail on the posting delay is the early sign that consumers are falling behind.

Data and control flow

Flow 1: a purchase with a split

1. Checkout creates a payment with an idempotency key; the payment row starts in created.
2. The service calls the PSP with the same key. A timeout leaves the payment unknown and a reconcile job asks the PSP for the outcome.
3. On authorisation and capture, one DB transaction updates the state, posts ledger entries (PSP receivable debit; seller pending, courier pending and platform fee credits, computed by largest remainder) and writes outbox rows.
4. The relay publishes events; the notifier tells the seller; the split engine has already written exact shares in step 3.
5. After the dispute or holding window, a release job moves the seller's share from pending to available with a ledger transaction.

Flow 2: a weekly payout

1. A timer fires for the seller's schedule and starts a workflow with a deterministic id (seller, period), so a duplicate start is a no-op.
2. An activity computes payable: available balance minus holds and reserve, rounded to minor units, above the minimum.
3. It posts a ledger transaction moving funds from seller available to a payout-in-flight account, and records a bank instruction in the same transaction.
4. The bank instruction is sent through the outbox; the workflow waits for the rail's confirmation with a timeout.
5. On confirmation it posts payout-in-flight to cash; on failure it reverses the earlier posting and flags the seller account.
6. Reconciliation later matches the transfer against the bank statement.

Flow 3: refund after payout

1. A refund request is validated against the original payment and remaining refundable amount.
2. Ledger: debit seller available (or create a negative balance), credit PSP receivable; the PSP refund call uses an idempotency key derived from the refund id.
3. If the seller's balance is negative, later payouts net against it, or the platform covers it from a reserve; Stripe describes a reserved balance for this purpose on the platform side.

Scaling

1. Correctness before throughput. Single-writer-per-account ordering and idempotency are limits that matter at one transaction per second as much as at a thousand.
2. Hot accounts. The platform's fee account takes a posting per payment; batch or shard it into sub-accounts and sum, rather than serialising every payment on one row.
3. Database writes. Group commit; a dedicated ledger store for very high rates; read replicas for seller dashboards.
4. Payout fan-out. Timers and activities are cheap; the limit is the bank API rate. Queue per rail with rate limits and backoff.
5. Reconciliation volume. Partition by day and PSP; hash join keyed by reference; stream rather than load.
6. Regions and currencies. Multiple currencies mean multiple balances per account; cross-border payouts add netting and cut-offs. Keep each currency's ledger separately balanced.

Failure modes

Real implementations

Stripe Connect and Adyen for Platforms are proprietary services of this shape with public documentation: https://docs.stripe.com/connect (account balances, separate charges and transfers, payout schedules) and https://docs.adyen.com/platforms/ (legal entities, balance accounts, transfer instruments, split payments). The graph lists Uber, PayPal, DoorDash, Airbnb and eBay as users of this kind of system.

Sources

• Stripe Connect documentation: https://docs.stripe.com/connect ; account balances https://docs.stripe.com/connect/account-balances ; separate charges and transfers https://docs.stripe.com/connect/separate-charges-and-transfers ; payout schedules https://docs.stripe.com/connect/manage-payout-schedule
• Stripe API idempotent requests, https://docs.stripe.com/api/idempotent_requests ; webhooks, https://docs.stripe.com/webhooks
• Adyen for Platforms documentation, https://docs.adyen.com/platforms/
• Transactional outbox pattern, https://microservices.io/patterns/data/transactional-outbox.html
• PostgreSQL logical decoding, https://www.postgresql.org/docs/current/logicaldecoding.html (streaming row changes from the write-ahead log); continuous archiving and point-in-time recovery, https://www.postgresql.org/docs/current/continuous-archiving.html
• Repositories listed in the table; licences read from each LICENSE file because the GitHub API was rate-limited.