← Back
Home › Systems › Finance and trading

casino-gaming-system

Collects meter readings from gaming machines and tracks each player's activity and outstanding vouchers so a casino knows exactly what was taken in and paid out.

Light & Wonder, IGT ADVANTAGE, Konami Synkros

Open the casino-gaming-system pagecasino-gaming-system Hdr histogramdata-structures: Log-linear bucketed histograms with bounded relative error for latency percentilesDeterministic simulationtestingdistributed: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.›Durable job queuedistributed: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.›Message sequence numbersdistributed: A per-stream counter on every message lets the receiver detect loss, duplicates and reordering, ack cheaply and resume after reconnect with "send from N".›Primary-backup replicationdistributed: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.›Benefit accumulatorseconomics: Track deductible and out-of-pocket accumulators per member and plan year, applying them in order to split claims into plan and member sharesHot-standby redundancyembedded: A second, always-current controller shadows the primary and takes over within a few cycles of silence; epochs stop the old primary from fighting back.›Double-entry ledgerfinance: Append-only journal of transactions whose postings sum to zero per currency; balances are derived, so money can only move, never appear or vanish.›Idempotency keysfinance: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.›Money representationfinance: Store money as integer minor units with an explicit currency exponent, compute in wider integers, and round once with a named rounding mode.›Payment state machinefinance: Persisted state machine for a payment lifecycle (authorize, capture, void, refund) that treats timeouts as unknown, retries with the same key and applies webhooks idempotently.›Reconciliationfinance: Match internal ledger records against bank and processor statements by reference, then amount and date window; every unmatched or mismatched item becomes an exception.›Process supervisionhardware-os: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.›Resource usage accountinghardware-os: Cgroup/procfs/GPU counters sampled into per-workload usageSas g2s protocolsnetworking: Slot machine communication protocols SAS and G2S: polling, meters, events, vouchersTime decay scoringsearch-sort: Scoring items by recency with exponential decay or half-life, combined with relevanceCode signingsecurity: Signing binaries/apps with certificate chains, entitlements, timestamping and notarizationCoverage-guided fuzzingsecurity: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.›Csprngsecurity: OS-backed cryptographically secure random numbers (getrandom), and when a fast PRNG is not enoughHardware security module (HSM)security: A tamper-resistant device or service that generates keys, signs, decrypts and wraps with them internally, and releases a key only as ciphertext.›Message authentication codes(HMAC, CMAC, Poly1305, KMAC)security: A keyed short tag proving a message is unmodified and came from a key holder; pick HMAC, CMAC, KMAC or a one-time polynomial MAC by what primitive you already have.›Mutual TLSsecurity: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.›Signed update metadata (TUF)security: Separate signing roles with thresholds sign small metadata pinning file hashes, versions and expiry, so clients detect rollback, freeze and mix-and-match and recover by rotating the root.›Tamper-evident audit logsecurity: An append-only log whose Merkle root, signed per batch, lets anyone verify an entry is present and history was never rewritten, with log-size proofs.›Time-series anomaly detectionstatistics: Predict the expected value from a seasonal baseline, scale the residual by a robust spread (MAD), threshold at a chosen false-alarm rate, then merge flags into incidents.›Gapless sequence numbersstorage: Gap-free, monotonic numbering (invoices, journal entries) under concurrency and rollbacksIdentity resolutionstorage: Merge anonymous and known identifiers into one profile (union-find over identity edges)Point-in-time recoverystorage: Periodic base backups plus a continuously archived change log let you rebuild the state as of any instant, then branch from there.›Event condition action rulessystems: Event-condition-action rule engines for automations, edit checks and alertsGraceful drainingsystems: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.›Health checkingsystems: Decide per backend whether to route to it using probes and real-traffic errors, with consecutive-count hysteresis, capped ejection and a panic fallback.›Publish subscribe event bussystems: In-process or on-box topic bus decoupling producers and consumers with bounded queuesCooperative schedulingconcurrency: Green threads/coroutines yielding at await points on a user-space schedulerFault injectiondistributed: Inject latency, errors, kills and partitions via proxies, netem or hooksRecord replaydistributed: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuildDead-letter queuedistributed: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.›Leasesdistributed: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.›Watchdog timerembedded: A hardware down-counter that resets the chip unless software proves, in time and in a plausible pattern, that it is still healthy.›Exponential backoff with jittersystems: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Largest remainder roundingfinance: Splitting an amount into parts that sum exactly (largest remainder / Hamilton) for allocations, splits and prorationsPRNGs (PCG, xoshiro)numerics: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.›Event sourcingfinance: Persist state changes as an append-only stream of immutable events per entity; current state is a fold over them, guarded by expected-version appends.›Hierarchical state machinesystems: Nest states so a substate inherits its parent's reactions and only overrides what differs, with ordered entry and exit actions along the path.›Write-ahead logstorage: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.›+ 28 moreTransparent interceptionnetworking: Redirect traffic to a local proxy (iptables TPROXY/REDIRECT, eBPF) without app changesSyscall interceptionsecurity: Intercept syscalls via ptrace, seccomp user notify or a user-space kernelFencing tokensdistributed: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.›Optimistic concurrency controlstorage: Run transactions without locks, record what they read, and validate at commit that nothing changed; fastest when conflicts are rare.›CRC32C checksumshardware-os: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.›fsync, O_DSYNC and O_DIRECThardware-os: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.›+ 11 moreEbpfhardware-os: Verified in-kernel programs on hooks (kprobes, XDP, tc) with mapsPtrace process controlhardware-os: Ptrace: attach, read and write registers and memory, single-step, intercept signalsSeccomp bpfsecurity: Syscall filtering with seccomp BPF programsSIMD programminghardware-os: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.›+ 16 more+ 10 more+ 1 more

How it is built

Architecture overview

A casino floor holds hundreds to thousands of electronic gaming machines (EGMs) plus tables, and an operator must know, to the cent, what each machine took in and paid out, who played, and which vouchers are outstanding. The casino management system (CMS) is the back-office service that does this: it polls machines for meters and events, accounts for cashless play and ticket-in/ticket-out (TITO) vouchers, tracks players through loyalty cards, runs promotions and bonuses, and reports to the regulator. Commercial examples are Light & Wonder, IGT ADVANTAGE and Konami Synkros; their internals are proprietary.

The central design fact is that money moves through a device that cannot be trusted to stay online, and the books must still balance. So the machine is the source of truth for its own cumulative meters, the host stores every event with a sequence number, and every cash-like movement (a voucher printed, a voucher redeemed, a transfer to the machine) is a small state machine with a unique id that is reconciled afterwards. A second fact shapes the rest: gaming is regulated, so the software that touches money is signed, verifiable and audited, and a change to it needs approval outside the engineering team.

Parts:

 EGM ──RS-232 / SAS──► SMIB / gateway ──┐
 EGM ──Ethernet / G2S─────────────────────┤
 table, kiosk, cage ───────────────────────┤
                                           ▼
                              floor server (poll, parse, sequence, store)
                                           │ events + meters (append-only)
                 ┌─────────────────────────┼───────────────────────────┐
                 ▼                         ▼                           ▼
        ticket / cashless engine    meter accounting             player tracking
        (state machine per ticket,  (deltas per machine,         (card in/out, rating,
         idempotent, reconciled)     ledger, variance checks)     offers, bonuses)
                 │                         │                           │
                 └───────────┬─────────────┴───────────────────────────┘
                             ▼
              database: primary ─► hot standby (WAL)     audit log (hash-chained)
                             │
                             ▼
              regulator reports, finance, anomaly alerts

How the parts fit together

SAS (Slot Accounting System) is described in the public record as the de facto casino communication standard for automating slot meter reporting and event logging, player tracking, bonusing, ticketing and cashless gaming (source: US patent 8,429,229, background section; saspy README). The README of the open-source saspy library adds the shape: the link is a serial line between the EGM and a slot machine interface board (SMIB); the master polls with 1-byte addresses (a general poll returns a 1-byte event code, called an exception) and long polls carry commands, and all long polls except read-type include a CRC-16; the host-to-SMIB side is not standardised by SAS. G2S (Game to System) replaces the SMIB with Ethernet and administers services from G2S servers (source: US patent 8,429,229). Patent literature describes G2S as XML data over TCP/IP. Casinos therefore run a mixed floor for years, and the CMS keeps a protocol adapter per generation (sas-g2s-protocols, no page yet) that emits one internal event schema.

The adapter's contract: every event gets a per-machine sequence number (Message sequence numbers) at the point it enters the host, so a gap, a duplicate or a reorder is detectable; frames are checksummed (CRC-32C checksums is the common fast check for the host's own storage and links, while SAS itself uses CRC-16 on the wire); and the adapter never blocks the poll loop on a database write. Text-like G2S messages and configuration files are parsed with a hand-written, bounded parser (Recursive-descent parsing) and nothing from the floor is trusted to be well formed.

Once an event has its sequence number it is published to a publish-subscribe-event-bus (no page yet) topic per machine group, and the consumers subscribe independently: meter accounting, the voucher engine, player tracking, the compliance log and the floor-monitoring screens. The poll loop therefore does one thing, publish, and a slow consumer (a report job, a promotion rule) falls behind on its own subscription instead of slowing the poll of the machines. Per-machine order is kept (all events of one machine go to one partition), and a consumer that restarts resumes from its stored position and relies on the sequence numbers to skip what it has seen.

Meter accounting: cumulative counters and deltas

An EGM keeps cumulative meters. The host does not trust its own copy and does not add events; it reads meters at defined times, subtracts the previous reading, and stores the delta with the reading time. Events (game end, door open, power loss) are stored as an append-only stream (Event sourcing); the current state of a machine is a fold over its events. Money is held in integer minor units with an explicit currency (Money representation), and day-end figures are posted to a Double-entry ledger, so a bucket never gains value without another losing it. Shares, such as a progressive contribution split across several pools or revenue shared with a venue partner, must sum to the whole amount, which is the case for largest-remainder-rounding (no page yet). The check that catches faults is a variance report: the sum of machine meters against the cage's counts, the vouchers printed against vouchers redeemed, using the matching approach of Reconciliation.

Usage is attributed as well as totalled. resource-usage-accounting (no page yet) takes each metered delta (coin-in, coin-out, games played, time on device) and assigns it to the holder it belongs to: the machine, the open card session, the active promotion, and the progressive pool. Each delta is booked once to each holder with the reading time, so a session's wagers, the time the player was seated and a machine's hold over a shift all add up to the same machine meters, and a delta that arrives with no open session is booked to the machine alone and not guessed onto a player.

Ticket-in/ticket-out and cashless: money as a state machine

A voucher is printed by a machine, held by a person for hours or days, and redeemed at another machine or a kiosk. Each ticket therefore has a lifecycle (issued, redeemable, redeeming, redeemed, expired, voided) modelled as a Payment state machine with persisted states: a timeout between "redeem requested" and the machine's answer is unknown, never failure, and is resolved by asking again. The Hierarchical state machine shape gives shared behaviour (any state can be voided by the cage) without repeating transitions. A redemption request carries an Idempotency key, so the machine retrying after a lost response does not pay twice. A ticket must be redeemed once even if two machines see it concurrently; the check is a conditional update on the ticket's version (Optimistic concurrency control). Where a rule or an auditor wants numbers without gaps (validation or journal numbers), that is gapless-sequence-numbers (no page yet), and number allocation is serialised even though the rest is not.

A voucher's validation number is a bearer secret: whoever presents it at a kiosk can be paid. It therefore comes from a csprng (no page yet), a cryptographically secure random generator seeded by the operating system, with enough bits that guessing a valid number is not feasible, and never from a counter or a clock. The sequential voucher number needed for the journal is a separate field, because a gapless sequence is predictable by design; the same generator makes the nonces for cashless sessions and the keys that are then wrapped under the master key.

Player tracking and promotions

A card inserted at a machine ties the play to a person. The same person may hold several cards, appear under different spellings, or be a guest at several properties of a group, so the loyalty database needs identity-resolution (no page yet): Entity resolution with blocking and scoring to find candidate pairs and Union-find to merge the matches into one profile. A rating uses the recorded wagers and time; promotions are rules of the form "when this event happens and this condition holds, do this" (event-condition-action-rules, no page yet) with timers for expiry and cool-down (Hierarchical timing wheel). Offers weigh recent play more than old play, which time-decay-scoring (no page yet) formalises. Keep promotional credit in separate ledger accounts from cash credit so each can be reported on its own.

benefit-accumulators (no page yet) are the per-player balances that rating and promotions work with: points earned per unit of play, tier credits for the current period, comp value and free-play credit. Each accumulator is updated by an event from the rated session, has its own earn rule, expiry and cap, and is written as a posting against a separate promotional account, so a balance can always be rebuilt from the events. Tiers and offers read the accumulators (with their decay), not the raw wagers, which keeps the promotion rules simple and the rating stable when the earn rate is changed.

Compliance and reporting

Regulators publish technical standards and operators must show that their systems comply. Gaming Laboratories International publishes freely downloadable standards, including GLI-13 (monitoring and control systems and validation systems), GLI-16 (cashless systems and technologies) and GLI-11 (gaming devices) (source: GLI standards page). The system therefore keeps a Tamper-evident audit log of significant events and of every operator change, because an inspector will ask who changed a setting, when and with whose approval. Statistical checks sit beside the rules: Time-series anomaly detection compares each machine's hold to its own seasonal baseline and flags outliers, with Isolation forest as a model-light alternative for multi-feature checks. An alert is an investigation trigger (a faulty meter, a tampered machine, collusion), not a verdict.

Software integrity: signing, keys and dual control

Because game and system software determines payouts, a regulator-facing system verifies what runs. Release artefacts are signed (code-signing, no page yet), and update clients check small signed metadata files that pin versions and hashes with separate roles (Signed update metadata); verification uses a normal signature scheme such as ECDSA signatures or Ed25519. Signing keys live in Hardware security modules, data keys are wrapped under master keys (Envelope encryption) and rotated without downtime (key-rotation, no page yet), and host-to-device messages that need integrity use a keyed tag (Message authentication codes). Back-office connections use Mutual TLS so each gateway and kiosk has its own identity. Sensitive operations such as changing a payout parameter or releasing a software version follow dual-control-approval (no page yet): two named people, recorded.

Reliability: the floor must keep running

A CMS outage must not stop play; machines keep playing and accumulate meters, and tickets can still be printed. The host's job is to catch up without double-counting. The database writes through a Write-ahead log and replicates to a hot standby (Primary-backup replication, Hot-standby redundancy); Point-in-time recovery answers "what was the state at 03:12?" for investigations. Poll workers and report jobs run under Process supervision, drain on deploy (Graceful draining), and jobs such as regulator exports go through a Durable job queue so a crash retries instead of skipping.

Health checking decides what the rest does about a failure: the poll workers and the machine links report liveness per link and per machine group, a stalled link is marked unreachable instead of silently quiet, and the standby takes over on a failed check of the primary together with a lease, as in the section above. A machine that is silent for longer than its poll interval raises an alert and its meters are caught up on reconnect.

hdr-histogram (no page yet) records latency percentiles for the paths where the floor feels delay: poll cycle time per link, event-to-ledger time, ticket redemption round trip (request to answer at the machine) and promotion award time after a rated event. The tail shows a slow serial link or a lock held on a hot voucher row, and merged histograms from all poll workers give the property-wide picture.

Assurance

Deterministic simulation testing replays failover, duplicate and out-of-order scenarios from a seed; this is the right tool for voucher races. Coverage-guided fuzzing targets the protocol parsers, which read bytes from devices on a shared floor.

Data and control flow

One ticket, printed and redeemed:

  1. A player cashes out at machine A. The machine prints a ticket with a validation number and reports the event (ticket out) with the amount.
  2. The host assigns the event a sequence number, creates the ticket in state "issued" with the amount, validation number and machine id, and posts the liability to the ledger.
  3. Hours later the player inserts the ticket in machine B. The machine sends the validation number; the host looks up the ticket and moves it from "issued" to "redeeming" with a conditional update.
  4. The host answers with the amount; machine B credits and reports "ticket in". The host moves the ticket to "redeemed" and clears the liability.
  5. If the host's answer is lost, machine B asks again with the same idempotency key and receives the first answer; if the ticket was in "redeeming" for too long the engine asks the machine and resolves it. Time goes mostly to the lookup and the conditional write; the machine's wait is the user-visible latency.

One day-end meter close:

  1. At the cut-off the poller reads every machine's meters and stores readings with times.
  2. Deltas are computed against the previous close; gaps in the sequence of events are flagged.
  3. Deltas are posted to the ledger; the variance report compares the sum to cage and ticket totals.
  4. Anomaly checks run; exceptions go to a queue for a person. Reports are generated from the closed figures and stored with the audit log entry for the run.

One failover:

  1. The standby loses the primary's heartbeat and promotes.
  2. Pollers reconnect; machines still hold their meters, so the host reads them again and recomputes deltas from the last stored reading.
  3. Retried ticket requests are answered from the idempotency records.

Scaling

In the order you meet the limits:

  1. Poll rate per serial line. SAS uses master-slave polling with one machine at a time per line; the README of saspy gives polling rates from 200 ms to 5 s, and a faster real-time-event mode. Levers: more gateways, fewer addresses per line, real-time event reporting.
  2. Event ingest for large floors. Thousands of machines emit game-end events. Levers: batch writes, a partition by machine, aggregate before storing detail.
  3. Ticket validation latency. The path is a single-row read and a conditional write. Levers: index on validation number, keep hot tickets in memory, a regional database near the floor.
  4. Report and analytics load. Regulator and finance reports are heavy. Levers: run on replicas or a warehouse copy, schedule off-peak, precompute day-end aggregates.
  5. Multi-property groups. Loyalty is shared across sites. Levers: a central identity service with local caches, and local operation when the link is down.
  6. Audit and history growth. Logs are kept for years. Levers: partitions, cold storage in Object storage, anchoring only roots.

Failure modes

FailureSymptomCauseFix
Meter variancemachine total differs from cage and ticket totalsmissed poll, meter rollover, failed eventdelta accounting with sequence gaps flagged, re-read meters, reconcile and document
Double payoutone ticket paid twicerace between two machines, retry without idempotencyconditional update on ticket version, idempotency keys, one redeemer
Stuck ticketticket in "redeeming" for hourshost or machine crashed mid-redemptiontreat as unknown, query the machine, resolve with a timeout rule and an audit entry
Lost events during outagegap in machine historypoller down while the machine kept playingread cumulative meters afterward, store deltas, flag unattributed intervals
Duplicate or merged playerswrong comp, split historycards and names not resolvedentity resolution with review of uncertain merges, split tools
Unsigned or modified softwareregulator audit failure, or tamperingmissing signature check, key leaksigned metadata, HSM-held keys, dual control for release
Replay or spoofed gatewayforged eventsno per-device identity or integrity tagmutual TLS per device, keyed tags, sequence numbers
Parser crashpolling stops on a bad framemalformed bytes from a devicefuzzed bounded parser, per-line isolation, supervised restart
Split-brain after failovertwo hosts accept ticketspromotion without fencingsingle writer with lease or fencing, then restart old primary as standby
Report mismatchregulator figure differs from financereports built from live databuild reports from closed figures, keep inputs and version of the report code

Real implementations

Open-source casino management systems are practically absent; the licensed products are proprietary and regulator-certified. What is public is mostly the protocol side and the standards.

namewhat it isrepolicencenotes
saspyPython library for talking SAS to an EGM over RS-232 on single-board computersthomas-pythonas/saspyMITreads meters and exceptions; its README is a good compact description of the SAS layers
GLI standardsfreely downloadable regulator-facing standards: GLI-11, GLI-13, GLI-16 and othersGLI standards page—documents, not code; versions listed on the page

Proprietary systems named in the graph: Light & Wonder, IGT ADVANTAGE, Konami Synkros. They are not described here.

Sources

  • GLI standards (free downloads, including GLI-11, GLI-13, GLI-16): https://gaminglabs.com/gli-standards/
  • saspy README (SAS layers, polling, exceptions, TITO and AFT functions): https://github.com/thomas-pythonas/saspy/blob/master/README.md
  • US patent 8,429,229, background on SAS, G2S and SMIB architecture: https://patents.google.com/patent/US8429229

Similar systems

Share the most building blocks with this one.

mobile-money 38 %p2p-payments 35 %retirement-recordkeeping 34 %treasury 33 %lottery-system 32 %