← Back
Home › Systems › Payments and commerce

warehouse-management

Turns customer orders into physical warehouse work by deciding which stock to take, who picks it and how it is packed, and handles inbound goods, using an event ledger of inventory movements.

Amazon fulfillment

Open the warehouse-management pagewarehouse-management Partitioned commit logdistributed: A topic split into replicated append-only partitions: key hash keeps per-key order, offsets give position, and a high watermark limits readers to fully replicated records.›Transactional outboxdistributed: Write the event as a row in the same local transaction as the state change; a relay publishes it later at-least-once, so state and events never diverge.›Event sourcingfinance: Persist state changes as an append-only stream of immutable events per entity; current state is a fold over them, guarded by expected-version appends.›Fifo lot consumptionfinance: Consuming quantities from dated lots in FIFO/FEFO order (points expiry, stock rotation)Idempotency keysfinance: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.›Inventory reservationfinance: Reserving stock with TTL holds, release on timeout and no overselling under contentionOrder lifecycle state machinefinance: Order states (new, partial fill, cancel/replace, rejected, expired) with strict transitions and idempotent eventsA* searchgraphs: Dijkstra ordered by cost-so-far plus an admissible estimate of the remaining cost; optimal paths while exploring far fewer nodes toward the goal.›Dijkstra's algorithmgraphs: Single-source shortest paths with non-negative edge weights by settling vertices in order of distance with a priority queue.›k-means clusteringmachine-learning: Partition points into k clusters by alternating nearest-centroid assignment and centroid averaging; k-means++ seeding makes it dependable.›Bin packingoptimization: Pack items of given sizes into the fewest fixed-capacity bins; sort descending, then first-fit, gives at most 11/9 of optimal and runs in O(n log n).›Mixed-integer programming(branch and bound)optimization: Solves LPs with integer variables by solving the LP relaxation, branching on a fractional variable, and pruning subtrees whose LP bound cannot beat the incumbent.›Tsp heuristicsoptimization: Nearest-neighbour, 2-opt and Or-opt tours for picking and visiting ordersMulti agent path findingrobotics: Conflict-based search for collision-free paths of many robots on a gridTime-series forecasting(baselines, model families, ba…statistics: The end-to-end recipe: fix horizon and uncertainty, beat naive baselines, choose a model family, and validate with rolling-origin backtests and scale-free metrics.›Optimistic concurrency controlstorage: Run transactions without locks, record what they read, and validate at commit that nothing changed; fastest when conflicts are rare.›Fast hash functionsdata-structures: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.›Consumer offsetsdistributed: Each consumer group stores one integer per partition, the next position to read, so a restarted consumer resumes where it left off with at-least-once delivery.›Primary-backup replicationdistributed: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.›Segmented logdistributed: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.›Zero-copy I/Onetworking: Move bytes between files, sockets and NICs by passing page references instead of copying through user space: sendfile, splice, MSG_ZEROCOPY, kTLS.›Event deduplication windowdistributed: Keep the ids of recently accepted events for a time window and drop repeats; memory stays bounded by rotating time buckets instead of per-entry timers.›Change data capturestorage: Decode the database's own commit log into row-change events in commit order, with a resume position, so any consumer can mirror the data exactly.›Leasesdistributed: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.›Cryptographic hashes (SHA-2,BLAKE3)security: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.›Exponential backoff with jittersystems: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Conditional writesdistributed: Put-if-absent / if-match compare-and-swap on objects or keys as a commit primitiveHierarchical timing wheelsystems: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.›+ 17 moreHealth checkingsystems: Decide per backend whether to route to it using probes and real-traffic errors, with consecutive-count hysteresis, capped ejection and a panic fallback.›Message sequence numbersdistributed: A per-stream counter on every message lets the receiver detect loss, duplicates and reordering, ack cheaply and resume after reconnect with "send from N".›Fencing tokensdistributed: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.›Write-ahead logstorage: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.›Incremental sync cursorsdistributed: The client keeps one opaque token; each pull returns only the changes after it plus the next token, so sync cost scales with what changed, not with the dataset.›Slab allocatorsystems: Pre-carve pages into equal-size slots per size class with an in-place free list: O(1) alloc and free, no external fragmentation, cache-warm reuse.›+ 11 moreCRC32C checksumshardware-os: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.›fsync, O_DSYNC and O_DIRECThardware-os: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.›Cursor paginationnetworking: Keyset/opaque-cursor pagination for stable incremental reads from APIs and tables+ 4 moreSIMD programminghardware-os: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.›

How it is built

Architecture overview

A warehouse management system (WMS) turns customer orders into physical work: which stock to take from which bin, which picker or robot does it, in what route, how it is packed and handed to a carrier, and how inbound goods are received, put away and counted. It also owns the inventory record, the system's belief about how many units of each item are at each location and in what state (available, reserved, picked, damaged). Retailers, wholesalers, pharmaceutical distributors, spare-parts businesses and fulfilment centres run the same core loop at very different sizes.

The idea that makes it work is that the inventory record is a ledger of events, and every physical step is a task with a small state machine. Stock never changes by "set quantity to 7"; it changes by a movement (received, reserved, picked, moved, adjusted) that is appended once, with an idempotency key, and the current balance is a fold of the movements. Tasks (pick this line, pack that order) move through states that scanners and robots advance, and planners sit on top: they group orders into waves, assign them to workers, and choose routes, but they never mutate stock directly. This split makes the system recoverable (replay the ledger), auditable (every unit has a history) and parallelisable (planners are pure functions of the current state).

Parts:

 ERP / e-shop --orders--> order intake (idempotent) --> order state machine
                                                           | allocate
                                                           v
 receiving --movement--> +-----------------------+  reserve   +----------------+
 scanners  --movement--> | inventory ledger      |<---------->| planner:       |
 robots    --movement--> | (events + balances)   |            | waves, slotting|
                         +-----------+-----------+            +-------+--------+
                                     | outbox -> commit log          | pick tasks
                                     v                               v
                         +-----------------------+            +----------------+
                         | consumers: pick UI,   |<-----------| route + robot  |
                         | carrier, billing, BI  |  tasks     | planner        |
                         +-----------------------+            +----------------+

How the parts fit together

Inventory ledger

Two people scanning the last unit of an item must not both get it. The ledger gives each (item, location, lot, state) a balance, and every change is a movement with an Idempotency keys value (the scan id, or order line id plus step), so a scanner that retries after a dropped Wi-Fi packet produces exactly one movement. Event sourcing stores the movements as the source of truth and derives balances from them, which is what lets an auditor ask why a pallet disappeared.

Reserving stock for an order is inventory-reservation (no page yet): a hold with a time limit that is released on timeout, which prevents unpaid or abandoned orders from locking stock forever; expiry is cheap on a Hierarchical timing wheel. The reservation must be an atomic "if available >= n then decrement", so it is written as a conditional operation, conditional-writes (no page yet), or as an Optimistic concurrency control retry loop on the balance row (read version, check, write if version unchanged). OCC wins when conflicts are rare (a popular SKU is the exception); for the hot SKUs the ledger partitions available quantity into slices per zone or per channel so that writers do not collide. Lot-controlled goods (food, medicine) consume stock by fifo-lot-consumption (no page yet): first-expiry-first-out, so the oldest expiry is picked first and recalls can find exactly which orders received a lot.

The contract with everything else: the ledger exposes append movement and reserve/release operations that are idempotent, and publishes every committed movement exactly once to the event backbone.

Order and task lifecycle

Orders and pick tasks are state machines: new, allocated, released, picking, picked, packed, shipped, with side branches for short picks, cancels and returns. The model is order-lifecycle-state-machine (no page yet); a Hierarchical state machine keeps it manageable because "picking" has substates (assigned, at location, confirmed) that share parent reactions, such as "order cancelled" valid anywhere in the parent. Every transition is triggered by an event and is idempotent, so replaying an event does nothing. The graph lists the FIX protocol because its sequence-numbered, resend-on-gap session model is the same discipline that carrier and customer EDI interfaces need; a warehouse speaks an EDI or API dialect instead, with the same needs: per-direction sequence numbers, acknowledgements and replay.

Event backbone

The ledger, order service, pick UI, carrier integration and analytics must agree without sharing one database. A Partitioned commit log carries events: keyed by order or by item so per-key order holds, stored in a Segmented log with Zero-copy I/O sends, replicated with Primary-backup replication on top of a Write-ahead log, and consumed with Consumer offsets. The graph lists a durable job queue as the alternative to a log; it fits "do this one task once" work, but a log lets a new consumer (a new analytics store) replay history, which warehouses need often.

The service that commits a movement must also publish the event, and doing both in separate steps loses events on a crash. The Transactional outbox writes the event row in the same database transaction as the balance change; Change data capture reads that table from the database log and publishes in order, with Incremental sync cursors for a consumer that needs a snapshot first. The graph names two-phase commit as the alternative; it blocks on a failed coordinator, which a warehouse running 24 hours cannot tolerate. Because delivery is at-least-once, consumers use Event deduplication on event ids; Fast hash functions key partitions and dedup tables.

Planner: waves, slotting, bin packing

The planner has three jobs. Slotting decides where each SKU lives: fast movers near the dock, items often ordered together close to each other; K-means on co-occurrence or location vectors proposes clusters. Wave planning decides which orders to release now, so each wave fits worker capacity, carrier cut-off times and dock doors; at small scale a heap of orders by due time (D-ary heap) is enough, at larger scale it is an assignment model for Mixed-integer programming, where Linear programming (simplex) relaxations, Cutting planes and Presolve reductions do the work. Cartonisation decides which items go into which box: Bin packing (first-fit decreasing is the fast heuristic, with an exact check when a box is nearly full) minimises cartons and carrier charges. The contract with the ledger: the planner reads a snapshot of balances and writes reservations, so its plan is only valid as long as reservations commit; a rejected reservation triggers a re-plan for those lines.

Pick routing

A picker walks a list of locations; order matters because walking is most of the labour (the share varies by site). The route is a travelling-salesman problem on the aisle graph. Distances between locations come from Dijkstra (or precomputed all-pairs for a fixed layout) and BFS and DFS for reachability. tsp-heuristics (no page yet) give a first tour (nearest neighbour, or a tree-based tour from a Minimum spanning tree with minimum-weight-perfect-matching (no page yet) in Christofides style), and local-search-2opt-or-opt (no page yet) improves it by swapping segments, which in a warehouse with 10 to 50 picks per tour runs in well under a second (order of). Many layouts have simple structure (parallel aisles), where a dynamic-programming route beats the generic heuristic. A K-d tree finds the nearest candidate pick to the worker's position when locations are coordinates, and Union-find groups locations into zones or merges aisles into connected components.

Robot traffic

Where mobile robots carry shelves or totes, many robots share aisles and intersections. multi-agent-path-finding (no page yet), for example conflict-based search, finds collision-free paths for all robots; the single-robot search inside it is A*, and reservations of (cell, time interval) pairs are kept in an Interval tree so a new path can ask "is this cell free in this window?". The contract with the task layer: robots receive a task and return a progress event; the traffic planner never changes inventory.

Demand and replenishment

Reorder points depend on forecasts of demand per SKU. Time-series forecasting applies; Exponential smoothing and STL decomposition cover regular items, arima (no page yet) autocorrelated ones, and croston-method (no page yet) the many slow movers with mostly zero days. Time-series cross-validation validates by rolling origin, scored with Evaluation metrics and a suitable Loss functions. Robust statistics keeps one big order from setting the reorder point, and bootstrap-confidence-intervals (no page yet) give a range on safety stock. A Kalman filter can fuse noisy counts (RFID reads, weight sensors) into a stock estimate with a variance. L-BFGS fits the models. The graph also lists Finite differences, Explicit Euler and thomas-tridiagonal-solver (no page yet) for simulating flows such as conveyor throughput; they are off the core path.

Durability

The ledger is the asset. Segment writes are checksummed with CRC32C checksums and committed with the rules from fsync and direct I/O. Failover uses Leases for primary ownership and Fencing tokens so that an old primary that wakes late cannot write; Health checking feeds the decision. Clients retry with Exponential backoff with jitter. Content hashes (Cryptographic hashes (SHA-2, BLAKE3)) fingerprint label and document payloads, and a Slab allocator serves fixed-size in-memory records for the hot balance cache.

Data and control flow

One order, from intake to shipment.

  1. The e-shop sends the order with its id; intake stores it under that id (a repeat returns the stored result).
  2. The state machine moves it to "allocating"; the planner asks the ledger to reserve each line, preferring stock by lot expiry and location. Short stock produces a backorder or split.
  3. Reservations commit; outbox events flow to the log; the pick service creates tasks.
  4. A wave is released; the router orders the picks; the picker's device shows the first location.
  5. Each scan confirms a pick: the ledger appends a "picked" movement (idempotent), the reservation turns into a move to the pack station.
  6. Packing runs cartonisation, prints the label, records the parcel; the carrier integration sends the shipment event.
  7. A "shipped" movement leaves the stock; the order closes. Time is spent in step 4 (human walking) and step 2 (conditional writes on hot items).

One inbound receipt.

  1. A supplier notice creates an expected receipt; at the dock the clerk scans items and lots.
  2. Each scan appends a "received" movement into a staging location with the lot and expiry.
  3. Putaway rules choose a bin by slotting and space (bin packing for pallets); the move is a pair of movements (staging out, bin in), committed together.
  4. Discrepancies against the notice raise a task, not a silent adjustment.

One cycle count.

  1. The system picks locations to count (high value, high movement, long unchecked).
  2. The counter submits the count; if it differs from the balance, an "adjustment" movement with a reason is appended after review.
  3. Adjustments feed back into the slotting and forecasting data.

Scaling

  1. Hot SKUs. The first limit is write contention on a few popular balances. Slice available quantity per zone or channel and rebalance in the background.
  2. Event volume. Scans and robot events grow with throughput. Add partitions (keyed so one order stays in order), batch movements per scan session, and keep consumers separate from the ledger.
  3. Planner cost. MIP and routing grow quickly with size. Plan by zone, use heuristics first, limit time per wave, and keep exact solvers for slotting that runs overnight.
  4. Robot count. Path conflicts grow faster than robot count. Plan in short horizons, split the floor into regions with handoff, and fall back to simple priority rules.
  5. Read load. Dashboards and carrier queries read from replicas built from the log, not from the ledger.
  6. Multi-site. One ledger per site; a global view built from each site's log. Transfers between sites are a pair of movements joined by an in-transit state.

Failure modes

FailureSymptomCauseFix
Phantom stockSystem says 3, shelf has 0; pickers walk to empty binsMissed scans, damage not recorded, misplaced putawayCycle counts; short-pick events that trigger a count; reconcile movements
Double pick or double shipTwo parcels for one lineRetried scan without idempotency; two tasks for one lineIdempotency keys; one task per line; state machine rejects duplicates
Oversold stockOrders accepted that cannot be fulfilledRead-then-write reservation without atomicityConditional reservation; slicing for hot SKUs
Reservation leakStock shows reserved but nothing movesHold without expiry on abandoned or failed ordersTimeouts; periodic sweeper; alert on old reservations
Lost eventCarrier never told about a shipped parcelCommit and publish as separate stepsTransactional outbox; reconciliation job
Duplicate event effectStock decremented twice downstreamAt-least-once delivery, consumer not idempotentEvent ids and deduplication window
Split brainTwo primaries accept writes after a failoverLease expired, old primary kept runningFencing tokens checked by the storage layer
Robot deadlockRobots stop in a cycleMutual blocking at an intersectionReservations with time windows; deadlock detection and a re-plan
Wrong expiry shippedRecall hits stock already sentLot not tracked per movementLot on every movement; FEFO consumption; lot-to-order trace
Slow wave planningPickers idle waiting for the next waveSolver runs to optimality on large instancesTime limit with heuristic fallback; plan next wave while the current runs

Real implementations

Large operators (the Examples line names Amazon fulfilment) run in-house systems that are not public. The open-source items below cover the inventory and order model, and the event and solver building blocks.

namewhat it isrepolicencenotes
OpenBoxesOpen-source warehouse and inventory management for supply chainsopenboxes/openboxesEPL-1.0Ledger-style stock movements, lots and expiry
Odoo (Community)ERP with an Inventory app: warehouses, locations, operation types, cycle counts, reordering rulesodoo/odooLGPL-3.0docs; Enterprise edition is separate
Apache OFBizOpen-source ERP system in Javaapache/ofbiz-frameworkApache-2.0Broad business suite
Apache KafkaDistributed partitioned commit logapache/kafkaApache-2.0Event backbone
DebeziumChange data capture from database logsdebezium/debeziumApache-2.0Outbox relay
OR-ToolsOptimisation suite: MIP, CP-SAT, routing and bin packinggoogle/or-toolsApache-2.0Planner and routing; routing docs
HiGHSLP and MIP solverERGO-Code/HiGHSMITPlanner
StatsForecastStatistical forecasting: automatic ARIMA, ETS, ThetaNixtla/statsforecastApache-2.0Replenishment forecasts

Sources

Similar systems

Share the most building blocks with this one.

loyalty 22 %container-terminal 19 %e-commerce 17 %workflow-engine 17 %virtual-economy 16 %