mining-pool

How it is built

Architecture overview

A single ASIC rarely finds a Bitcoin block, so miners join a pool. The pool builds a block template from a full node, cuts it into jobs, hands them to every connected miner, and counts shares: hashes that miss the network target but meet a lower one, which prove real work. A share is a statistical receipt. The pool pays miners in proportion to shares and keeps the block reward when one hash meets the full target. The system has five jobs: a job distribution edge speaking Stratum, a template builder talking to the node, a share validator, an accounting core that turns shares into balances and payouts, and a monitoring side watching hash rate and hardware. The one idea: validation is cheap and exact (one hash and a Merkle fold), so everything else is bookkeeping over a stream of small, idempotent events.

Parts:

• Job distribution — Stratum connections, job broadcast, per-miner difficulty — stratum-job-distribution (no page yet), Pub/sub fan-out, Noise protocol (no page yet), JSON-RPC (no page yet), Backpressure (no page yet)
• Template builder — block templates and the coinbase transaction — block-template-construction (no page yet), Merkle tree (no page yet), Nakamoto consensus (no page yet)
• Share validator — checks each share against the share and network target — Proof-of-work challenge (no page yet), Cryptographic hashes (SHA-2, BLAKE3), share-difficulty-adjustment (no page yet)
• Edge protection — limits connection and submit floods — Distributed rate limiting
• Payout accounting — shares to balances to on-chain payments — pool-payout-schemes (no page yet), Double-entry ledger, Idempotency keys, Durable job queue, ECDSA signatures
• Fleet monitoring — hash rate, worker health, alerts — Exponential smoothing (ETS, Holt-Winters), Time-series chunk storage, Telemetry sampling (no page yet), Alert grouping (no page yet), HDR histogram (no page yet)
• Operations — keep the pool up and recoverable — Process supervision, Graceful draining, Primary-backup replication, Point-in-time recovery, Deterministic simulation testing, Coverage-guided fuzzing, Mutual TLS

 Bitcoin node ──getblocktemplate / templates──► template builder ──job──► job distributor ──notify──► ASIC miners
   ▲  submitblock                                (coinbase, merkle path)    (Stratum, fan-out)         │  (hash headers)
   │                                                                              ▲ set difficulty      │ submit share
   │                                                                              │                    ▼
   └── block found ◄── share validator ◄── rate limit / backpressure ◄────────────┴──────────── edge connections
                            │ accepted shares (job, worker, difficulty)
                            ▼
              share log ─► round/window accounting ─► double-entry ledger ─► payout queue ─► signed transactions
                            │                                    ▲ idempotency keys
                            ▼
   hash-rate estimators ─► time-series store ─► alert grouping ─► operators / miners' dashboards

How the parts fit together

Job distribution

Miners speak Stratum. Version 1 is newline-delimited JSON-RPC over TCP (JSON-RPC, no page yet): the pool sends mining.notify (job id, previous block hash, the two halves of the coinbase transaction, the Merkle branch, version, difficulty bits, time, a clean-jobs flag) and mining.set_difficulty; the miner answers with mining.submit carrying worker name, job id, ExtraNonce2, time and nonce (source: Bitcoin wiki, Stratum mining protocol). Version rolling (varying header version bits in hardware) is an extension negotiated with mining.configure (BIP 310). The V2 authors cite V1's several dialects as a motivation.

Stratum V2 is the binary replacement. It is specified as a protocol with Noise-based authenticated encryption (Noise protocol, no page yet): a Noise_NX handshake over secp256k1 with ChaCha20-Poly1305, and a pool authority key that signs the server's certificate (source: SV2 spec, Protocol Security). It drops mining.subscribe, supports header-only mining so the miner never touches the coinbase, allows several channels on one connection, and notes that share messages are in the range of 20 bytes, so aggregating channels in a proxy saves header overhead. Two optional companion protocols move work selection to the miner: Template Distribution (the node pushes templates, replacing polled getblocktemplate) and Job Declaration (the miner declares its own transaction set, and the pool only accounts shares and pays) (source: SV2 spec). The spec allows translating to and from V1 at a proxy.

Pub/sub fan-out is the heart of the broadcast. A new job is the same bytes for every miner on the same extranonce layout, so the pool serialises it once per variant and writes it to every connection. The cost spikes on every new block, when every connection must hear within a fraction of a second, because work on the old tip is wasted. Backpressure (no page yet) protects the other direction: if a connection's send buffer is full, coalesce notifications (only the newest job matters) instead of queueing without bound, and if the validator is behind, slow reading at the edge rather than buffering submits.

stratum-job-distribution (no page yet) is the design that ties these together: job ids, how long a job stays valid, what clean_jobs means for in-flight shares, and extranonce partitioning so no two miners search the same space.

Template builder

block-template-construction (no page yet) turns node state into a job. The pool asks Bitcoin Core for a template with getblocktemplate (BIP 22, with the BIP 23 extensions), which returns the transactions to include, the previous block hash, the target bits, the coinbase value (subsidy plus fees) and the segwit witness commitment; it can also validate a proposed block in "proposal" mode (source: BIP 22, BIP 23). The pool then builds the coinbase transaction: block height in the input script (BIP 34), a pool tag, an extranonce area that the miners will vary, and outputs that pay the reward out (to the pool, or directly to miners for non-custodial schemes) plus the witness commitment output. The Merkle root of the block is computed from the coinbase transaction id and the other transaction ids. The coinbase is the first leaf, so a job needs only its Merkle path and the coinbase split in two pieces; the miner inserts its extranonce between them and re-folds the path (Merkle tree, no page yet).

The template must be fresh in two ways. On a new block the pool rebuilds and pushes immediately. When the mempool changes it refreshes periodically to include better-paying transactions, trading fees against churning every miner. This follows from the network rule (Nakamoto consensus, no page yet): the longest valid chain wins, so extending the tip fastest matters, and a reorganisation can orphan a block the pool found.

If the pool pays out of block rewards it must wait for coinbase maturity, 100 confirmations, before spending them (Bitcoin consensus rule).

Share validator

Each submission goes through the same checks, in the order of cheapness:

1. The job id exists and is not too old; the extranonce2 has the agreed length; time is within bounds; the version bits are inside the negotiated mask.
2. It is not a duplicate: the tuple (job, extranonce2, time, nonce, version) is looked up in a set kept per job.
3. Rebuild the coinbase from the two halves plus extranonce1 and extranonce2, hash it, fold the Merkle path to get the Merkle root, assemble the 80-byte header and compute its double SHA-256 (Cryptographic hashes (SHA-2, BLAKE3)).
4. Compare the hash, read as a 256-bit number, to the share target: at or below it is a valid share. Compare to the network target: at or below it is a block, which goes to the node immediately with submitblock before any bookkeeping.

That is a few hashes per share, so a CPU pool can serve silicon miners. Proof-of-work challenge (no page yet) is the rule being checked: find a header whose hash is below a target; a difficulty-1 share takes about 2^32 hashes on average, since the difficulty-1 target is about 2^224.

share-difficulty-adjustment (no page yet) sets the share target per connection. The pool wants a steady share rate per miner: too few and hash-rate estimates and payouts are noisy, too many and the pool spends its time validating. Difficulty is retargeted from the observed share rate (vardiff), sent as mining.set_difficulty in V1 or SetTarget in V2. Each share is credited at the difficulty of the job it was issued under, and shares at the previous difficulty are accepted for a short grace period after a retarget, or honest in-flight shares get rejected.

Shares for a job superseded by a new block are stale: normally rejected or counted separately. Paying a fraction for stale shares caused by the pool's own latency is a payout policy, not a validation rule.

Edge protection

A pool listens on public ports. Distributed rate limiting caps new connections and failed authorisations per address and subnet across all edge nodes, and submits per connection; the invalid-share ratio finds broken or malicious devices. Limits need a burst allowance so a mass reconnect after a restart is not mistaken for an attack.

Payout accounting

pool-payout-schemes (no page yet) chooses how risk is split. The classic analysis of the options is Rosenfeld (2011):

• Proportional and PPLNS: reward from actual blocks only, divided by shares in a round (proportional) or in a sliding window of the last N shares (PPLNS). The miner carries the luck risk; PPLNS removes the incentive to hop between pools.
• PPS and FPPS: a fixed payment per share, equal to the expected value (share difficulty over network difficulty, times the block subsidy; FPPS adds an estimate of fees). The pool carries the luck risk and needs reserves.
• Direct-from-coinbase schemes such as Ocean's TIDES pay miners in the block's coinbase outputs, so the pool never holds the money (source: ocean.xyz).

Whichever is chosen, money needs a ledger. Every share batch posts to a Double-entry ledger: shares earned credit the miner's account and debit the pool's liability account, so the books balance, and a fee, reserve or payout is another pair of entries. Amounts are integer satoshi, never floats. Since a scheme like PPS is a function from (share, difficulty) to an amount, the ledger can be rebuilt from the share log.

A payout is the dangerous step: a retry must never pay twice. Each payout carries an Idempotency key derived from (miner, period); a Durable job queue holds the batch, a crashed worker's job is retried, and the ledger entry and queue record are written together. The wallet signs with ECDSA keys (Bitcoin's older scheme; newer outputs use Schnorr, BIP 340, which Stratum V2 also uses for certificates), ideally in an isolated signer that approves only transactions matching ledger entries. Batch many payees per transaction and keep a minimum balance so dust is not paid.

Fleet monitoring

Miners want hash rate, which cannot be read, only inferred: expected hashes per share times shares in a window, over the window length. Shares arrive randomly, so a one-minute estimate is noisy; the pool keeps several windows (for example 1 minute, 1 hour, 1 day) with exponential smoothing, constant time per share and no event window kept. The estimates feed rollups in a Time-series chunk store: per-worker series in compressed chunks, with coarser rollups for long ranges.

Telemetry sampling (no page yet) bounds cost: aggregate every share, keep per-share detail for a sample. Alert grouping (no page yet) matters when a hall loses power: thousands of workers go offline in a minute and the owner should get one alert for the site. Latency of the validate-and-ack path goes into an HDR histogram (no page yet) so tail latency shows, because slow acknowledgements cause stale shares.

Operations

Services run under Process supervision; a restart drains connections with Graceful draining (V2 has Reconnect messages for this). The share log and ledger sit on a Primary-backup replicated database with Point-in-time recovery, so a bad deployment can be rolled back. Internal links use Mutual TLS. Public-facing Stratum parsers are fuzzed, and accounting is tested under Deterministic simulation testing: simulated miners, a simulated node that finds blocks and reorganises, injected crashes, all replayed from a seed.

Data and control flow

One share, end to end:

1. A miner hashes headers for a job; a hash meets its target; it sends mining.submit.
2. The edge reads the frame, applies the rate limit and hands it to a validator.
3. The validator runs the checks above. If the hash meets the network target it calls submitblock first.
4. It writes the accepted share (worker, job, difficulty, time) to the share log and answers the miner. Latency here decides how many shares go stale.
5. An aggregator adds the difficulty to the worker's counters and estimators.
6. At the end of the round or window, accounting posts ledger entries.

One new block on the network:

1. The node learns of a block and notifies the template builder.
2. The builder gets a template, builds the coinbase and Merkle path, and creates a job with the clean-jobs flag.
3. The job is serialised once per layout and fanned out; old jobs are retired.
4. In-flight shares for old jobs are classified stale. The pool measures time from block seen to last miner notified.

One payout run:

1. A scheduled job selects accounts above the minimum balance.
2. It writes a batch record with idempotency keys and enqueues one job per transaction.
3. A worker builds the transaction, the signer checks it against the ledger, the node broadcasts it.
4. On confirmation the ledger moves the amounts from "payable" to "paid"; failures are retried with the same key.

Scaling

In the order they bite:

1. Connections and share rate. Shares per second equal hash rate divided by (share difficulty times 2^32), so the main lever is higher per-miner difficulty: fewer shares, less validation, a noisier hash-rate estimate (offset by longer smoothing). Then shard connections over edge processes.
2. New-block fan-out. A latency spike per block. Levers: pre-serialised jobs, one writer per connection group, a local template push from the node, regional edges.
3. Validation cost. Linear in shares; add validators behind the edge. V2 header-only mining removes the coinbase rebuild for most miners.
4. Write volume. Append shares to a log, batch counters per worker and minute, post ledger entries per round, never one ledger row per share.
5. Fleet telemetry. Sampling, rollups and grouped alerts keep monitoring bounded as workers grow.

Failure modes

Real implementations

The alternative: solo mining against your own node: no pool and no accounting, but a small miner may wait years for a block. Pooled-solo schemes like DATUM keep your own templates and still share rewards.

Sources

• Bitcoin wiki, Stratum mining protocol (V1 messages): en.bitcoin.it.
• BIP 310, Stratum protocol extensions (version rolling): BIP 310.
• Stratum V2 specification, sections on design goals, security, mining, job declaration and template distribution: sv2-spec; overview at stratumprotocol.org.
• BIP 22 and BIP 23, getblocktemplate and submitblock: BIP 22, BIP 23.
• M. Rosenfeld, Analysis of Bitcoin Pooled Mining Reward Systems, 2011. paper
• S. Nakamoto, Bitcoin: A Peer-to-Peer Electronic Cash System, 2008. paper
• Bitcoin wiki, Difficulty: en.bitcoin.it.
• Ocean, reward system TIDES and non-custodial payouts: ocean.xyz.