cpu-core

How it is built

Architecture overview

A modern CPU core is a machine for finding independent work inside a single instruction stream. The front end guesses where the program goes next and delivers several decoded micro-operations (µops) per cycle. The rename stage gives every result a fresh physical register, so only true data dependencies remain. The µops wait in schedulers until their inputs are ready and then run on whichever execution unit is free, in any order. A reorder buffer (ROB) puts the results back into program order at retirement, so the outside world still sees one instruction at a time and an exception or a wrong guess can be undone. The load-store unit and the L1 cache are the part of this machine that touches memory, and they are the hardest part to get right. The one idea that ties the design together is speculation with in-order commitment: run ahead on a guess and throw the work away if it was wrong. Intel Golden Cove, AMD Zen 5 and Apple Firestorm differ in widths and buffer sizes, not in this shape.

Parts:

• Front end — fetch, decode and µop cache feeding the back end — instruction-decode-uop-cache (no page yet), Branch prediction
• Rename and allocate — physical registers, move elimination, dispatch into schedulers and ROB — tomasulo-out-of-order (no page yet), instruction-pipelining (no page yet)
• Schedulers and execution units — integer, branch, address-generation and vector pipes — SIMD programming, Floating-point error, arithmetic-circuits (no page yet)
• Reorder buffer and retirement — in-order commit, precise exceptions, flush on a wrong guess — tomasulo-out-of-order (no page yet)
• Load-store unit — load and store queues, store forwarding, memory dependence prediction — load-store-queue-disambiguation (no page yet), Memory ordering
• Caches, TLB and prefetchers — L1 and L2 next to the core, translation, hardware prefetch — CPU caches and the memory hierarchy, Virtual memory and the TLB, hardware-prefetching (no page yet)
• Coherence and chip fabric — keeping per-core caches consistent over a ring or mesh — cache-coherence-protocols (no page yet), network-on-chip (no page yet), False sharing
• Memory controller — DRAM channels behind the last-level cache — dram-organization (no page yet)
• Security and power — side channels from speculation, frequency and sleep control — microarchitectural-side-channels (no page yet), power-management-dvfs (no page yet)

        ┌───────────────────────────── one core ─────────────────────────────────┐
        │  branch predictor ──► fetch ──► decode / µop cache ──► µop queue       │
        │        ▲ redirect                                          │ N per cycle│
        │        │                                                   ▼            │
        │        │                              rename + allocate (physical regs) │
        │        │                                  │                 │           │
        │   branch units ◄── schedulers ◄───────────┘                 ▼           │
        │        │      ALU │ AGU │ FP/vector pipes            reorder buffer     │
        │        │              │                                ▲  (in-order      │
        │        ▼              ▼                                │   retire)      │
        │   resolve guess   load queue ◄─► store queue ─ forward─┘                 │
        │                       │                                                  │
        │                  L1 data cache ◄─► DTLB          L1 instruction cache    │
        └───────────────────────┬──────────────────────────────┬─────────────────┘
                                ▼                              ▼
                       private L2 ── shared last-level cache ── memory controller ── DRAM
                                       (coherence over ring/mesh to other cores)

How the parts fit together

Front end and branch prediction

The front end has one job: never let the back end starve. It predicts the next fetch address before the previous instruction is even decoded, so the branch predictor runs ahead of the decoders. A wrong prediction is only found when the branch executes, deep in the back end, and everything fetched after it is discarded. Agner Fog puts the cost of a misprediction at roughly the length of the pipeline, which is why predictor accuracy matters more with every widening of the core. Branch prediction covers the software side; the seam here is that the predictor must redirect fetch within a cycle or two, so real designs are tiered: a small target buffer that redirects in one cycle, and bigger, slower predictors that correct it later. SonicBOOM, the open core described below, shows this in a design small enough to read: a single-cycle micro-BTB in front of a TAGE predictor, with the return-address stack and loop predictor snapshotted and repaired after a misprediction. The repaired stack made about 10 times fewer mispredicts and predicted 98 % of ret instructions correctly.

Decoding is the second half. x86 instructions have any length from 1 to 15 bytes, so finding instruction boundaries is serial work, and the decoders were for years the narrowest part of the machine. Agner reports that earlier Intel and AMD cores fetched and decoded at most 16 bytes or 4 instructions per cycle, and that Zen 5 breaks this limit with two decoders that each fetch 32 bytes per cycle. Intel's Golden Cove widened to 32 bytes per cycle and six decoders, per the Intel manual. The shared remedy is the µop cache: keep already decoded µops and replay them from there. Agner measures a 6144-entry µop cache on Zen 5 that can serve most critical loops; Golden Cove's cache delivers up to eight µops per cycle. Zen 5 and Firestorm also fuse a compare with the following conditional jump into one µop.

The ISA decides how much decode machinery you need: SonicBOOM pays for RISC-V compressed 2-byte instructions, which make every 2-byte parcel a possible instruction start. Moving scheduling into the compiler instead (vliw-static-scheduling (no page yet)) is the opposite trade-off to the one described next.

Rename, scheduling and retirement

The core of the design is the Tomasulo scheme (tomasulo-out-of-order (no page yet)): rename registers, hold µops in reservation stations or schedulers until their operands arrive, execute out of order, retire in order. Agner's guide shows why renaming matters: a write to a register gets a new temporary register, so two unrelated pieces of code that both use eax do not wait on each other. Only a true read-after-write dependency (a dependency chain) forces order. Tomasulo's 1967 paper is the origin of the scheme; Smith and Pleszkun's 1988 paper covers keeping interrupts precise in such a pipeline, which the reorder buffer does.

Renaming also removes work. Golden Cove's allocation stage performs move elimination and zero idioms (a xor r,r costs no execution unit). Firestorm's tables list mov x0, x1 as usually handled in rename and a zero-immediate move as free. Zen 5 has a stack engine that renames the stack pointer so that push and pop have zero latency on it.

The back end is wide and many-ported. Zen 5 has six integer ALUs, four address-generation units, three branch units and four vector ALUs, per Agner. Golden Cove goes from 10 to 12 execution ports, 6-wide allocation and 8-wide retirement. Firestorm sustains 8 µops per cycle through rename and exposes 14 units, with a ROB of about 330 coalesced entries and about 380 integer physical registers; these are black-box estimates, not vendor numbers. With so many units, Agner concludes for Zen 5 that latencies, and hence long dependency chains, limit real code. The ROB size sets the window: how far ahead the core can look for independent work.

Load-store unit

Loads and stores are the one place where the out-of-order engine cannot rename away the dependency, because memory addresses are only known after the address-generation µop executes. The load-store unit (load-store-queue- disambiguation (no page yet)) holds all in-flight loads and stores in program order, checks each executing load against older stores, forwards data from a matching store when it can, and otherwise lets the load go to the cache. Letting a load run before an older store's address is known is a bet; when it fails, the core flushes. Golden Cove adds partial store forwarding (a load can take data from a store that covers only part of it when the offsets match) and raises load bandwidth from two to three per cycle. Firestorm's measured queues hold about 130 loads and about 60 stores in flight.

This is also where Memory ordering becomes hardware: the load-store unit must obey the ISA's ordering rules while loads reorder freely underneath. The SonicBOOM authors call the load-store unit and data cache the most bug-prone parts of an out-of-order core and built dedicated stress and trace-checking tools for them. Their two-bank L1 data cache issues two loads or one store per cycle without dual-ported SRAM, and a miss fills a line-fill buffer first so an eviction can run in parallel.

Caches, TLB and prefetching

Every load and store first needs a translation (the DTLB) and then a cache hit. The L1 and L2 caches are private; CPU caches and the memory hierarchy gives the sizes and latencies, and Virtual memory and the TLB the reach of the TLB. Golden Cove has a 48 KB L1 data cache and a mid-level cache of 1.25 MB on client and 2 MB on server parts; its load DTLB for 4 KB pages grew from 64 to 96 entries and it allows four page walks in parallel instead of two.

Out-of-order execution hides a miss only as far as the ROB reaches. SonicBOOM's authors do the arithmetic: an L3 hit of the order of 50 cycles on a 4-wide core needs about 200 instructions in flight, the whole reorder buffer. So the cache side adds hardware prefetchers (hardware-prefetching (no page yet)) that bring lines in before the load issues, and non-blocking caches with many outstanding misses: Golden Cove raises L1 fill buffers to 16 and the mid-level-cache miss queue from 32 to 48, again per Intel.

Coherence, interconnect and memory

When a chip has many cores, each private cache can hold a stale copy of a line another core just changed. The cores run a coherence protocol (cache-coherence-protocols (no page yet)) over a ring or mesh (network-on-chip (no page yet)): a write must invalidate other copies, so a line written by two cores bounces between them. False sharing is this cost appearing in code that did not intend to share. On Zen 5, Agner notes, communication inside a group of eight cores is faster than across groups. Behind the last-level cache sit the memory controllers and DRAM (dram-organization (no page yet)).

Security and power

Speculation leaks. Spectre and Meltdown showed that wrongly speculated instructions leave traces in the cache state, and that an attacker can read them through timing, even though the architectural results were discarded. The defences (microarchitectural-side-channels (no page yet)) stop speculation across a boundary or keep speculative effects out of shared state; SonicBOOM's authors sketch writing refills into the L1 only once the load is known to be on the correct path. The core also runs under a power budget (power-management-dvfs (no page yet)): Agner notes that Zen 5 gates idle execution units and varies frequency with load and temperature.

Data and control flow

1. One instruction, normal case. The branch predictor supplies a fetch address; the instruction cache and decoders (or the µop cache) deliver a group of µops to the µop queue. Rename assigns physical registers, allocates a ROB entry and, for memory operations, a load or store queue entry, and dispatches the µop to a scheduler. When its operands are ready the scheduler issues it to a unit. The result is written to the physical register file and broadcast to waiting µops. The ROB marks the µop complete. At the head of the ROB it retires: the architectural state is updated and, for a store, the data is released toward the cache.
2. A branch mispredicts. The branch unit finds that the predicted direction or target was wrong. Fetch is redirected to the right address, the younger µops in the ROB, schedulers and queues are squashed, and the rename map is restored to its state at the branch. The front end then refills the pipeline. Intel's top-down method files these slots under Bad Speculation.
3. A load misses all the way to DRAM. The load issues, misses the L1 and L2, and waits. Younger independent µops keep executing until the ROB or a queue fills; then allocation stalls. Intel's top-down method counts those slots as Backend Bound, memory bound. Prefetchers and non-blocking caches try to start the refill earlier.

Scaling

• Width and window. Widen allocation and execution and enlarge the ROB, schedulers and register files together. Each alone is pointless: wide issue with a small window finds nothing to issue. Generations move in steps (Golden Cove 5→6 wide allocation).
• Specialised units. Wider vector units (Zen 5 has 512-bit paths on desktop and server, half on mobile) and matrix engines such as Intel's AMX cut work per instruction.
• Threads. Simultaneous multithreading feeds one core from two instruction streams. Per Agner, each thread gets half the fetch and queue bandwidth, and caches, predictors and execution units are shared. It helps when one thread cannot fill the machine and hurts when decode or cache is the limit.
• Hybrid cores and more cores. Alder Lake pairs Golden Cove performance cores with Gracemont efficiency cores (Intel manual). More cores add coherence and interconnect cost.

Failure modes

Real implementations

Sources

• A. Fog, The microarchitecture of Intel, AMD, and VIA CPUs, optimization guide, last updated 2026-05-23: out-of-order execution, renaming, branch prediction, Zen 5 pipeline, SMT. pdf
• Intel, Intel 64 and IA-32 Architectures Optimization Reference Manual: Volume 1, 248966-051: Golden Cove, top-down method. manual (the vendor site answers 403 to the generic link-check user agent; it loads with a browser or default curl)
• D. Johnson, Firestorm Overview, Apple microarchitecture research, black-box measurements. page
• J. Zhao, B. Korpan, A. Gonzalez, K. Asanovic, SonicBOOM: The 3rd Generation Berkeley Out-of-Order Machine, CARRV 2020. paper
• BOOM documentation. docs
• R. M. Tomasulo, An Efficient Algorithm for Exploiting Multiple Arithmetic Units, IBM J. Res. Dev., 1967. DOI
• J. E. Smith and A. R. Pleszkun, Implementing precise interrupts in pipelined processors, IEEE Trans. Computers, 1988. DOI
• A. Yasin, A Top-Down method for performance analysis and counters architecture, ISPASS 2014. DOI
• P. Kocher et al., Spectre Attacks: Exploiting Speculative Execution, 2018. arXiv, site
• M. Lipp et al., Meltdown, 2018. arXiv, site
• Amazon Web Services, AWS Graviton. page