ultrasound-imaging

How it is built

Architecture overview

An ultrasound scanner makes a picture from echoes. A transducer array of 64 to 256 piezoelectric elements sends a short pulse into the body; tissue boundaries and tiny scatterers reflect it; the same elements record the returning waves as radio-frequency (RF) signals. Software turns those channel signals into a live image: it beamforms them (delays each channel so echoes from one point add up in phase), takes the envelope and compresses its dynamic range into a grey-scale B-mode picture, converts the acquisition geometry to a display grid, and estimates motion from repeated pulses (colour and spectral Doppler). The same chain feeds the screen at tens of frames per second, a cine memory, and an export path to the hospital archive in DICOM format.

Sound travels at about 1540 m/s in soft tissue, the value scanners assume, so depth maps to time: an echo from 15 cm returns after about 195 µs (2 × 0.15 m / 1540 m/s). That sets the pulse repetition rate and, with the number of image lines, the frame rate: 128 focused lines at that depth take about 25 ms, roughly 40 frames per second, before any parallel-beam or plane-wave tricks.

Parts:

• Acquisition sequencer — fires transmit events on an exact schedule and hands raw channel data on — Fixed-rate control loop, SPSC ring buffer, Deterministic simulation testing
• Beamformer — turns channel data into one focused line or pixel at a time — beamforming (no page yet), with spline-interpolation (no page yet) for fractional delays and IIR and FIR filters for the band-pass step, on GPU compute shaders
• B-mode chain — envelope, log compression, speckle reduction, display grid — hilbert-envelope-detection (no page yet), Fast Fourier transform, image-denoising (no page yet), scan-conversion-interpolation (no page yet), Image resampling filters
• Doppler — blood velocity and spectrum from pulse ensembles — doppler-autocorrelation-estimation (no page yet), IIR and FIR filters for the wall filter, Fast Fourier transform for the spectrum
• Image analysis — organ and structure outlines, automatic measurements — image-segmentation (no page yet), Convolutional networks, Weight quantization
• Review, storage and streaming — cine, compression, export, remote viewing — image-codecs (no page yet), video-codecs (no page yet), low-latency-video-encoding (no page yet), dicom-protocol (no page yet), Mutual TLS for the archive and remote-viewing links, Coverage-guided fuzzing of the DICOM reader
• Device software update — safe field updates of the scanner — A/B partition updates, built on Secure boot chain, Watchdog timer, CRC32C checksums and Signed update metadata (TUF)
• Operations — keeps the scanner software running through an exam, measured and cleanly stoppable — hdr-histogram (no page yet), Process supervision, Graceful draining

 transducer ◄── transmit pulses ── sequencer (fixed period, exact deadlines)
     │ echoes (per-channel RF, tens of MS/s)
     ▼
 front end ADC ──► ring buffer ──► beamformer (delay, apodize, sum) ──► I/Q lines
                                        │                                │
                           ┌────────────┴───────────┐                    ▼
                           ▼                        ▼             envelope + log
                     Doppler (ensemble        B-mode chain        compress, denoise
                     autocorrelation, FFT)    scan conversion            │
                           │                        │                    │
                           └─────────► compose colour over grey ◄────────┘
                                                    │
                          ┌─────────────────────────┼───────────────────────┐
                          ▼                         ▼                       ▼
                     screen / UI          segmentation (CNN)       cine ─► codec ─► DICOM / stream

The product splits into a hard-real-time side (firing and capturing, microsecond deadlines, no allocation) and a soft-real-time side (image formation and display at 30 to 60 frames per second). The seam between them is a lock-free buffer.

How the parts fit together

Acquisition sequencer

A scan is a repeating program: transmit event 1 (aperture, focus, angle, waveform), listen for the echo from the deepest depth of interest, then transmit event 2, and so on, then start the frame again. The listening time is fixed by physics (depth divided by the speed of sound), so the schedule must hit exact deadlines; a late pulse changes the frame rate and the Doppler sampling. Fixed-rate control loop gives the pattern: absolute deadlines on a fixed period instead of "sleep for X after the last step", so errors do not accumulate, and a fixed dt for anything derived from the schedule. In a real scanner the tightest part runs in an FPGA or a dedicated real-time core; the host software sets up sequences and receives data.

The sequencer's output is a stream of channel data blocks. SPSC ring buffer is the hand-off to the processing side: one producer (the acquisition or DMA thread) and one consumer (the beamformer thread), wait-free, with the buffer sized for a few frames so a slow display frame does not make the front end overrun. A full buffer is counted and dropped at the frame boundary, never allowed to block acquisition.

Data rate is large. As an example, 128 channels sampled at 40 MS/s with 2 bytes per sample is about 10 GB/s, which is why scanners reduce data close to the front end (sample at lower rates, beamform in hardware, or use fewer channels per event).

The sequencer and its ring buffers are timing-critical and hard to test on a probe, since a missed transmit slot depends on interrupt and scheduling luck. Deterministic simulation testing runs the sequencer, the beamformer queue and a fake transducer in one thread under a seeded scheduler with a simulated clock. Mode changes at frame boundaries, a slow beamformer that lets a ring buffer fill, and a dropped transfer then replay from their seed, so the rule that settings switch only between frames is checked under thousands of orderings.

Beamformer

beamforming (no page yet) (here receive beamforming; the phased-array idea is the same as in radio) computes, for each image point, the time the echo needs to reach each element: the distance from the point to the element divided by the speed of sound, plus the transmit delay. It reads each channel at its own delay and sums the values. Echoes from the focal point add coherently; echoes from other places add with random phase and cancel. Two refinements are standard:

• Apodization: weights across the aperture (a Hann-like window) trade a wider main lobe for lower side lobes, which are what show up as haze around bright reflectors.
• Dynamic receive focusing: the delays are recomputed for every depth, so the whole line is in focus on receive.

Delays are rarely whole samples, so the channel signal needs sub-sample interpolation. spline-interpolation (no page yet) (linear for speed, cubic when the signal is sampled near twice its bandwidth) provides it; the alternative is to upsample once with a polyphase filter and take the nearest sample. Before beamforming, the channel data usually pass a band-pass or matched filter to remove out-of-band noise (IIR and FIR filters; FIR for linear phase, which keeps echo timing exact).

The computation is the same small kernel (gather, scale, sum) for millions of pixels per frame, with no dependence between pixels: a natural fit for GPU compute shaders. One workgroup takes a tile of pixels, loads the needed channel window into shared memory and loops over elements. The cost is memory traffic, not arithmetic, so channel data are kept in the order the gather reads them. SUPRA is an open GPU pipeline from beamforming to B-mode.

Newer modes beamform from unfocused transmissions (plane or diverging waves) and coherently sum several steering angles: frame rates reach hundreds or thousands per second because one transmit event forms the entire image, at the cost of more compute per frame. USTB's Generalized Beamformer covers plane-wave, diverging-wave and focused cases in one pixel-based framework.

B-mode chain

The beamformed line is an RF waveform (a carrier at the transducer frequency, a few MHz, with the echo amplitude as its envelope). hilbert-envelope-detection (no page yet) recovers the envelope: form the analytic signal (the Hilbert transform gives its quadrature part) or demodulate to baseband I/Q, then take the magnitude. The Hilbert step can be done with Fast Fourier transform: transform the line, zero the negative frequencies, inverse transform. Envelope values span a huge range, from a bright bone surface to faint tissue, so a logarithmic compression maps them into the 8-bit display range; the dynamic range shown, commonly some tens of dB, is a user control.

The picture is full of speckle: interference of echoes from sub-resolution scatterers, a granular pattern that is a property of tissue structure and imaging system, not noise in the electronic sense. It hides low-contrast lesions, so scanners reduce it. image-denoising (no page yet) supplies the tools: bilateral or non-local-means filters that smooth within regions and preserve edges, and temporal averaging across frames when the probe is still. Spatial compounding (averaging images from different steering angles) is the acoustic counterpart. Over-smoothing removes texture clinicians read.

Lines are acquired in the transducer's own geometry: straight lines for a linear array, fanned lines in polar (range, angle) for a sector or curvilinear probe. scan-conversion-interpolation (no page yet) resamples this polar grid to the Cartesian display grid, usually bilinear, one lookup of four neighbouring samples per screen pixel with the source coordinates precomputed. The final resize to the window uses a kernel from Image resampling filters; when shrinking, a proper filter avoids aliasing the speckle into moire patterns.

Doppler

Motion shows up as a phase change between consecutive pulses at the same place. doppler-autocorrelation-estimation (no page yet) transmits an ensemble of pulses along one direction, takes the I/Q sample at each depth, and estimates the lag-one autocorrelation along the pulse (slow-time) axis. The phase of that value gives the mean Doppler frequency, hence velocity along the beam: v = c · f_d / (2 · f₀ · cos θ), where f₀ is the transmit frequency and θ the angle between beam and flow. This is the autocorrelation approach introduced for blood-flow imaging by Kasai et al. (1985); it is cheap enough to run for every pixel in a colour box. Velocities beyond ±PRF/2 alias, so scanners expose the velocity scale (PRF) as a control and offer baseline shift.

Tissue and vessel walls move far more slowly than blood but much more strongly, so a wall filter (high-pass in slow time, IIR for short ensembles with a good initialisation, or FIR where the ensemble is long) removes this clutter before estimation (IIR and FIR filters). For spectral Doppler the system takes samples from one range gate and computes a short-time spectrum with the Fast Fourier transform, showing velocity against time as a scrolling display and usually playing the audio signal too.

Colour is composed on top of the grey image: pixels where the Doppler power exceeds a threshold and the B-mode echo is weak get colour; others keep grey. Both modes share the transmit budget, which is why frame rate falls when the colour box is large.

Image analysis

Measurements (organ size, fetal biometry, ejection fraction) used to be manual calipers. image-segmentation (no page yet) gives them automatically: a Convolutional networks model outputs a mask per frame, from which areas, diameters and volumes are computed. Ultrasound is hard for segmentation: speckle, shadowing, variable probe angles and low contrast mean training data from many devices and operators is needed, and the model has to be checked on images from the target device class.

Where the network runs matters. A handheld probe with a phone as the display has a fraction of a workstation's compute, so the model is shrunk with Weight quantization (8-bit or 4-bit weights with per-group scales), which also cuts memory traffic, the usual limit. Results are drawn as an overlay, not burned into the pixels.

Review, storage and streaming

A clinician replays the last seconds (cine) and saves stills and clips. Stills use image-codecs (no page yet) (lossless or high-quality lossy; DICOM can carry JPEG and JPEG-LS, among others); clips use video-codecs (no page yet). Compression of diagnostic images is a clinical choice, so the default is conservative.

For remote guidance, a specialist sees the live image over a network. low-latency-video-encoding (no page yet) fits that: no B-frames (they need future frames), intra refresh in place of periodic keyframes, rate control with a small buffer. A frame of delay is perceptible when someone is guiding a probe by voice.

Everything saved leaves the machine through dicom-protocol (no page yet). An ultrasound study is a DICOM object with patient and study identifiers, probe and acquisition parameters, and either single or multi-frame pixel data. The scanner sends it to the archive with the C-STORE service, looks up the worklist with C-FIND so patient details are not typed by hand, and, when the hospital supports the web profile, uses the DICOMweb equivalents over HTTPS.

The links that leave the scanner carry patient data, so both ends must be known. Mutual TLS secures the C-STORE and worklist connections to the hospital archive and the remote-guidance stream: the scanner presents a device certificate, the archive or viewer presents its own, and each side accepts only peers it recognises. The device certificate is rotated with the update channel, so a decommissioned scanner stops being accepted.

The scanner also parses DICOM objects it did not create, such as worklist replies and imported studies. Coverage-guided fuzzing mutates such files and messages (tag lengths, sequence nesting, pixel-data headers), runs the instrumented parser and keeps inputs that reach new code edges, so a malformed object from the network cannot crash the imaging software mid-exam.

Device software update

A scanner is a regulated medical device that runs for years; software must be updated in the field without bricking it. A/B partition updates holds two bootable slots and a small atomic metadata block: write the new image to the inactive slot, mark it for trial boot, and revert automatically if it does not confirm. The boot chain is verified by Secure boot chain so each stage checks the signature of the next, a Watchdog timer forces a reset and fallback if the new image hangs, and CRC32C checksums detect corruption of the downloaded blocks. Signed update metadata (TUF) signs the manifest with separate roles and thresholds, so a stolen key does not allow a rollback to an old, vulnerable image.

Operations

Frame latency. hdr-histogram (no page yet) records the time from a transmit event to the finished frame on screen, and the beamformer's time per frame. A log-linear histogram keeps bounded relative error over microseconds to tens of milliseconds, so the p99 frame time, not only the mean, is checked against the frame period (about 25 ms for the 40 frames per second case above). A creeping p99 tells the team that a mode, such as a wide colour box, no longer fits the compute budget.

Staying up. The scanner runs the sequencer, beamformer, image chain, export queue and user interface as separate processes. Process supervision starts them in order, reaps a crashed one and restarts it under a delay and burst limit, so a crash in the export client does not end an exam, and a crash loop in the image chain stops after a few tries and reports a fault instead of flickering. The hardware watchdog still covers the case where the supervisor itself hangs.

Stopping cleanly. On shutdown, a user logout or the reboot into a new software slot, graceful draining runs first: the scanner stops starting new acquisitions, finishes the frame in flight, flushes the cine buffer to the study being saved, lets queued C-STORE transfers finish until a deadline, and keeps the rest queued on disk to resume after restart. The exam is never lost to a power-button press or an update.

Data and control flow

One frame, B-mode with a colour box:

1. The sequencer fires the B-mode events line by line, then the Doppler ensembles, on its fixed schedule.
2. Channel data land in ring buffers; the host receives a frame's worth.
3. The beamformer forms lines or pixels; the band-pass filter and apodization are applied.
4. B-mode: envelope, log compression, speckle reduction, scan conversion. Doppler: wall filter, autocorrelation, velocity, threshold.
5. The composed image is drawn with overlays and sent to the display; if freeze is active, the last frames go into the cine buffer.
6. On save, frames are encoded and wrapped in a DICOM object; the C-STORE to the archive is queued with retry.

Control: when the user changes depth, focus or mode, the host rebuilds the sequence and delay tables and switches at a frame boundary, never mid-frame.

Scaling

1. Channel data rate is the first limit: more elements, higher sampling, deeper range. Levers: beamform in the front end, fewer channels per event, sub-aperture schemes, lower sample rates after demodulation.
2. Frame rate vs. image quality: each focused line is one transmit event. Levers: multi-line receive, plane-wave compounding, smaller field of view.
3. Compute per frame for 3D and plane-wave modes: millions of pixels times elements. Levers: GPU beamforming, precomputed delay tables, lower precision, restricting work to the visible region.
4. Power and heat on handhelds: the probe touches the patient, so surface temperature is regulated. Levers: fewer transmit events, lower voltage, processing on the host.

Failure modes

Real implementations

Commercial scanners (GE, Philips, Siemens, Butterfly) run proprietary code; the projects above cover research, simulation and test.

Sources

• Y. Kasai, K. Namekawa, A. Koyano, R. Omoto, Real-time two-dimensional blood flow imaging using an autocorrelation technique, IEEE Transactions on Sonics and Ultrasonics 32(3), 1985.
• USTB documentation: unioslo.github.io/USTB
• SUPRA: github.com/IFL-CAMP/supra
• DICOM standard: dicomstandard.org
• Background: Beamforming, Doppler ultrasonography