tirogla
Home
Systems
API
About
Log in
Home
Systems
API
About
Log in
← Back
Home
›
Systems
›
Simulation and scientific computing
astronomy-pipeline
sky survey alert pipeline
Rubin LSST
Open the astronomy-pipeline page
astronomy-pipeline
Object storage
cloud: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Pub/sub fan-out
distributed: Deliver one published message to every subscriber of a topic by sharing one encoded payload and giving each connection a bounded queue with an explicit slow-consumer policy.
›
Healpix sky pixelization
geometry: HEALPix and HTM equal-area hierarchical sphere pixelization for sky indexing and cross-matching
k-d tree
geometry: A binary tree that splits k-dimensional points by one coordinate per level, giving fast nearest-neighbour and box queries on static low-dimensional data.
›
Image resampling filters
graphics: Resize an image by convolving with a kernel (box, triangle, cubic, Lanczos) whose width scales with the shrink factor, done as two separable 1-D passes with precomputed weights.
›
Gradient-boosted decision trees
(GBDT)
machine-learning: Add small regression trees one at a time, each fitted to the loss gradient; histogram splits over 256 bins make it the strongest tabular default.
›
Astrometric plate solving
media: Match detected star patterns (quad hashes) to a catalog to solve the image WCS
Image differencing psf matching
media: Subtract a reference image after PSF-matching convolution (Alard-Lupton, ZOGY) to find transients
Source extraction
media: Detect and measure sources in images: background estimation, thresholding, deblending, photometry
Fft convolution
signal-processing: Overlap-add and uniformly partitioned convolution for long FIR filters and reverb
Robust statistics
statistics: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.
›
Columnar storage
storage: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.
›
SPSC ring buffer
data-structures: Bounded wait-free queue for exactly one producer and one consumer thread, built from an array and two indices with acquire/release ordering.
›
Topic matching trie
data-structures: Trie over topic levels with + and # wildcards for subscription matching
Backpressure
systems: Bounded queues and credit-based flow so slow consumers throttle producers
Quantile sketches
data-structures: Mergeable quantile sketches: t-digest, DDSketch, HDR histograms
Loss functions
machine-learning: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Regularization
machine-learning: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.
›
Map projections
geometry: Web Mercator, UTM and equal-area projections, datums and coordinate transforms
Convolution image filters
graphics: Separable 2D convolution kernels for blur, sharpen (unsharp mask) and edges with border handling
Ransac
robotics: Random sample consensus for robust model fitting with outliers (homographies, planes, poses)
Image warping remap
graphics: Resampling with lookup maps: resize, lens undistortion, homographies, bilinear/bicubic
LU and Cholesky factorization
numerics: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Union-find
graphs: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
+ 9 more
False sharing
concurrency: Independent variables written by different cores on one cache line force the line to bounce between cores; pad or regroup hot data to fix it.
›
Memory ordering
concurrency: What Relaxed, Acquire, Release and SeqCst atomics actually guarantee, what they cost on x86 and ARM, and which one each pattern needs.
›
+ 1 more
CPU caches and the memory
hierarchy
hardware-os: Sizes, line widths and latencies of L1/L2/L3/DRAM on modern x86 and Apple M-series, and the data-layout rules that follow from them.
›
Open the astronomy-pipeline page
astronomy-pipeline
Object storage
cloud: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Pub/sub fan-out
distributed: Deliver one published message to every subscriber of a topic by sharing one encoded payload and giving each connection a bounded queue with an explicit slow-consumer policy.
›
Healpix sky pixelization
geometry: HEALPix and HTM equal-area hierarchical sphere pixelization for sky indexing and cross-matching
k-d tree
geometry: A binary tree that splits k-dimensional points by one coordinate per level, giving fast nearest-neighbour and box queries on static low-dimensional data.
›
Image resampling filters
graphics: Resize an image by convolving with a kernel (box, triangle, cubic, Lanczos) whose width scales with the shrink factor, done as two separable 1-D passes with precomputed weights.
›
Gradient-boosted decision trees
(GBDT)
machine-learning: Add small regression trees one at a time, each fitted to the loss gradient; histogram splits over 256 bins make it the strongest tabular default.
›
Astrometric plate solving
media: Match detected star patterns (quad hashes) to a catalog to solve the image WCS
Image differencing psf matching
media: Subtract a reference image after PSF-matching convolution (Alard-Lupton, ZOGY) to find transients
Source extraction
media: Detect and measure sources in images: background estimation, thresholding, deblending, photometry
Fft convolution
signal-processing: Overlap-add and uniformly partitioned convolution for long FIR filters and reverb
Robust statistics
statistics: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.
›
Columnar storage
storage: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.
›
SPSC ring buffer
data-structures: Bounded wait-free queue for exactly one producer and one consumer thread, built from an array and two indices with acquire/release ordering.
›
Topic matching trie
data-structures: Trie over topic levels with + and # wildcards for subscription matching
Backpressure
systems: Bounded queues and credit-based flow so slow consumers throttle producers
Quantile sketches
data-structures: Mergeable quantile sketches: t-digest, DDSketch, HDR histograms
Loss functions
machine-learning: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Regularization
machine-learning: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.
›
Map projections
geometry: Web Mercator, UTM and equal-area projections, datums and coordinate transforms
Convolution image filters
graphics: Separable 2D convolution kernels for blur, sharpen (unsharp mask) and edges with border handling
Ransac
robotics: Random sample consensus for robust model fitting with outliers (homographies, planes, poses)
Image warping remap
graphics: Resampling with lookup maps: resize, lens undistortion, homographies, bilinear/bicubic
LU and Cholesky factorization
numerics: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Union-find
graphs: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Spline interpolation
numerics: Linear, bilinear/trilinear and cubic spline interpolation on grids and tables
Peak detection
signal-processing: Robust peak and event detection in noisy signals with adaptive thresholds and refractory periods
SIMD programming
hardware-os: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Fast Fourier transform
numerics: Computes the DFT in O(n log n) by recursively splitting into even and odd halves; the basis of fast convolution and spectral methods.
›
Bit packing and PFOR
compression: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Delta encoding
compression: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.
›
Dictionary encoding
compression: Map values to small integer codes via a dictionary for compact, comparable columns
Vectorized execution
storage: Query operators pass batches of about 1-2K values per column instead of single tuples, amortizing interpretation overhead and enabling tight SIMD loops.
›
Zone maps
storage: Per-block min/max (and sparse primary) indexes to skip data during scans
False sharing
concurrency: Independent variables written by different cores on one cache line force the line to bounce between cores; pad or regroup hot data to fix it.
›
Memory ordering
concurrency: What Relaxed, Acquire, Release and SeqCst atomics actually guarantee, what they cost on x86 and ARM, and which one each pattern needs.
›
Varint (LEB128)
compression: Variable-length integers in 7-bit groups with a continuation bit; small numbers take 1 byte, the wire format of Protobuf, WebAssembly and DWARF.
›
CPU caches and the memory
hierarchy
hardware-os: Sizes, line widths and latencies of L1/L2/L3/DRAM on modern x86 and Apple M-series, and the data-layout rules that follow from them.
›
Copy for AI