grid-ems

How it is built

Architecture overview

A grid control centre runs two layers on one data stream. SCADA polls substations and power plants every few seconds, keeps the latest value of every measurement and status point, and lets operators send commands (open a breaker, change a setpoint). The energy management system (EMS) sits on top: it takes those measurements, estimates the state of the whole network (every bus voltage and angle), and runs analysis on that state: power flow, "what if line X trips" screening, economic dispatch and the control loop that holds system frequency. The idea that makes it work is that raw telemetry is too noisy, too incomplete and sometimes plainly wrong, so everything downstream runs on a state estimate produced by a weighted least-squares fit of the measurements to a physical network model, with bad data detected and removed.

Control is closed through the same network: the EMS computes generator setpoints, sends them back down the SCADA protocol, and a fixed-period loop repeats this every few seconds. Because the system is critical infrastructure it is run as a redundant pair of control centres, on a segmented operational network that is cut off from the corporate IT network.

Parts:

• Field protocol — outstation and master messaging with event buffers and time stamps — DNP3 protocol (no page yet)
• Phasor measurement — GPS-timed voltage and current phasors, frequency and its rate of change — Synchrophasor estimation (no page yet), Phase-locked loop
• Time base — a common clock for every measurement — Clock synchronization, PID control, Robust statistics
• State estimation — the best-fit network state from redundant, noisy measurements — WLS state estimation (no page yet), Newton root finding, Sparse matrices (CSR), LU and Cholesky
• Power flow and contingency analysis — solve the network, then try every credible outage — AC power flow (no page yet), Contingency analysis (no page yet), Sparse direct solver, Fill-reducing ordering
• Dispatch optimisation — cheapest generation that respects the network and unit constraints — Optimal power flow (no page yet), Interior-point method, Linear programming (simplex), Mixed-integer programming, Cutting planes, Presolve reductions, d-ary heap
• Frequency control — keep frequency and tie-line flows on schedule — Automatic generation control (no page yet), Droop control (no page yet), Fixed-rate control loop
• Operator interface — alarms the operator can act on — Alarm management, Hierarchical state machine, Debouncing and throttling (no page yet)
• Historian — years of measurements at bounded cost — Swinging-door compression (no page yet)
• Availability — a control centre that survives a failed server or site — Hot-standby redundancy, Watchdog timer, Primary-backup replication
• OT security — zones, conduits and one-way transfers — Industrial network segmentation (no page yet), Packet classification (no page yet), Deep packet inspection (no page yet), Data diode (one-way transfer) (no page yet)

 substations / plants                         control centre (primary) ◄── hot standby ──► control centre (backup)
 ┌────────────────────┐   DNP3 / IEC 104      ┌────────────────────────────────────────────────────────┐
 │ RTU / IED / meters │ ───── polls, events ─►│ front end: protocol drivers, time-stamp, quality flags │
 │ PMU (GPS-timed)    │ ── C37.118 stream ───►│            │                                           │
 └─────────▲──────────┘                       │            ▼                                           │
           │ setpoints, breaker commands      │  real-time database ──► alarm processor ──► operator HMI│
           │                                  │            │                                           │
           │                                  │   topology + measurements                              │
           │                                  │            ▼                                           │
           │                                  │  state estimator ─► power flow ─► contingency screening│
           │                                  │            │                       ▼                   │
           └──────────────────────────────────│── AGC / dispatch (OPF, unit commitment) ──► historian │
                                              └────────────────────────────────────────────────────────┘
        firewalled conduits between zones; one-way export to the corporate network

How the parts fit together

Field protocol

DNP3 protocol (no page yet) was designed for slow, lossy serial and radio links. An outstation keeps typed points (binary inputs, analog inputs, counters) and an event buffer of changes with their own time stamps. The master polls for static data now and then (integrity poll) and for events often (event poll); the outstation can also send unsolicited responses when something changes. This is what lets SCADA get a breaker trip within a second without polling everything every second. IEC 60870-5-104 and IEC 61850 serve the same role in other regions and for substation-internal buses. The contract to the layers above is a point with a value, a quality flag and a time stamp from the field, not from the receiving server: the estimator must know how old a value is and whether the device marked it as substituted or invalid.

Time base and phasors

Time stamps are only as good as the clock behind them. Clock synchronization estimates a clock's offset from four timestamps and steers it smoothly, using ideas from PID control; the filters that discard queueing outliers are Robust statistics (median, MAD). For ordinary SCADA, millisecond accuracy is enough. For phasor measurement units it is not: the phase angle of a 50 Hz or 60 Hz wave changes by 360 degrees per period, so one microsecond of time error is about 0.018 degrees at 50 Hz and 0.022 degrees at 60 Hz (derived: 360 × f × 1 µs). PMUs therefore take their time from GPS.

Synchrophasor estimation (no page yet) computes, at a fixed reporting rate aligned to UTC seconds, the amplitude and angle of the fundamental by a DFT over a window, plus frequency and its rate of change; the IEEE synchrophasor standard C37.118.1 defines the accuracy tests, and it has since been republished as the IEEE/IEC dual standard 60255-118-1 (per the IEEE standards page). A Phase-locked loop is the alternative for tracking frequency when the grid frequency drifts away from nominal. A phasor data concentrator aligns streams from many PMUs by time stamp and waits a bounded time for late ones before forwarding the set.

State estimation

Redundant measurements (more than the minimum needed) are fused by WLS state estimation (no page yet): minimise the sum of squared residuals between measured and model-predicted values, each divided by its variance. The nonlinear measurement equations are linearised and solved by Gauss-Newton, an application of Newton root finding; each iteration solves a symmetric positive-definite system, the gain matrix, which is very sparse for a transmission network. Sparse matrices (CSR) store it, and LU and Cholesky factor it (see below for the ordering). After the fit the estimator tests whether the leftover error is plausible (chi-square on the objective) and finds the single largest normalised residual to identify a bad measurement or a wrong breaker status; it removes it and repeats. This is the step that turns a flaky telemetry stream into a consistent model and is why the downstream applications can trust their input.

Power flow, contingencies and the sparse solver

AC power flow (no page yet) solves for all bus voltages given injections by Newton-Raphson on the bus admittance matrix: a Jacobian that is sparse and re-factored each iteration, typically converging in a handful of iterations (order of 3 to 6). Speed comes from the linear algebra, not the nonlinear loop: a Sparse direct solver does symbolic analysis once per topology and numeric factorisation per iteration, and Fill-reducing ordering (minimum degree or nested dissection) keeps the factors from filling in.

Contingency analysis (no page yet) asks, for every credible single outage (N-1), whether any line would overload or voltage would leave its band. A full AC solve for thousands of outages per cycle is too slow, so screening uses a fast DC or decoupled approximation to rank cases and runs the full solve only on the dangerous ones. The output is a ranked violation list for the operator and a set of constraints for dispatch.

Dispatch optimisation

Optimal power flow (no page yet) chooses generator outputs that minimise cost subject to the network equations and limits. The AC problem is nonlinear and is solved with an Interior-point method, whose main cost per iteration is a sparse Cholesky (again the sparse solver); the DC approximation is a linear program and uses Linear programming (simplex) or interior point. Deciding which units to switch on for the next day (unit commitment) adds on/off integers, so it is a Mixed-integer programming problem, made tractable by Presolve reductions and Cutting planes; the solver's open-node queue is a d-ary heap. Network constraints from contingency analysis are added lazily, only the violated ones each round, which keeps the model small.

Frequency control

Frequency is the grid's shared signal: generation above load raises it, below lowers it. Two controls act on different time scales. Local droop (Droop control (no page yet)) makes each generator change its output in proportion to the frequency error, with no communication, so that all units share a disturbance; this is the first seconds. Automatic generation control (no page yet) acts next: it integrates the area control error (frequency deviation and tie-line flow deviation from schedule) with a PI controller of the kind in PID control, and sends participation-weighted setpoints to the units on the SCADA channel. The cycle runs at a fixed period of a few seconds (order of 2 to 4 s), so it must be a Fixed-rate control loop: absolute deadlines and a fixed time step, so that jitter in the host does not accumulate as a control error.

Operator interface

A grid event produces hundreds of alarms in seconds. Alarm management turns limits into few actionable alarms with delay, deadband, acknowledge and shelving states, modelled as a Hierarchical state machine per alarm; Debouncing and throttling (no page yet) coalesces chattering contacts. The rule of thumb is that an alarm exists only if the operator has an action for it.

Historian

Every point is stored for years. Swinging-door compression (no page yet) keeps a sample only when a straight line through the previous kept point and a tolerance "door" can no longer represent the data, so a flat signal costs a few points per day and a moving one is kept at tolerance accuracy. It is lossy by design, with a tolerance set per point class.

Availability

Hot-standby redundancy keeps a second server (and a second site) current and takes over within a few cycles of silence; epochs stop a revived primary from acting after the standby has taken over. A Watchdog timer in the front end and AGC process forces a restart if the control loop stalls. The real-time database is shipped to the standby by Primary-backup replication, with a rule that control actions are acknowledged only after the standby has them, if the loss of one command is unacceptable.

Security

The operational network is split into zones with defined conduits between them (Industrial network segmentation (no page yet); the IEC 62443 series and the Purdue model describe the zoning). Conduits are enforced by firewalls with allow-lists per protocol: Packet classification (no page yet) for the rules, Deep packet inspection (no page yet) to check that a DNP3 write is permitted for that master. Where data only needs to leave the control zone, a Data diode (one-way transfer) (no page yet) makes the export unable to carry commands back.

Data and control flow

One state-estimator cycle.

1. The front end delivers the latest value, quality and time stamp of every telemetered point to the real-time database (order of seconds old).
2. The topology processor builds the bus-branch model from breaker and switch status.
3. The estimator checks observability, runs the WLS iterations (a few sparse factorisations), and tests the residuals.
4. Bad data are removed or corrected and the estimator repeats; the state (voltages, angles, flows) is published.
5. Contingency analysis and, on demand, dispatch use the published state. Time goes to the sparse factorisations; the whole cycle runs on the order of seconds to a few minutes depending on network size.

One AGC cycle.

1. Read frequency and tie-line flows (measured, not estimated; they are the fastest points).
2. Compute the area control error, update the integral, apply deadband.
3. Allocate the correction to units by participation factor and ramp limits.
4. Send setpoints through the SCADA protocol; units acknowledge; the loop sleeps to the next absolute deadline.

An operator command.

1. The operator selects a breaker; the HMI asks the master to select, then operate (select-before-operate, a two-step command in DNP3).
2. The outstation checks and operates; the confirming status change returns as an event.
3. The estimator sees the changed topology in the next cycle; the alarm processor suppresses expected alarms for that action.

Scaling

1. Network size. Estimation and contingency cost grows with buses and credible outages. Levers: sparse ordering and factor reuse, screening before full solves, decomposition by region with neighbouring areas exchanging boundary data.
2. Telemetry volume. Event reporting (change-based, not polled) and PMU streams dominate. Levers: deadbands at the source, unsolicited responses, compression in the historian.
3. Optimisation size. Unit commitment is the hard one. Levers: presolve, lazy network constraints, warm start from the last schedule, time limits with a best-bound gap.
4. Redundancy and recovery time. Lever: keep the standby warm so that takeover is a few control cycles, not a restart.

Failure modes

Real implementations

Commercial control-centre systems from the large automation vendors implement the same structure but are not open.

Sources

• R. D. Zimmerman, C. E. Murillo-Sánchez, R. J. Thomas, MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education, IEEE Trans. Power Systems, 2011. doi
• L. Thurner et al., pandapower: An Open-Source Python Tool for Convenient Modeling, Analysis, and Optimization of Electric Power Systems, IEEE Trans. Power Systems, 2018. doi
• IEEE C37.118.1 Standard for Synchrophasor Measurements for Power Systems, superseded by IEEE/IEC 60255-118-1-2018 per IEEE Standards.
• Repositories above (licences and push dates from the GitHub API, 2026-10-21; opendnp3 archived).
• The structure of WLS estimation (normal equations, chi-square and largest-normalised-residual tests), N-1 screening, AGC and droop follows standard power-systems textbooks; no single source was checked for this page.