Time-Series Data & Historians

Strip any building measurement to its essentials and you get three things: when (a timestamp), what value (a number, usually), and which point (a tag or identifier naming the source). Supply-air temperature on air handler one at 09:10 was 22.6 - that is one data point in a series. String millions of these together, per point, across a building, and you have its entire operational record.

A few properties make this shape distinctive. It is append-only in practice: new readings are constantly added, and old ones are essentially never changed - you do not go back and edit what the temperature was at 09:10. It is ordered by time, and almost always queried by time - a chart of the last 24 hours, an average for last month, the peak last summer. It is high-volume and regular: a single point sampled every five seconds is over 17,000 readings a day, and a real building has thousands of points.

That combination - write-heavy, append-only, time-ordered, read-by-range, enormous - is precisely what a general-purpose relational database is worst at. Try to keep a year of five-second data for a whole building in a spreadsheet and it dies; do it naively in a relational table and queries crawl and storage balloons. The data's shape demands a purpose-built tool, which is the whole reason the next section exists.

It is worth pausing on how uniform this shape is, because it is the field's great simplifier. A temperature, a kilowatt reading, an occupancy count, a valve position, a CO2 level - wildly different quantities, yet every one of them arrives as the same humble triple of time, value and tag. That uniformity is what lets a single kind of database, a single kind of chart and a single kind of analytic serve an entire building. It is also why the tag carries so much weight: the value 22.6 is meaningless until the tag tells you it is a supply-air temperature and not a boiler pressure or a car-park humidity. We spend the whole of the next lesson on making that tag mean something; here, simply notice that the value is nothing without it.

Two families of tool specialise in this data, and it helps to know both.

A time-series database (TSDB) is a database engineered for timestamped data - examples include InfluxDB, TimescaleDB and Prometheus. It exploits the shape ruthlessly: because data arrives in time order and is rarely edited, it can store readings in tight, compressed blocks, index heavily on time, and answer range and aggregation queries (averages, min/max, rates) far faster and in far less space than a relational database. Compression is dramatic - regular sensor data often shrinks by an order of magnitude - which is what makes keeping years of history affordable.

The historian (or process historian) is the older, industrial cousin, born in factories and plants and long used in building automation. A historian is a system dedicated to reliably capturing and storing streams of tag-based sensor readings - the classic examples come from industrial control - with a heavy emphasis on never losing a sample, storing efficiently, and serving trends to operators. Your BMS almost certainly has a historian function quietly logging 'trend' data behind every graph it draws.

The line between them is blurring - modern historians look more like time-series databases and vice versa - but the vocabulary persists. When a building engineer says 'pull it from the historian' and a data engineer says 'query the TSDB', they usually mean the same act: reading a point's history from the store built to hold it. For a digital twin, this store is its memory - without it, the twin only knows now, and can neither show a trend nor learn a pattern.

Two dials govern every time-series store, and understanding them prevents most real-world grief.

Resolution (or sampling interval) is how often a point is recorded. Every 5 seconds gives you fine detail and catches fast transients - a valve hunting, a brief spike - but generates mountains of data. Every 15 minutes is far leaner but blind to anything quick. Choosing resolution is a genuine engineering decision: a safety-critical or fast-moving signal needs fine sampling; a slowly drifting space temperature does not. Sampling too coarsely loses events you needed; too finely wastes storage and money on detail no decision uses.

Retention is how long you keep data at a given resolution. You almost never keep raw high-resolution data forever - it is too much. Instead a retention policy downsamples as data ages: keep raw 5-second data for a week, roll it up to 5-minute averages for two years, then to hourly means for a decade. Recent data stays detailed for troubleshooting; old data survives as summaries for long-term trends and benchmarking, at a tiny fraction of the size. This laddering is standard practice and quietly essential - it is the difference between an archive that costs a fortune and one that runs for pennies.

The worked trade-off: a chiller's power at 5-second resolution lets you diagnose a fault this week, but for comparing this year's summer to last, an hourly average is plenty - so you keep both, at different ages, deliberately. Design these dials per point, not once for the whole building.

Everything the twin and its analytics do beyond the present moment depends on this store being complete and correct. A trend chart, a benchmark, a fault-detection rule, a machine-learning model predicting next week - all of them read history, and history is only as good as the time-series store that kept it.

The practical hazards are mundane but ruinous. Gaps appear when a sensor or network drops out, leaving holes a naive average will silently mis-read. Clock problems - devices with the wrong time or timezone - scramble the ordering that everything depends on, so disciplined time-stamping (ideally synchronised, in a consistent timezone such as UTC) matters enormously. Bad values - a sensor reporting a stuck reading or an impossible spike - poison any statistic computed over them unless flagged. And inconsistent resolution across points makes them hard to compare or align.

Good practice treats the time-series store as a first-class asset: monitor the health of the data itself, flag gaps and outliers rather than hiding them, keep timestamps synchronised and unambiguous, and set retention deliberately so the history you will actually need survives. Do this and the twin has a reliable memory to reason over. Neglect it and every clever thing built on top inherits the rot - which is why, once again, the humble store beneath the twin decides how far the twin can be trusted.

There is a cultural point buried here too. Building people and data people often meet across this store and use different words for the same thing - a controls engineer says 'trend log' and 'the historian', a data engineer says 'series' and 'the TSDB' - but they are describing one shared asset. The best smart-building teams are the ones who can speak both dialects and treat the history as common ground: engineers who understand why a data scientist frets about clock drift, and data scientists who understand why an engineer sampled that pressure every second. When those two worlds respect the same store, the twin gets a memory it can rely on. When they do not, each blames the other for a fault that was really a gap nobody owned.