Data Quality & Calibration

There is a hierarchy of value in a smart building - sensing, networks, data, analytics, the twin - and a hard rule runs through all of it: each layer is only as good as the data beneath it. A brilliant fault-detection algorithm fed a drifting temperature sensor will confidently diagnose faults that are not there, or miss ones that are. A machine-learning model trained on a stuck meter learns fiction. A digital twin synchronised to bad readings mirrors a building that does not exist. This is garbage in, garbage out (GIGO), and no amount of sophistication downstream can rescue it - clever analytics on bad data just produce confident garbage, which is worse than obvious garbage because people believe it.

So the mature practitioner inverts the usual excitement. Before trusting any analysis, they ask: can I trust the numbers it rests on? Where did each come from, is the sensor healthy and calibrated, are readings arriving on time and complete, do the units and timestamps mean what they claim? This is not pessimism; it is the professionalism that makes the rest of the stack worth building. Data quality is not a cleanup step you do at the end - it is a property you design in and monitor forever, because sensors degrade the day after they are installed. The rest of this lesson names the specific ways data goes bad, and the habits that catch it.

Every sensor drifts. Over months a CO2 sensor reads gradually high, a humidity sensor low, a pressure sensor off - not through a dramatic failure but a slow, silent bias away from truth. The danger of drift, drawn in the figure, is precisely that it is gradual and confident: the numbers keep coming, they look plausible, nothing alarms, and every analysis built on them quietly tilts. A room held to a drifted CO2 setpoint is under-ventilated while the dashboard shows green; energy figures from a drifted meter are wrong in a way that compounds over a whole reporting year.

The answer is calibration - periodically comparing a sensor against a known reference (a freshly calibrated instrument, a reference gas, a controlled condition) and correcting it back to truth. The figure shows the pattern: the true value is steady, the sensor drifts away from it, and at the calibration point the reading is pulled back onto track. Different sensors drift at different rates - a good NDIR CO2 sensor might need checking annually, some cheap sensors far sooner - and good practice is to record when each sensor was last calibrated and to treat readings from an overdue sensor with suspicion. Some sensors self-correct (automatic baseline calibration assumes a room reaches fresh-air CO2 overnight), which helps but can mislead in spaces occupied around the clock. The instinct to build: a reading without a calibration history is a reading you cannot fully trust.

Drift is the subtle failure; the figure of a data stream shows the blunter ones you will meet constantly. Gaps (missing data) happen when a device sleeps, a battery dies, a radio drops or a gateway reboots - and how you handle them matters: silently filling a gap with the last value can invent a flatline that fools an analysis, so gaps should be marked, not papered over. Spikes are single wild values - a momentary electrical glitch or a dropped bit - that can wreck an average or trigger a false alarm if not filtered. Flatlines are the opposite and more insidious: a sensor that has frozen keeps reporting the same value forever, looking perfectly healthy while measuring nothing - a stuck reading is often more dangerous than a missing one, because missing data is obvious and a flatline hides.

Beyond these, a validation routine watches for out-of-range values (a room at 400 degrees), duplicates (the same timestamp twice), and the classic silent killer - units that changed (a meter reporting watts where the system expects kilowatts, off by a thousand and no one notices). The practical response is a data-validation pipeline: automated checks that range-test, flag stuck and missing streams, quarantine the suspicious, and surface the health of the sensing itself as a monitored quantity. A genuinely smart building watches its own data quality as carefully as it watches the building - because a sensor that has quietly died is worse than no sensor, since it is still trusted.

Building data is time-series data - every reading is a value and a time - and the time half causes as much grief as the value half. Timestamps must agree: if two devices disagree about the clock, or one logs in local time and another in UTC, or a daylight-saving shift is mishandled, then correlating them - the whole basis of analytics and twins - produces nonsense. Aligning a chiller's power with the temperature it was supposed to control is meaningless if their clocks are an hour apart. So time synchronisation (via NTP or the network) and a single, explicit timezone convention - almost always store everything in UTC and display local - are foundational, not fussy details.

Sampling rate - how often you read - is the other half. Sample too slowly and you miss real behaviour: a fifteen-minute meter interval cannot see a two-minute surge, and worse, undersampling can cause aliasing, where a fast signal read too rarely masquerades as a slow one that is not real. Sample too fast and you drown in data that costs storage and bandwidth for detail no decision uses - which is exactly why battery sensors sample slowly on purpose. The right rate matches how fast the quantity actually changes and how quickly a decision must react: sub-second for vibration on critical plant, every few minutes for room temperature, hourly for slow trends. Get timestamps honest and sampling matched to the phenomenon, and the time-series database (Module 4) becomes a trustworthy record; get them wrong and every downstream conclusion inherits the error.

Data quality is not won once; it is maintained. Two habits keep a building honest over its life. The first is commissioning the data at handover: before anyone trusts a dashboard, each sensor and meter is checked against reality - does the space temperature sensor agree with a handheld reference, does the energy sub-meter match a clamp meter, do the points map to the equipment they actually claim, are the units and timezones correct? A great deal of bad building data is not degradation at all but errors baked in on day one - a sensor labelled for the wrong room, a meter scaled by the wrong factor, a point wired to the wrong channel - and only deliberate commissioning catches them before they mislead for years.

The second habit is continuous monitoring of the sensing itself. A mature smart building treats data quality as a live, measured quantity: automated checks watch every stream for drift, gaps, flatlines and out-of-range values, raise a flag when a sensor goes quiet or stuck, and track when each device was last calibrated. In effect the building watches its own senses as carefully as it watches the rooms, so a sensor that has quietly died is caught and replaced rather than believed.

None of this is glamorous, and that is precisely why it is valuable - and so often skipped. But it is the difference between a twin whose decisions can be trusted and a wall of confident, wrong dashboards that slowly loses everyone. Trust, once lost, is expensive to rebuild; designing and operating for data quality from the very start is what keeps a smart building genuinely smart rather than merely well-instrumented.