Data Quality & Uncertainty

Start with the data at its source, before anyone has projected anything. A real weather record - the station measurements that everything else is built from - is far messier than the tidy file it eventually becomes. Instruments fail and drift: a temperature sensor slowly reads high as it ages, a humidity sensor fouls, a solar sensor gathers dust, and unless caught and corrected these introduce quiet, systematic errors. Readings go missing: power cuts, faults, communication failures and human lapses leave holes in the record, sometimes hours, sometimes months, and those holes must be filled by estimation - interpolating from before and after, or borrowing from a neighbouring station - each fill a small guess standing in for a measurement that never happened.

Stations also change in ways that corrupt the record without anyone tampering with a single number. A station gets relocated a few hundred metres, or its surroundings change - a field becomes a car park, a new building throws a shadow or blocks the wind, a city grows up around what was once open ground - and suddenly the 'same' station is measuring a subtly different place, so a long record can show a warming trend that is partly real climate and partly the creeping urbanisation of the site itself. Instruments get upgraded to new types that read slightly differently. Observation times shift. Each of these creates a discontinuity - a step in the data that is an artefact of the measurement, not the weather.

Because of all this, raw weather data is never used raw. It is put through quality control and homogenisation: automated and expert checks that flag impossible or suspect values, fill gaps, and try to detect and correct the artificial steps from relocations and equipment changes so that a long record reflects real climate rather than the history of the instrument. This is skilled, valuable work and it makes the data far more usable - but it is also, unavoidably, a layer of estimation and judgement laid over the measurements. The corrected record is better than the raw one, but it is not pristine truth; it is a cleaned-up reconstruction carrying its own residual uncertainty. The honest starting point, then, is that even a single good station's history is a patched, corrected, partly estimated thing - and that is *before* we ask whether that station has anything to do with the building you are designing.

Suppose the data is impeccably measured and cleaned. There remains a deeper problem that no amount of quality control can fix: the measurement was taken somewhere else. A weather file's numbers describe the weather at the station, and the single most important - and most overlooked - question a designer can ask is whether that station is *representative* of the actual building site. Very often it is not, and the mismatch can be large enough to overturn a comfort or safety judgement.

Official stations tend to sit in open, standardised, ventilated locations - classically an airport, on the grassy edge of a city, deliberately away from buildings and heat sources so the reading is a clean regional measurement. Your building, by contrast, may sit in a dense, paved, walled, traffic-heavy urban quarter with little breeze and much stored heat. The gap between the two is not noise; it is the urban heat island - the well-documented tendency of built-up areas to run hotter than their rural surroundings, especially at night when concrete and asphalt release the day's stored heat, keeping the city warm long after the airport has cooled. That difference can be several degrees, and it lands exactly where it hurts most: it makes the real site hotter than the file says, and it robs the city of the cool nights that passive cooling depends on. A design checked against a leafy airport file can be judged safe and still cook its occupants a few kilometres away in the concrete.

Elevation, distance from the coast, aspect, altitude and local topography add further mismatch - a hillside, a valley floor, a seafront and an inland plain a short distance apart can have meaningfully different climates, and the nearest station may sit in the wrong one. The crucial mental shift is from accuracy to representativeness: a file can be perfectly accurate for its station and still be the wrong climate for your building. This is why the recurring warning of the whole course - *a station is not your site* - belongs here most of all. The competent response is not to distrust all data but to ask, every time, how well the source location matches the real site, to expect a dense urban site to run hotter (and its nights warmer) than a standard station suggests, to lean on local observation and microclimate knowledge to bridge the gap, and to treat the urban-heat-island and site-specific corrections as questions for qualified specialists and verified local data rather than assumptions a downloaded file has already handled. It has not.

Put the previous two sections together and a quietly radical conclusion follows: the historical record - the thing we instinctively treat as solid fact - is itself an uncertain estimate. This is worth stating plainly because so much confusion in climate work comes from forgetting it. People readily accept that the future is uncertain but imagine the past is known exactly, so they treat a historical weather file as the reliable anchor and worry only about the projections layered on top. But the historical file is not a perfect transcript of what the atmosphere did. It is built from instruments that drift and miss, at stations that move and whose surroundings change, cleaned by homogenisation that is itself estimation, often patched with interpolation, drawn frequently from reanalysis or satellite estimates rather than direct local measurement, and - crucially - representative of some location that is usually not your site. Every one of those is a source of uncertainty riding along inside numbers that *look* exact.

And the exactness is precisely the trap. A weather file presents its values to a decimal point, hour by hour, with all the visual authority of precise measurement - the same false-precision illusion that later modules flag for future files applies, in gentler form, to historical ones. A number written as 34.2 degrees is not known to a tenth of a degree at your site; it may be off by a degree from measurement issues and by several more from representativeness, yet it wears the costume of precision. Reading historical data honestly means mentally attaching an error bar to it - a sense of how much it could be wrong, and in which direction (for a dense urban site, systematically too cool) - rather than accepting the tidy figure at face value.

This is not a counsel of despair or an excuse to ignore data - the historical record remains genuinely valuable and is the essential foundation for everything, and good data is far better than guesswork. It is a counsel of calibrated humility: use the data, but hold it with an honest sense of its limits; know that even the measured past is a best estimate carrying real uncertainty; expect that uncertainty to be larger for poorly served places and mismatched sites; and never let the decimal points fool you into a confidence the data cannot support. The whole discipline of designing under uncertainty, which the rest of the course develops for the future, has its root right here: an unflinching honesty about how well - and how poorly - we actually know even the climate that has already happened.

All of this converges on one hard-headed principle that must sit at the very front of climate analytics: garbage in, garbage out. Everything downstream - the future weather file, the simulation, the elegant chart of overheating hours in 2050 - is built on the historical data, and inherits its flaws. If the base data is biased too cool because the station is not your site, every future file morphed from it and every simulation run on it is biased too cool as well, and the polished final result is confidently, invisibly wrong. And the uncertainty does not stay constant as it flows downstream - it compounds: the future file adds projection and scenario uncertainty on top of the data's, and the simulation adds modelling uncertainty on top of that, so the neat number at the end sits on a stack of uncertainties, each layered on the last. You cannot polish at the end what was broken at the start, and a beautiful output is not evidence of a sound input.

The discipline that follows is not perfectionism - waiting for flawless data that will never come - but honesty and robustness. First, audit the data before trusting the analysis: know its source, vintage, coverage and representativeness, and how well it matches your site, before you believe a single downstream result. Second, calibrate confidence to data quality: lean harder on results built from good, representative data, and treat results from thin, mismatched or heavily estimated data as broad direction rather than fact. Third, design for robustness, not false precision: because the whole chain is uncertain, favour buildings that stay safe and comfortable across a range of conditions - passive, resilient, forgiving - over ones finely optimised to specific numbers that may be wrong, which is exactly the stance the rest of the course builds. Fourth, keep the binding results with those equipped to handle the uncertainty: qualified building-physics, energy and climate-risk engineers, verified and site-appropriate data, validated tools, and the governing codes (NBC India, ECBC, IS), who can apply proper corrections, margins and judgement rather than taking a downloaded file at face value.

In a data-poor, rapidly warming country like India, where stations are sparse, urban heat islands fierce and the most vulnerable people live in the least-measured places, this honesty is not academic pedantry - it is how you avoid designing dangerously to reassuring but wrong numbers. The measured past is the foundation of climate analytics, and it is an uncertain foundation; building well on it starts with admitting exactly that.