Reading a Simulation Critically

A simulation does not predict reality; it computes the consequences of your inputs, exactly and obediently. If the inputs are wrong, the output is wrong - confidently and to three decimal places. So critical reading starts by asking which inputs actually move the result, because they are not equal.

A handful of assumptions dominate most whole-building energy and comfort results. The weather file sets the whole climatic frame - run the same model on a different city's EPW and the answer changes wildly. Occupancy - how many people, how densely, doing what - drives internal gains, fresh-air needs and, in offices, a large slice of the cooling load. Schedules - when the space is used, when lights and equipment are on, when the AC runs - reshape the entire annual profile. Set-points - the thermostat target - are brutal: a cooling set-point of 24 C versus 26 C can shift cooling energy by a large margin in a hot climate. And operation - how the real building is actually run, doors propped, blinds down, systems overridden - is the biggest wildcard of all.

By contrast, some inputs matter less than beginners expect. Fabric details - a glazing U-value tweaked in the third decimal, a wall construction refined slightly - usually move the answer only a little on their own. The lesson: spend your care where the sensitivity is - weather, people, schedules, set-points, operation - not polishing inputs that barely register.

Even a careful model diverges from the real building. This is the well-documented performance gap: measured energy use is routinely higher - sometimes much higher - than the design simulation predicted. The causes are mostly the assumptions above coming apart in reality. Occupants use the space more intensively or unpredictably than the schedule assumed. Systems are commissioned imperfectly or overridden. The building is operated with windows open and set-points nudged. Construction quality introduces thermal bridges and infiltration the model idealised away.

The gap is not a sign the simulation is broken; it is a sign that an absolute prediction of a single building's energy use is inherently uncertain. Understanding this reframes how you use any result. The headline number is an estimate with error bars you should mentally attach, not a meter reading of the future. Calibration - tuning a model against measured data once a building is running - narrows the gap and is essential for retrofit and operational work, but during design you should treat the absolute figure as indicative. Respecting the performance gap is not pessimism; it is the difference between a modeller who oversells and one who can be trusted.

This has a direct effect on how you should communicate. If you hand a client a single number - '87 kWh/m2/yr' - you have made an implicit promise the building may not keep, and the performance gap will eventually embarrass you. Present the same finding as a range, or better as a comparison against a baseline, and you have said something both honest and durable. Good practice quotes results with their uncertainty acknowledged, not stripped of it for a cleaner slide. The gap is a permanent feature of the field, not a flaw to hide - and naming it openly is what marks out a professional from an enthusiast.

Before trusting any result, check whether it is even plausible. Experienced analysts run fast reality checks, and you should build the habit early. Order of magnitude: does the energy use intensity (EUI) land where a building of this type and climate should? An air-conditioned office landing at 5 kWh/m2 per year is not efficient, it is broken input; one at 900 is equally suspect. Compare against known benchmarks - typical ranges for the building type, or code baselines like ECBC and ASHRAE 90.1.

Energy balance: the pieces should sum sensibly - cooling should dominate in a hot climate, heating in a cold one; if a Chennai office shows a big heating load, something is wrong. Direction of change: when you improve the design - add shade, better glazing, more insulation - the result should move the expected way. If a deeper overhang increases cooling energy, you have a bug, not an insight. Peaks and profiles: does the daily and seasonal shape make physical sense - cooling peaking on hot afternoons, lighting following occupancy? A result that fails any of these is telling you about your model, not your building.

These checks take minutes and catch the majority of gross errors - a floor area entered in the wrong units, a schedule left on around the clock, a construction facing the wrong way, a zone with no ventilation. Such mistakes are not rare; they are routine, and they produce outputs that look perfectly formatted and are completely wrong. The discipline is to distrust your own first result on principle and prove it plausible before you let anyone act on it. Never report a number you have not sanity-checked; the software will happily compute nonsense and format it beautifully.

Here is the resolution to all this uncertainty, and the most important habit in the course: read simulations comparatively, not absolutely. When you run option A and option B through the same model with the same assumptions, the errors those assumptions carry are shared - and when you take the difference between the two results, the shared error largely cancels. So even if the absolute EUI of each is uncertain by twenty percent, the statement 'B uses 30% less than A' is far more robust than either number alone.

This is why a rough model, used to compare, is genuinely useful while a rough model quoted as an absolute is dangerous. It is also why the honest way to present findings is relative: 'the shaded option cuts overheating hours by 40% versus the base case', not 'the building will have 312 overheating hours'. The first survives the performance gap; the second pretends the gap does not exist.

The practical rule: always simulate against a baseline. A result with nothing to compare it to is hard to trust and easy to misread. A result expressed as a difference from a clear reference - the current design, a code-minimum case, the previous option - is both more reliable and more decision-useful. Comparison is not a limitation of simulation; it is how you extract its most trustworthy signal.

Some results are not just uncertain - they are actively misleading, and learning to smell them is a professional skill. Watch for the too-good-to-be-true: a design that shows a huge saving usually hides an optimistic assumption - a set-point relaxed, an occupancy dropped, a schedule shortened - doing the work rather than the design. Always ask what input is driving this and whether it is honest.

Watch for cherry-picked baselines: a saving is only as meaningful as what it is measured against. '40% better than a deliberately terrible base case' can be worse than the code minimum. Insist on a fair, disclosed baseline. Watch for false precision - a report quoting energy to four significant figures while the occupancy was a pure guess; the precision is decoration over a shaky input. Watch for hidden defaults: a tool's out-of-the-box schedules, constructions or climate may not be your building at all, and unexamined defaults are a classic source of confidently wrong results.

The defence is always the same short interrogation: What are the key assumptions? Which one dominates this result? What is the baseline, and is it fair? Does the number pass a sanity check? Is this being read comparatively? Run that on every simulation - your own and others' - and you will rarely be fooled. That habit of disciplined suspicion, more than any software skill, is what makes a simulationist worth trusting.