Measuring the Effect

The most common and most consequential confusion in reading research is treating statistical significance as if it meant importance. They are different questions with different answers. Statistical significance is a test of chance: it asks whether a result is unlikely to have arisen by random fluke if there were really no effect at all. When a study reports that a finding is 'significant' (often via a p-value below 0.05), it is saying, roughly, 'this is probably not just noise.' That is genuinely useful - it is a guard against mistaking randomness for a pattern - but notice what it does not say. It says nothing about how large the effect is, how much it matters in practice, or whether you should change a design because of it.

Here is the trap. With a large enough sample, almost any difference, however trivial, becomes statistically significant, because the test gets more sensitive as numbers grow. A study of ten thousand office workers might find that a certain layout 'significantly' reduces reported stress - and the reduction might be so tiny that no one inside the building would ever feel it. 'Significant' has become a technical word meaning 'probably real', but the everyday ear hears 'big and important', and marketing exploits that gap relentlessly. A headline that says an effect is significant has told you it is probably not chance; it has told you nothing about whether it is worth a single design decision.

The reverse trap matters too. A real and useful effect can fail to reach significance in a small study simply because there were not enough people to rule out chance - so 'not significant' does not always mean 'no effect', it can mean 'not enough data to be sure yet'. This is why a lone small study, significant or not, is fragile, and why replication and larger samples matter so much. The honest reader treats significance as a first gate - is this probably real? - and then immediately asks the question significance cannot answer: how big is it? That second question, effect size, is where the practical meaning lives, and it is the one most often left out of the headline. Ask both. A result that is real but minuscule is a curiosity, not a mandate; only a result that is both probably real and meaningfully large earns a place in a design decision.

Once you know an effect is probably real, the decisive question is how big. Researchers answer it with effect size - a measure of the magnitude of a difference or relationship, independent of sample size - and by convention effect sizes are loosely described as small, medium or large. You do not need the formulas; you need the habit of asking for the size and refusing to be satisfied by 'significant' alone. An effect large enough to change how a space feels or performs is worth designing for. An effect real but tiny is a footnote, whatever the press release says.

The most abused vehicle for hiding size is the bare percentage. 'Reduced stress by 30 percent' sounds dramatic, but it is meaningless until you ask: thirty percent of what baseline, measured how, and across what range? A thirty percent drop on a scale where everyone was barely stressed to begin with may be trivial; the same figure can be inflated by a relative framing that hides a small absolute change. Percentages also travel without their context - a change measured in a single lab session gets quoted as if it were a permanent life effect. Whenever you meet a percentage, mentally translate it back into something concrete: how many actual people, how much actual change, would you notice it if you were standing in the room?

Size also has to be judged against cost and reality. A design move that produces a genuine but small benefit may still be worth it if it is cheap and harmless; the same small benefit is not worth it if it drives an expensive, permanent decision or crowds out something that matters more. This is the difference between statistical significance and practical significance - between 'the effect is probably real' and 'the effect is big enough, in this context, to act on'. The literature can help you with the first; only judgement, applied to your brief and your budget, settles the second. So the discipline is simple to state and rare in practice: never let 'significant' stand in for 'big', always ask for the effect size, always unpack a percentage into something you can picture, and always weigh the size against what acting on it would cost. A number that survives those questions is worth something. Most impressive numbers do not survive them.

The oldest warning in statistics is the one neuroarchitecture most needs, because so much of its evidence comes from field studies where researchers observe rather than control. Correlation does not imply causation. When two things move together - offices with plants have happier staff, daylit classrooms have higher scores, hospital rooms with views have faster recoveries - it is tempting to conclude that the first caused the second. But a correlation is compatible with several very different stories, and only some of them justify a design decision.

There are three explanations to weigh whenever you meet a correlation. The first is the one you hope for: A genuinely causes B - the greenery really does lift mood. The second is reverse causation: B causes A, or the arrow runs the other way than assumed - perhaps happier, better-run firms are the ones that can afford green offices, so success buys the plants rather than the plants buying success. The third, and the most treacherous, is a confounder - a hidden third factor, C, that causes both A and B and creates a correlation between them with no direct link at all. Wealthier organisations tend to have both nicer buildings and better-treated, happier staff; the building and the happiness correlate, but a common cause, resources, is driving both. The daylit classroom may also be the newer, better-funded school with better teachers. The room with a view may be the private room given to less critical patients. Untangling these is exactly what a real building makes hard, because the good features cluster together.

This is why the study design matters so much. A controlled experiment, which randomly assigns people or conditions, is the strongest tool for isolating cause, because randomisation tends to spread confounders evenly. Most neuroarchitecture evidence, though, is observational - it watches the world rather than intervening - and observational findings can strongly suggest causation but rarely prove it alone. That does not make them useless; it makes them claims to hold provisionally and to strengthen through replication, experiments and converging methods. The practical rule for a designer: when you meet 'buildings with X have more Y', do not silently upgrade it to 'X causes Y'. Ask whether the arrow might run backwards, and above all what third factor might be quietly producing both. Often the honest answer is that the causal story is plausible but unproven - which is a fine reason to treat it as a hypothesis worth testing, and a bad reason to sell it as a fact.

The last question is the one most often skipped: who was in the study? Every finding is produced on some specific population, and it travels only as far as that population reasonably extends. A great deal of neuroarchitecture research is run on convenient samples - university students, Western office workers, volunteers who signed up - who are younger, healthier, wealthier and more culturally narrow than the people most buildings serve. An effect found on undergraduates in a North American lab may or may not hold for elderly patients in an Indian hospital, children in a crowded school, or a multigenerational family in a dense city. The finding is not wrong; it is simply local until shown otherwise. Ask who was measured, how many, and how similar they are to the people you are designing for, and downgrade your confidence honestly when the gap is large.

Put the four questions together and you have a compact, humane way to read any neuroarchitecture claim without a statistics degree. Is it probably real, or could it be chance (significance)? How big is it actually, once you unpack the percentage (effect size)? Does the study show cause, or only that two things moved together with a possible third factor lurking (correlation versus causation)? And who was measured, and do they resemble your people (population)? A claim that passes all four - a reasonably large, replicated effect, from a design that supports causation, on a population like yours - is strong evidence you can lean on. A claim that fails several - a tiny effect, from a small observational study on undergraduates, quoted as a dramatic percentage - is a hypothesis at best and marketing at worst.

This numeracy is not about becoming a scientist; it is about self-defence and honesty. It protects you from repeating overstated claims to clients, from spending a budget on a trivial effect, and from the neuro-washing that dresses thin numbers in scientific authority. It also keeps you appropriately humble: even the strongest study speaks to averages, not to the particular person in your particular room, and none of it displaces the codes or the clinical judgement of qualified professionals. Read the number for size, cause, chance and population - and it will tell you the truth, which is usually more modest, and more usable, than the headline.