Regression to the Mean Makes Almost Anything Look Like It Worked

What follows is the working version of regression to the mean in health claims: the decisions in the order you actually meet them, with the reasoning attached.

The statistical fact underneath

Any measurement that fluctuates will produce occasional extreme values, partly through real variation and partly through measurement noise. A value that is unusually high or low is likely to be followed by one closer to the typical level. This happens without any intervention, because the extreme value was partly the product of factors that will not repeat.

It is not a force pulling things back, it is simply what probability produces when you select on extremes. The effect appears in blood pressure, pain scores, symptom diaries, exam results and sports performance alike.

Why it targets health claims specifically

Nobody buys a remedy on an average day, because on an average day there is no reason to. Purchases and consultations cluster at the worst point of a symptom, which is by definition an extreme value. Improvement from that point is the most probable outcome regardless of what is taken.

The remedy then receives credit for a change that the timing of the purchase made likely. This mechanism is silent, universal and requires no dishonesty from anyone involved.

How it manufactures testimonials

Every product with any customers will accumulate accounts of dramatic improvement through this route alone. The accounts are sincere, and the people writing them experienced exactly what they describe.

At the dose actually studied, sincerity is why testimonials are so persuasive and why they carry almost no evidential weight. Conditions that come and go, such as back pain, migraine, eczema and fatigue, generate the most striking examples. Serious illnesses with fluctuating courses produce the same pattern, which is how dangerous claims acquire their believers.

Why control groups solve it

A control group experiences the same regression, because its members also enrolled at their worst point. Comparing the two groups removes the effect from both sides and leaves whatever the treatment genuinely added.

This is the main reason uncontrolled before-and-after studies overstate benefits so consistently. A study reporting that patients improved after treatment, with no comparison group, has not measured a treatment effect.

Recognising that phrase is one of the quickest ways to sort useful research from decorative research.

Where it appears in ordinary life

A hangover cure taken at the worst moment appears to work, because the worst moment was going to pass anyway. A back treatment started during an acute flare gets credit for the resolution most flares undergo naturally.

Speed camera placement at accident hotspots produces apparent reductions partly through this effect, which is a well-known example. Sports performance after a change of coach following a bad run shows the same pattern for the same reason. Noticing it in unrelated domains makes it much easier to spot in health claims.

Never stop or alter a treatment a clinician has prescribed on the strength of something you have read.

Testing it on yourself

Note how bad the symptom is before starting anything, and note what you would expect without treatment. Consider how the symptom has behaved historically, since most fluctuating problems have a typical cycle. If a remedy only ever seems to work when things are at their worst, that pattern is informative.

At the dose actually studied, stopping and restarting several times gives more information than a single trial, for symptoms where that is safe. Never apply that approach to a prescribed medicine, because stopping treatment to test it is a decision for your prescriber.

The takeaway

People buy remedies at their worst moment, and worst moments tend to pass. That timing has sold more products than any active ingredient.

The body already has organs for this, and none of them are sold in a box.