The Question an Advertisement Never Answers: Compared to What

Treat the sections below as a sequence. With the missing comparison in health claims, getting the early decisions right makes the later ones much easier.

Why the comparison is the whole thing

Any measurement taken before and after an intervention includes natural recovery, regression to the mean, seasonal change and everything else happening simultaneously. Without a comparison group experiencing the same period without the product, none of that can be separated from the effect of the product itself. This is why uncontrolled studies consistently report larger benefits than controlled ones investigating the same intervention.

It is also why the phrase participants improved after taking the product is compatible with the product doing nothing whatsoever. The comparison is not a technical nicety, it is the mechanism by which a claim becomes evidence rather than a description.

Better than nothing and better than something

A trial comparing a product against an inactive substitute answers whether it does anything at all. A trial comparing it against the best available alternative answers whether anyone should choose it, which is the practical question. Active comparisons are more expensive, more likely to produce an unflattering result, and correspondingly rarer in commercial research.

At the dose actually studied, a product that beats nothing while performing worse than an established option has technically succeeded and practically failed. Asking which comparison was used sorts a great deal of health research into useful and decorative within a single question.

The comparison against free

Many wellness purchases should be assessed against options that cost nothing, including sleep, walking, sun protection and drinking water. Those alternatives have substantially better evidence than most of what is sold, and none of them are advertised because nobody profits. A supplement claiming to improve energy is competing against fixing sleep, which is free and considerably better supported.

A device claiming to reduce stress is competing against exercise, which has evidence across mood, sleep and physical health together. The free option almost never appears in the comparison because it would win, which is the reason for its absence.

Opportunity cost is a real cost

Money spent on an unproven product is unavailable for something with better support, which matters most for people with least to spare. Attention is finite too, and the effort spent managing a complicated protocol is effort not spent on things that would help more. Time is the most serious version, since a delayed diagnosis can change an outcome permanently in a way that a refund cannot fix.

On the label, wellness marketing presents its products as additions rather than substitutions, which conceals every one of these costs.

Asking what this is replacing is as informative as asking what it is compared against.

Where the comparison is hidden deliberately

Studies comparing a supplement against nothing are described as showing benefit without the qualifier that everyone knew what they were taking. Percentages of improvement are quoted without the corresponding figure from the comparison group, which is often nearly as large.

Once you look at who funded it, reviews sometimes pool studies with different comparators, producing an average that describes no actual clinical question. Marketing frequently quotes a study of a treatment for a diagnosed condition to support use in people without it. Each of these is a comparison problem wearing different clothes, and each is visible once you know to look.

Never stop or alter a treatment a clinician has prescribed on the strength of something you have read.

Using the question yourself

Compared with doing nothing, compared with the cheapest alternative, and compared with the best available option are three separate questions. Ask them in that order, since a product failing the first never needs the other two.

Ask what would happen over the same period with no intervention at all, which for many complaints is improvement. Ask what a clinician would suggest for the same problem, since that comparison is available free in most health systems. The question is short, applies everywhere and requires no technical knowledge, which makes it the most portable tool on this site.

The takeaway

Compared with what, over what period, and instead of what. Three words that most health claims cannot survive.

An extraordinary mechanism needs better evidence than testimonials, and usually has less.