Reading a Study Without a PhD

You don't need to be able to interpret a confidence interval. You need about five questions, and they'll get you further than most of the coverage you'll encounter.

1. Relative or absolute?

The single most useful question, and the one that explains the last twenty years of menopause anxiety.

"A 26% increase in breast cancer risk" is a relative figure. It tells you how much a risk changed, but nothing about the size of the risk to begin with. If a risk goes from 2 in 1,000 to 2.5 in 1,000, that's a 25% increase — and also an extra half a case per thousand women.

Both descriptions are accurate. One produces panic and one produces a decision.

Headlines almost always use the relative figure, because it's larger. When you see a percentage increase, the follow-up question is: increase from what, to what, in how many women?

This is exactly why NICE publishes a decision aid for HRT giving numbers per 1,000 women with and without treatment. It's the most useful single document in this field, and it exists because relative risk was doing so much damage.

2. Who was actually studied?

A finding applies to the people it was found in.

The Women's Health Initiative studied women with an average age of 63, many more than a decade past menopause, taking one particular oral formulation. Its results were then applied, for two decades, to 51-year-olds starting transdermal estradiol for hot flushes. That mismatch is the root of most of the confusion.

Look for: how old were they, at what stage, taking what exactly, for how long.

3. Randomised or observational?

Randomised controlled trials allocate people to treatment or control by chance. Because allocation is random, the groups should be comparable in everything except the treatment, so differences in outcome can more confidently be attributed to it.

Observational studies watch what happens to people who chose things themselves. Useful, often the only ethical option, and vulnerable to a specific problem: the people who chose the thing may differ systematically from those who didn't.

In menopause research this has a name — healthy user bias. Women who took HRT in the 1990s tended to be wealthier, thinner and more engaged with healthcare. Observational studies showed excellent cardiovascular outcomes. Some of that was the treatment. Some was that healthier women were taking it.

A single observational study is a reason to be interested. It isn't proof.

4. What was measured?

Surrogate endpoints are proxies — bone density rather than fractures, cholesterol rather than heart attacks. Useful, cheaper, faster, and occasionally misleading, because a drug can improve the marker without improving the outcome.

Hard endpoints are the things you actually care about. Did people break bones. Did they have heart attacks. Did their symptoms improve.

Also worth checking: was the outcome measured or reported? Self-reported symptom improvement in an unblinded study is weak evidence, because expectation does a great deal of work. This matters especially in menopause research, where placebo response rates in hot flush trials frequently exceed 30%.

5. How many people, and is this the only study?

Small studies produce unstable results. A trial of 40 women can show a dramatic effect that vanishes in a trial of 400.

More importantly: almost no single study should change your mind about anything. The relevant question is what the body of evidence shows. That's why systematic reviews and meta-analyses — which pool multiple studies and assess their quality — sit at the top of the evidence hierarchy, and why professional bodies produce position statements rather than reacting to individual papers.

If a finding contradicts an established position, it's more likely to be an outlier than a breakthrough. Sometimes it isn't. But the base rate favours caution.

Some practical filters

A press release is not a study. University press offices overstate routinely. If coverage doesn't link the actual paper, be sceptical.

A preprint hasn't been reviewed. Legitimate and useful, but it hasn't been through the check yet.

Check who funded it, and note that funding doesn't automatically invalidate anything. Industry funds a great deal of legitimate research. It's context, not a verdict.

Watch for "linked to". Journalistic shorthand for a correlation, usually observational, frequently weak.

Mice are not women. A startling amount of confident nutritional and hormonal advice traces back to rodent studies.

Where to look instead

If you want a reliable current position on any specific question, the fastest route is the relevant professional body — NICE, The Menopause Society, ACOG, the British Menopause Society. Their statements are systematic reviews translated into recommendations, updated periodically, with the evidence quality graded.

That's less exciting than a headline about a new discovery. It's also, almost always, the thing that turns out to be true a year later.

Sources

- National Institute for Health and Care Excellence. HRT and the likelihood of some medical conditions: a discussion aid for healthcare professionals and patients. 2024. - Manson JE, Aragaki AK, Rossouw JE, et al. Menopausal hormone therapy and long-term all-cause and cause-specific mortality. JAMA. 2017;318(10):927–938. - Freeman MP, Hill R, Brumbach BH. Escitalopram for hot flashes: placebo response in menopause treatment trials. Journal of Women's Health. 2015;24(9):695–701. - Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions. Cochrane, current version.