FIVE STARS DOESN'T MEAN FIVE TIMES

By Dr D. Michael Elliott, M.Sc., D.C., CFMP 

A five-star rating attracts attention. It offers a quick impression that something deserves consideration. 

But what do those stars actually tell you? 

In health, a rating needs context. Five stars cannot tell you that an intervention delivers five times the benefit, works five times faster or is five times more suitable for you. 

Before using a rating to make a decision, we need to understand what it measures. 

Evidence, benefit and suitability are different questions 

An evidence rating might describe how confidently research supports a particular claim. A customer rating might describe satisfaction. A recommendation score might combine several considerations. 

These are different measures. 

Even when research provides strong evidence, the benefit may be modest. Conversely, an early study may report a large improvement while leaving considerable uncertainty about whether that result will hold up. 

Evidence certainty and effect size need to be considered separately. Clinical recommendations also require attention to benefits, harms and the outcomes that matter to patients. 

Think of three questions: 

• How confident are we in the evidence? 

• How much benefit was demonstrated? 

• How relevant is that benefit to this person? 

A row of stars cannot answer all three. 

A rating belongs to a specific purpose 

An intervention can have useful evidence for one outcome and limited evidence for another. 

For example, research supporting an improvement in a blood marker does not automatically demonstrate improved energy, better physical performance or longer life. 

The same principle applies to populations. Findings in people with a confirmed deficiency may not apply to people whose levels are already adequate. 

When you see a highly rated intervention, ask: Highly rated for what—and for whom? 

More highly rated interventions do not automatically make a better plan 

It is tempting to collect the highest-rated options and assume that combining them will maximise results. 

But several interventions may overlap. Some may add cost or complexity without adding a meaningful benefit. Others may introduce interactions or make a routine difficult to sustain. 

A useful health plan gives each intervention a clear job. 

What problem is it intended to address? What change would justify continuing it? How will we know whether it is helping? 

Those questions make a recommendation more useful than its rating alone. 

Read the explanation behind the stars 

A helpful summary should explain the evidence, the expected benefit, the limitations and the circumstances in which an intervention deserves consideration. 

It should also distinguish between outcomes. A change in a laboratory result and an improvement in daily functioning are both measurable, but they tell us different things. 

This is one of the principles guiding the health app I am developing: helping users understand the reasoning behind an option, including its purpose, evidence and practical limitations. 

Quick summaries can help us navigate information. The detail helps us make a decision. 

Five stars may earn your attention. Understanding what they represent helps you decide what deserves a place in your health plan.