Understanding Evidence Ratings

When a health recommendation is described as “strongly supported,” what does that actually mean?

Does it mean the treatment produces a large benefit? That it works for everyone? That it has no risks?

An evidence rating cannot answer all those questions on its own.

Its purpose is to help us understand how much confidence we can place in a particular claim. To use that rating well, we also need to understand what was studied, who was studied and which outcome was measured.

Confidence and benefit are different

We can be highly confident that an intervention produces a small benefit. We can also have considerable uncertainty about an intervention that appears to produce a large benefit in an early study.

Those are different situations.

For example, imagine one treatment has been studied repeatedly and produces a modest improvement in symptoms. Another produces an impressive result in one small trial, but the finding has not yet been confirmed.

The second treatment may deserve further investigation. The first may offer a more predictable result.

An evidence rating should help make that distinction visible.

What do ratings assess?

Formal systems such as GRADE describe certainty as high, moderate, low or very low. These ratings apply to the evidence for a particular outcome, rather than giving a treatment one permanent score.

Assessment considers issues such as study limitations, inconsistent findings, small samples, uncertainty in the results and whether the research directly answers the clinical question. Missing or selectively published results can also affect confidence.

This means that a large number of studies does not automatically equal strong evidence. Their quality and relevance matter.

Evidence belongs to a specific purpose

An intervention may have good evidence for correcting a deficiency, uncertain evidence for improving symptoms in people without that deficiency, and little evidence for preventing a completely different disease.

The name of the intervention has stayed the same. The claim has changed.

Similarly, evidence that a treatment improves a blood marker does not automatically establish that it prevents complications, improves daily functioning or extends life.

Whenever we see an evidence rating, a useful question is:

“Evidence for which outcome, in which people?”

A rating does not make the decision for you

Even when confidence in the research is high, the practical decision still involves the size of the benefit, possible harms, cost, inconvenience and personal preferences.

A treatment can be well researched yet unsuitable for a particular person. Another may involve uncertainty but still warrant a careful discussion in a specific situation.

Lower certainty does not automatically mean an intervention is ineffective. It means our confidence is limited. High certainty also does not guarantee that every individual will benefit.

Look for an explanation behind the label

Stars and labels can make information easier to scan, but their meaning depends on the system being used. A five-star scale is not automatically equivalent to a formal scientific grading system.

A useful rating should explain what it measures, identify the relevant outcome and acknowledge important limitations.

The philosophy guiding my work is to make that reasoning easier to understand: connect each option with a clear purpose, communicate uncertainty honestly and consider how benefit will be assessed.

A rating is a starting point for understanding the evidence. The explanation behind it is what helps us make a considered decision.

Dr D. Michael ElliottM.Sc., D.C., CFMP