Introduction
A foundational epistemological
principle underpinning evidence-based medicine (EBM) is based on the
assumption that the estimates of the effects of health interventions are
closer to the “truth” if they are based on higher than on lower
quality (certainty) of evidence (CoE).1 If the
estimated treatment effects are close to the “true” effects, this
would also imply that they would less likely to change as evidence
accumulates after new studies are completed. Conversely, because its
relation to the “truth” is less certain, this also implies that the
estimated effects when evidence is of low quality would more likely
change in future research. Research to date indicates that guideline
panels are willing to issue stronger recommendations when they deem
evidence to be of high quality, thus indirectly affirming this central
EBM assumption.2-5
However, whether this indirect assessment of quality of evidence based
on guidelines panels’ decision-making is accurate is not known. It is
possible that current methods of critical appraisal of CoE do not
discriminate well between “true” accurate from inaccurate estimates of
treatment effects. That is, the effects of health interventions based on
low quality of evidence may turn out to reflect “true effects” by
testing in subsequent studies. On the other hand, what was originally
deemed as high quality evidence may be undermined by future studies more
often than initially expected. Thus, it is not known if low quality
evidence is more often revised than high quality evidence. Empirical
evidence supporting this foundational principle of EBM is lacking.
The main purpose of this report is to assess if a) low certainty
evidence is more often revised than high certainty evidence in
subsequent studies, and if b) the magnitude of effect size differs
between high and low CoE.