Association between Uncertainty and Robustness of Treatment Ranking for
Network Meta-analyses: an empirical study
Abstract
Network meta-analysis (NMA) computes treatment ranking to assist with
clinical decision making, but it is not always clear how reliable the
ranking is and how likely the ranking may be altered by the accumulation
of new evidence. Uncertainty and robustness of ranking are two concepts
related to the reliability of ranking. The uncertainty of ranking can be
measured by the distribution of ranking probabilities, and the
robustness of ranking can be evaluated by the agreement between
treatment ranking of complete data and that of modified data with the
deletion of a specific trial. However, it is still unclear whether these
two approaches would always yield similar conclusions on the reliability
of ranking, i.e. a robust ranking is also one of low uncertainty. The
aim of this study was to investigate the relationship between the
uncertainty and the robustness of treatment ranking by using normalized
entropy and quadratic weighted Cohen’s kappa, respectively, to analyze
60 NMAs. We found that when the uncertainty of ranking is very low,
treatment ranking is unlikely to be altered by the deletion of a trial
from the complete data. However, good robustness of ranking does not
always correspond to low uncertainty. An NMA with a robust treatment
ranking may have high uncertainty of treatment ranking. The uncertainty
of ranking prevents us from naïve interpretation of treatment ranking,
and the robustness of ranking may identify trials included in the
network which have a substantial influence on the treatment ranking.
When an NMA is undertaken, both of them should be evaluated.