Network meta-analysis (NMA) computes treatment ranking to assist with clinical decision making, but it is not always clear how reliable the ranking is and how likely the ranking may be altered by the accumulation of new evidence. Uncertainty and robustness of ranking are two concepts related to the reliability of ranking. The uncertainty of ranking can be measured by the distribution of ranking probabilities, and the robustness of ranking can be evaluated by the agreement between treatment ranking of complete data and that of modified data with the deletion of a specific trial. However, it is still unclear whether these two approaches would always yield similar conclusions on the reliability of ranking, i.e. a robust ranking is also one of low uncertainty. The aim of this study was to investigate the relationship between the uncertainty and the robustness of treatment ranking by using normalized entropy and quadratic weighted Cohen’s kappa, respectively, to analyze 60 NMAs. We found that when the uncertainty of ranking is very low, treatment ranking is unlikely to be altered by the deletion of a trial from the complete data. However, good robustness of ranking does not always correspond to low uncertainty. An NMA with a robust treatment ranking may have high uncertainty of treatment ranking. The uncertainty of ranking prevents us from naïve interpretation of treatment ranking, and the robustness of ranking may identify trials included in the network which have a substantial influence on the treatment ranking. When an NMA is undertaken, both of them should be evaluated.