Consequences of multiple imputation of missing standard deviations and
sample sizes in meta-analysis
Abstract
Meta-analyses often encounter studies with incompletely reported
variance measures (e.g. standard deviation values) or sample sizes, both
needed to conduct weighted meta-analyses. Here, we first present a
systematic literature survey on the frequency and treatment of missing
data in published ecological meta-analyses showing that the majority of
meta-analyses encountered incompletely reported studies. We then
simulated meta-analysis data sets to investigate the performance of 14
options to treat or impute missing SDs and/or SSs. Performance was
thereby assessed using results from fully informed weighted analyses on
(hypothetically) complete data sets. We show that the omission of
incompletely reported studies is not a viable solution. Unweighted and
sample size-based variance approximation can yield unbiased grand means
if effect sizes are independent of their corresponding SDs and SSs. The
performance of different imputation methods depends on the structure of
the meta-analysis data set, especially in the case of correlated effect
sizes and standard deviations or sample sizes. In a best-case scenario,
which assumes that SDs and/or SSs are both missing at random and are
unrelated to effect sizes, our simulations show that the imputation of
up to 90% of missing data still yields grand means and confidence
intervals that are similar to those obtained with fully informed
weighted analyses. We conclude that multiple imputation of missing
variance measures and sample sizes could help overcome the problem of
incompletely reported primary studies, not only in the field of
ecological meta-analyses. Still, caution must be exercised in
consideration of potential correlations and pattern of missingness.