Improved statistical methods for meta-analysis of rare events

Meta-analysis (M-A) of proportions, such as the incidence of clinical events among a cohort of patients or the response rate among patients receiving a certain treatment regimen, provides systematically synthesized evidence for medical and public health decisions. Our research is motivated by a recently published M-A, which evaluated the proportion of patients with preventable adverse drug reactions (PADRs). Data for the M-A were extracted from 16 studies conducted in 11 countries. Among 48797 emergency visits or hospitalizations, only 3% were identified to have PADRs. The conventional random effects model for M-A of proportions assumes within-study variation to be normally distributed. Due to failure in our ability to check this assumption, the estimation of rare events such as PADRs is expected to be biased under the conventional method and the exact methods based on binomial distributions is recommended for use. We compare the operating characteristics of two existing exact approaches---normal binomial (N-B) and beta-binomial (B-B) models ---through an extensive simulation study. In addition, we incorporate the empirical ("sandwich") estimator of variance into both models to improve the robustness of the statistical inference. The simulation study shows that B-B tends to have significantly smaller bias and mean squared error than N-B for truly rare events (<=5%) , while N-B outperforms B-B for relatively common events (> 5% but <=25%). Use of the sandwich estimator of variance improves the precision of estimation for both models. We illustrate the two approaches by applying them to our motivating example.