Multilevel statistical modeling of hospital mortality after injury: Software comparisons

Background: Multilevel statistical models have theoretical advantages when subjects are clustered within groups, but methods for binary outcomes (e.g., mortality) have been more difficult to develop. The purpose of this study was to compare some of the software packages available for multilevel logistic regression modeling.

Methods: Data on injured patients were obtained from the National Trauma Data Bank (NTDB, Version 6.0, including 487,776 subjects in 165 hospitals) and the Nationwide Inpatient Sample (NIS, 2003, including 200,826 subjects in 461 hospitals). Two-level logistic regression models were specified in which the hospital effect on patient mortality was modeled as a random effect producing a different intercept for each hospital. Results were compared using the special-purpose software packages HLM and MLwiN, and the general-purpose packages Stata and SAS.

Results: The latest algorithms of the current versions of the four software packages produced similar results, although this was not true for recent but earlier algorithms. Covariate effects were virtually identical regardless of the method used, but random variance was not as robust to the computational algorithm. “User-friendliness” varied among the packages.

Conclusions: Rapid progress in computation is making multilevel methods increasingly available to public health researchers. Different statistical software products have different practical strengths and weaknesses, which may be important factors when deciding which to use for a given project. At the present stage of development, it may be reassuring to use more than one algorithm and/or more than one type of statistical software to verify results for binary outcome models.