Appropriate comparisons for Acute Myocardial Infarction (AMI) performance measures: The effect of hospital characteristics

Introduction: Since 2002, hospitals across the United States have collected and submitted data on National Hospital Quality (NHQ) Measures in order to meet accreditation and regulatory requirements of JCAHO, CMS, and other stakeholder organizations. Immutable characteristic differences between hospitals—region, patient volume, and setting—have a clear impact on performance. Gaining an understanding of these differences and their effects on performance is essential in developing appropriate comparisons and performance goals, particularly given the growing trend to use the NHQ measures for public reporting and pay-for-performance. This study addresses three questions: 1) Is there a significant trend in AMI performance over time?, 2) Is there a significant variance in performance among different hospital characteristic groups?, and 3) Are the trends in different groups significantly different? Methods: The study examines two and a half years' of performance data (1st quarter 2004 - 2nd quarter 2006) for approximately 500 U.S. hospitals submitting data through the Quality Indicator Project® for six AMI NHQ process measures. We conducted a preliminary analysis based on hospital composite scores, followed by a confirmatory analysis using a mixed random coefficients model to compare hospitals' performance over time to predict the effect of each characteristic on overall performance. The model examined five hospital characteristics: discharge volume; setting; region; facility type; and completeness of data, which is intended to examine the impact of data submission practices on performance. Results: Four of the five characteristics studied—the exception being facility type—were found to be statistically significant in predicting the level of hospital performance (showing a p-value of 0.01 or less). Within each of the characteristics, hospitals exhibiting specific traits (e.g., rural versus urban setting) had consistently lower or higher performance, though all groups showed similar rates of improvement. The study found that those hospitals submitting data on all relevant measures for all time periods, exhibited consistently higher performance. Conclusion: This study highlights differences in hospital performance for the NHQ AMI Measures. As regulators, payers, and other stakeholders move forward in using NHQ data for accreditation, public reporting, and pay-for-performance, they risk using arbitrary comparison groups and setting improvement goals not supported by research. Describing differences in performance is a critical first step to defining more appropriate comparison groups that take differences into account and make public reporting more meaningful and accurate.