Comparing Hospital Performance: Developing more appropriate method using Peer Groups

As public reporting and pay-for-performance for hospitals based on the National Hospital Quality (NHQ) Measures gains momentum, questions arise about appropriate comparisons of hospitals. Research shows that hospital bedsize, region, or teaching status can predict differences in performance, yet current methodologies continue to compare hospitals to national and state aggregates. Given the knowledge of variation in health care practices, current approaches may not be appropriate.

The Quality Indicator Project® conducted a retrospective analysis of Acute Myocardial Infarction (AMI), Heart Failure (HF), Pneumonia (PN) and Surgical Infection Prevention (SIP) on NHQ Measure data collected from 442 customer hospitals during 2005 and 2006. Principal Component Analysis and Analysis of Variance (ANOVA) suggested that bedsize, teaching status, type, setting, and region are the most important hospital characteristics explaining variation in performance. We used Factor Analysis to decompose relevant hospital characteristics and Cluster Analysis to stratify hospitals into four peer groups.

Peer Group 1 consisted predominantly of hospitals with more than 200 beds, Peer Group 2 included mostly non-teaching hospitals, all of the hospitals in Peer Group 3 are located in the Midwest, and Peer Group 4 consisted of Critical Access Hospitals. Peer Group 3 had the highest mean performance across all of the clinical conditions (AMI, HF, PN and SIP), while the Peer Group 4 had the lowest scores in all but AMI. In addition, when mean performance for AMI, HF, PN and SIP measure sets was compared across the established peer groups, significant difference in performance were apparent. For HF, Peer Groups 1 and 3 had means (0.833, 0.850) that are significantly different from those of Peer Groups 2 and 4 (0.794, 0.790, respectively). The mean performance in PN was highest for Peer Group 3 (0.892) and significantly different from those of Peer Groups 1 and 2 (0.855, 0.865), which also had significantly higher means than Peer Group 4 (0.833). There was no significant difference between the means for Groups 1 and 3 for SIP (0.826 vs. 0.802), but Group 3 had means that are significantly different from those of Peer Groups 2 and 4 (0.782, 0.779, respectively).

The results from this study have important implications for pay-for-performance and public reporting, since the differences in hospital performance and their capacity to care for patients is not currently being considered sufficiently in the methods and comparison groups that are used to assess hospitals.