Comparison of Combining Methods for Multiple Quality Indicators on the VBP Measures

Currently different composite scores are used to summarize the overall performance from multiple indicators for internal and public reporting of National Hospital Quality measures. As CMS proposes Value-Based Purchasing (VBP) as a potential model for determining rewards and incentives for pay-for-performance program based on single summary score for each hospital, we compare the VBP composite score with four others composite scores: overall pass rate (OPR), average patient rate (APR), average indicator rate (AIR) and perfect patient rate (PPR).

APR and PPR belong to the patient adjusted method that calculates patient pass rates first and AIR and VBP to the indicator adjusted method that calculates indicator (measure) pass rates first. With PPR as a derived method from APR requiring a perfect criterion on the patient rates, VBP can be viewed as a derived method from AIR scaling the indicator rates into “attainment” or “improvement” scores.

This study includes 387,624 patients from 453 hospitals participating National Hospital Quality Measures through the Quality Indicator Project for 16 process-of-care VBP measures for 2006. The comparisons were carried over in three ways: distribution analysis, pairwise agreement analysis, and linear regression on hospital characteristics. Five statistics were computed to measure the ranking agreement between any pair of two methods: rank correlation coefficient, number of >25% changes, average of rank changes, percentage of inconsistent pairs, kappa coefficient and misclassification rate. The results of agreement analysis can reflect the structure of the data.

PPR and VBP (derived methods) are distributed with lower centrals and wider variances than other methods, suggesting more opportunity for quality improvement. On the other hand, OPR, APR and AIR are distributed closed with higher centrals and smaller variances. However, similar distributions do not necessarily result in similar hospital rankings. High ranking agreement between OPR and APR suggests that the non-eligible cases are distributed quite evenly along patient arrays and that the variance of patient pass rates is small. Moderate agreement between OPR and AIR suggests variant indicator pass rates along the indicator arrays. High agreement between APR and PPR implies that the perfect patients are distributed quite randomly between hospitals. Disparity between AIR and VBP results from hospitals with credits gained by improvements. Lower predictive power of the VBP model reflects the loosened associations between the hospital characteristics and the rewarding criteria, encouraging more hospitals to improve quality regardless of their characteristics.