Benchmarking patient safety and quality in US hospitals: The stochastic frontier approach

Public reporting and paying for quality are used by public and private payers to incentivize hospitals to improve performance. Quality benchmarks of the Centers for Medicaid and Medicare Services' (CMS) are national average scores of quality and safety indicators. CMS ranks providers according to whether they are above, at, or below the national average on a number of indicators. However, there is no evidence that an average is a good indicator by which to measure the quality of hospital care or to compare care across hospitals. We use stochastic frontier analysis (SFA) to: 1) develop scientifically derived “best practice” targets that are based on the gaps between observed and best possible outcomes for the CMS' Hospital Quality Initiatives (HQI) outcomes measures; 2) ascertain the distributions of these gaps in a large national sample of U.S. hospitals; and 3) determine the factors that could be changed, and the direction and magnitude of change, in order to reduce those gaps. Quality inputs in the SFA model were indicators for capital, labor, and technology. Quality outputs in the SFA model were 6 CMS hospital quality initiatives (heart failure and pneumonia mortality and readmission, a patient safety composite, and composites of mortality due to selected medical conditions and surgical procedures). Variables in the predictor model included financial, organizational, and market indicators. The sample was all hospitals that matched on analysis variables from the following sources: American Hospital Association Annual Survey, CMS cost reports, Area Resource File, and the CMS RHQDAPU file (1,823 – 2,784). Truncated normal SFAs with bootstrapping for heteroskedasticity were conducted. SFA outputs were the best possible quality scores for each hospital and the gaps between those scores and the hospital's actual scores. Gap distributions were calculated. Poisson regressions were run for prediction analysis. All SFA inputs contributed significantly to variation in the best practice measures. The top 5% of hospitals had gaps of 0-9%. The top 10% of hospitals had gaps of 0-15%. Analysis of the distribution of gaps shows that 52 -94% of hospitals had > 10% quality and safety gaps while 0-15% of hospitals had > 50% quality and safety gaps. Profit margin, ownership and percent of for-profit hospitals in the region were not strong predictors of gaps. Payer mix, RN staffing, size, case mix index, accreditation, being a teaching hospital, market share, urban location, and region were strong predictors of gaps.