When profiling hospitals the shrinkage target matters

The Center for Medicare and Medicaid Services (CMS) uses shrinkage estimators (http://www.medicare.gov/quality-care-finder/) to “Compare Medicare-certified hospitals and the quality of care provided to patients in the country.” Although the aggregate outcomes for patients at different kinds of hospitals often differ markedly, CMS shrinks each hospital's observed outcome, O, towards an expected (E) that reflects only the pooled experience of similar patients admitted to all hospitals. The decision to disallow hospital characteristics to be used in computing E has serious consequences. First, the resulting weighted averages convey the message that our best estimate for the performance of each small hospital is extremely close to average, even for outcomes for which the pooled experience of all patients at small hospitals is notably worse than average. Furthermore, even for fairly large hospitals, the shrunken estimates assign more than half of the weighted average to E rather than O. This makes reported outcomes highly insensitive to any hospital-level factor, including the introduction of improvement initiatives (such as the use of a surgical checklist) or affiliation with a quality-enhancing network, even when such factors are reliably associated with better aggregate patient outcomes. In contrast, if E were allowed to reflect hospital size and the presence of proven quality-improvement initiatives, better hospitals, both small and large, would have more of a chance to have their merit recognized.