Online Program

283834
When profiling hospitals the shrinkage target matters


Wednesday, November 6, 2013 : 1:30 p.m. - 1:50 p.m.

Arlene Ash, PhD, Department of Quantitative Health Sciences, UMass Medical School, Worcester, MA
The Center for Medicare and Medicaid Services (CMS) uses shrinkage estimators (http://www.medicare.gov/quality-care-finder/) to “Compare Medicare-certified hospitals and the quality of care provided to patients in the country.” Although the aggregate outcomes for patients at different kinds of hospitals often differ markedly, CMS shrinks each hospital's observed outcome, O, towards an expected (E) that reflects only the pooled experience of similar patients admitted to all hospitals. The decision to disallow hospital characteristics to be used in computing E has serious consequences. First, the resulting weighted averages convey the message that our best estimate for the performance of each small hospital is extremely close to average, even for outcomes for which the pooled experience of all patients at small hospitals is notably worse than average. Furthermore, even for fairly large hospitals, the shrunken estimates assign more than half of the weighted average to E rather than O. This makes reported outcomes highly insensitive to any hospital-level factor, including the introduction of improvement initiatives (such as the use of a surgical checklist) or affiliation with a quality-enhancing network, even when such factors are reliably associated with better aggregate patient outcomes. In contrast, if E were allowed to reflect hospital size and the presence of proven quality-improvement initiatives, better hospitals, both small and large, would have more of a chance to have their merit recognized.

Learning Areas:

Biostatistics, economics

Learning Objectives:
Evaluate the consequences of the modeling choices used by CMS and other organizations that produce hospital performance assessments.

Keyword(s): Medicare, Statistics

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am qualified because of my appointment to serve on the COPSS-CMS White paper panel, my PhD in mathematics, my over 30 years experience in health services research, my role as author of many methods papers, and my role as author of several papers and book chapters on statistical issues in provider profiling. This is work that I have conceived and developed as a result of my participation in the above-mentioned COPSS-CMS panel.
Any relevant financial relationships? Yes

Name of Organization Clinical/Research Area Type of relationship
Verisk Health Predictive modeling Employment (includes retainer)

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.