Measurement problems in the National Healthcare Disparities Report

The Agency for Healthcare Research and Quality (AHRQ) has issued three National Healthcare Disparities Report, and will issue further such reports. The report principally addresses disparities in receipt of sound healthcare, measuring most of those disparities in terms of relative rates of receiving a procedure or particular quality of care. It also addresses disparities in some health outcome, measuring those in relative rates of adverse outcomes. In examining changes over time, AHRQ has failed to recognize the tendency whereby the rarer an outcome, the greater the relative differences in experiencing the outcome. Thus, it fails to recognize that, because procedure rates are usually increasing and adverse outcomes are usually declining, its approach to measuring will tend to find decreasing disparities in procedure rates and increasing disparities in outcome rates.

AHRQ also examines absolute differences between rates of different groups. In determining whether to report that a disparity has changed, it has adopted a policy of reporting a change only when the relative and absolute differences change in the same direction. That approach may reduce the number of situations where AHRQ reports that there occurred a meaningful change in disparity when the observed change is solely a consequence of change in overall prevalence of the procedure or outcome. But, because absolute differences between rates of different groups also change solely because of changes in prevalence, the approach does not solve the problem of identifying changes in measures of disparity that are not solely consequences of changes in prevalence.