Evaluation of a provider profiling intervention for substance abuse treatment programs in Connecticut

With the laudable purpose of improving quality of care, many behavioral health care programs have turned to performance measurement or profiling. However, few studies have documented the effectiveness of these programs. In October, 2005, the Connecticut (CT) Department of Mental Health and Addiction Services (DMHAS) began distributing quarterly profiles to nearly 350 publicly funded substance abuse treatment programs. The two key performance measures: (1) the percentage of clients admitted to a lower level of care within 30 days (continuity) and (2) the percentage of clients readmitted to acute treatment services within 30 days (readmissions). In addition to receiving profiles, two-thirds of CT's providers were randomized to an “enhanced profiling” arm where they received additional coaching and instruction in quality improvement from trained managers. In preliminary analyses, t-tests suggested both continuity and readmissions improved significantly in the two years following the intervention as compared to the two years prior. However, more sophisticated analyses using hierarchical generalized estimating equations with repeated measures analysis indicated that after controlling for time, clustering of patients within programs and case mix, the provider profiling initiative that was adopted in 2005 had no additional impact on either of the performance measures. In addition, there were no significant differences between the “enhanced” and the basic profiling group. Since findings from the recent NIATx evaluation suggest that intensive and well designed interventions can be successful, we discuss major design, implementation and evaluation issues that we encountered to inform the national dialog regarding the effective use of performance measures.