Mining for Treatment Moderators: An Illustration of Data Mining Techniques Involving Prevention of Behavior Problems in Children

Background: Psychosocial interventions for youth at risk of antisocial behaviors may save thousands or millions of dollars in incarceration, judicial and other costs. Because such interventions are expensive, their cost-effectiveness may depend on their ability to target those youth most likely to benefit, and thus on the ability to identify moderating variables that establish under which conditions or for which groups of participants a given program will be effective. The purpose of this study is to outline the use of data mining techniques as a tool for determining moderators of program effectiveness in a sample of youth receiving a comprehensive intervention to prevent antisocial behaviors. Methods: Using the WEKA 3.4.11 software program, we applied the J48 decision tree algorithm to determine combinations of baseline characteristics predictive of successful intervention outcome (reduction in conduct problems) at 3 different follow-up times. The resulting models represented an exhaustive search of all possible moderating variables in the data set. Results: Significant moderating variables differed according to year of follow-up. Upon cross-validation, models exhibited true positive rates of 0.065- 0.200, and were estimated to classify 82-85% of cases correctly. Conclusions: The models indicated that, for this intervention program, no combination of baseline characteristics consistently predicted successful outcome in all years. Rather, models identified several small subgroups defined by multiple characteristics. Data mining techniques illustrate that pre-screening of participants to determine those most likely to benefit is unlikely to be cost-effective when applied to this program.