Improving evaluation data quality by addressing nuances in commonly-used tobacco prevention and cessation indicators

The Health and Wellness Trust Fund (HWTF) began funding the Teen Tobacco Use Prevention and Cessation Initiative in 2003-04 as the first statewide program in North Carolina to reduce and prevent tobacco use among youth. The Initiative now includes support for 46 local coalitions and statewide organizations that work in several areas of tobacco control including tobacco prevention education, youth empowerment, tobacco-free schools policy, reduction of youth exposure to secondhand smoke, reduction of health disparities, and youth smoking cessation. The University of North Carolina School of Medicine independently evaluates the initiative primarily through an indicator-based progress tracking system (iPTS). Grantees submit monthly reports through iPTS, and UNC cleans the data to be sure they conform to the detailed guidelines set forth in trainings and in the program codebook. During this cleaning process and subsequent review by HWTF, we noted several measures whose definitions became unclear when applied to the local activities of the grantees (e.g. should venues where employees are allowed to smoke in their cars be counted as 100% smoke-free; should a training designed for 30 youth where only one actually attends be counted as a training). This presentation will describe several of the nuances observed in commonly-used indicators for tobacco control among youth and how creating decision rules and tightening indicator definitions can improve data quality.