163747 Testing a federal initiative designed to improve evaluation of innovative and emerging local substance abuse prevention programs

Tuesday, November 6, 2007

Wayne Harding, EdM, PhD , Social Science Research and Evaluation, Inc., Burlington, MA
Kim Dash, MPH, MA , Education Development Center, Inc., Newton, MA
Michele Cummins, BA , Social Science Research and Evaluation, Inc., Burlington, MA
Anu Sharma, PhD, LP , Minnesota Institute of Public Health, Mounds View, MN
Tom Griffin, PhD, MSW , Minnesota Institute of Public Health, Mounds View, MN
Brent Garrett, PhD , Louisville Center, Pacific Institute for Research and Evaluation, Inc., Louisville, KY
Tom James, PhD , Institute for Public Affairs, University of Oklahoma, Norman, OK
Eric Albers, PhD , University of Nevada, Reno, Reno, NV
This study examines the implementation and adoption of more rigorous evaluation practices through the Center for Substance Abuse Prevention's (CSAP), Substance Abuse and Mental Health Services Administration's (SAMHSA) Service to Science (STS) Initiative. STS is an intensive, theoretically-derived approach to provide direct and customized evaluation assistance and coaching to program developers, implementers and evaluators representing interventions to prevent substance abuse and related problems. STS is delivered through regional training and technical assistance centers. Using diffusion of innovations and social marketing frameworks, we examined programmatic, organizational and external factors associated with the adoption of rigorous program evaluation practices, such as those described in SAMHSA's National Registry of Evidence-based Programs and Practices (NREPP). Project directors representing 79 programs, which received STS services, participated in retrospective, semi-structured telephone interviews. Case studies were conducted with six of these programs. Results indicated that most programs (62%) participating in the STS initiative intended to (and did) implement more rigorous evaluation and apply for recognition as an evidence-based program. Essential to those decisions (and related evaluation progress) was the supportive working relationship between STS evaluation consultants and program representatives. Other common facilitating factors included organizational commitment to evaluation, clear expectations from the state, adequate funding, financial incentives, and access to other expertise. Common problems included inadequate funding, ambiguous innovation guidelines or evaluation criteria, methodological challenges, and the lack of competent and affordable evaluators. Thus, adoption of complex innovations, such as improving evaluation relevance and rigor, can be facilitated by customized and intensive expert coaching and assistance.

Learning Objectives:
1.Describe those factors that inhibit and facilitate diffusion of evidence-based prevention practices at the local level. 2.Identify which innovative programs are more likely to succeed in the Service to Science technical assistance initiative. 3.Implement strategies for promoting wider dissemination of evidence-based prevention practices.

Keywords: Federal Initiatives, Substance Abuse Prevention

Presenting author's disclosure statement:

Any relevant financial relationships? No
Any institutionally-contracted trials related to this submission?

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

See more of: Other Drugs - Adolescents
See more of: Alcohol, Tobacco, and Other Drugs