261458 Step 5: Evaluation using a strategic logic model

Sunday, October 28, 2012 : 1:40 PM - 2:20 PM

Adrienne Keller, PhD , Department of Student Health, National Social Norms Institute at the University of Virginia, Charlottesville, VA
This 40 minute presentation, the fifth in the full-day Learning Institute, “Social norms marketing interventions: Context, evidence, planning, implementation & evaluation," will guide the audience through the creation of a strategic logic model for social norms marketing interventions, and is based largely on the published work for Keller and Bauerle. The presentation will begin with a brief consideration of the gold standard for evidence of effectiveness, randomized control trials, and a brief guided discussion of what might constitute evidence in the absence of a randomized control trial. This will lead participants to the necessity of strengthening the “inferential line” connecting an action to an outcome.

Following this, the presentation will focus briefly on the differentiation of strategic planning from tactical planning. In general, logic models focus more on the tactical level rather than the strategic level of program planning and evaluation. That is, they specify in some detail the tactics that will be used to accomplish the project, rather than the strategic vision that informs the project. In applying a logic model to social norms interventions, we embedded the tactical within the strategic.

This approach offers three levels of integration. It integrates evaluation into program planning and promotes thinking of evaluation as integral task of program planning and implementation rather than as a separate necessity. It also integrates the strategic with the tactical, ensuring that the implementation of a program corresponds to the strategic vision for the program. And it integrates sources of data into program components, ensuring that adequate information exists to relate the process of the program to the outcome. The second advantage, related to the integration aspect, is simplicity and clarity that can be easily communicated to stakeholders, funders and other interested parties. It condenses the essential elements of an intervention and its evaluation into a single figure that can be demonstrated and explained in a straight-forward manner. Similarly, such a model can also be used to ensure that all program staff are oriented to the overall goals, strategies, tactics and evaluation of a program. Thirdly, as programs are held to higher standards of accountability, this form of logic model represents a reasonable compromise between relying exclusively on evidence of effectiveness from the literature and expert testimony, on the one hand, and facing the cost-prohibitive challenge of a formal research study, on the other hand.

Participants will then be guided through a discussion, with generous use of diagrams and illustrations, of the five components of the strategy logic model. The first component, identification of the underlying problem, is crucial: the effect of a social norms intervention is achieved solely through correction of a misperception, not by directly challenging unhealthy behaviors. If there is no misperception and/or if there is no correlation between misperception and personal behavioral choices, then theoretically a social norms approach could not have a beneficial effect. The second component, intervention strategy, specifies the type of intervention that will be used. In this part of the presentation, participants will be introduced to a variety of forms of interventions incorporating social norms at different levels of prevention (universal, indicated, targeted), including small groups, individualized feedback and Brief Motivational Interviewing. The third component specifies the immediate goal of the intervention and is crucial to anticipating the benefits of the intervention. The immediate goal is simply to reduce the misperception. The fourth component of this logic model specifies the outcome that is expected if the misperceptions are corrected. In guided discussion, participants will relate both component three and component four back to the identification of underlying problems in component one. Components three and four delineate the correction of the underlying problems. Component five then specifies the impact that correcting the underlying problems will have in epidemiological terms: reduced morbidity and mortality. The strategic logic model builds from individual effects to population effects. The presentation will conclude with a discussion of how to incorporate the tactical into such a strategic logic model; that is, what measurements and data are necessary for each component. In particular, the importance of gathering valid and reliable information relevant to the second component, the intervention strategy will be emphasized. In the absence of a research design with intervention and control groups, only by demonstrating the fidelity of the intervention to an evidence-based strategy can the remediation of the underlying problems be logically associated with the intervention. In the best sense, fidelity means that the essential components of a model are understood and then adapted, as necessary and while remaining “faithful” to original model, to the particular context and population

Following this presentation, participants will have the opportunity to work in small groups to identify the components of an applied logic model. The small group work will then be reviewed and critiqued with the participants as a whole.

Learning Objectives:
1. Discuss the problems with evidence of effectiveness in the absence of a randomized controlled trial. 2. Discuss the advantages offered by developing a strategic logic model 3. Identify the five components of a strategic logic model for social norms interventions 4. Formulate the evidence needed for each component 5. Demonstrate the ability to work with a group to create a logic model

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I have over 20 years as a behavioral epidemiologist, with experience in a wide variety of preventive and treatment interventions. For the last 5 years, I have been Research Director of the National Social Norms Institute. During that time, I have developed and published a strategic logic model for social norms interventions, as well as published and presented extensively on the social norms models, particularly evaluation and data management.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.