Evidence of effectiveness

This is the sixth and final 40 minute presentation in the full-day Learning Institute, “Social norms marketing interventions: Context, evidence, planning, implementation & evaluation." This presentation will consist largely of guided discussions based on three types of evidence. Each type will be referenced and illustrated using appropriate examples from the published literature.

The first type of evidence can be called informal evidence and consists of anecdotal, descriptive accounts and use of descriptive data in the absence of any formal methodology. The least time will be spent on this type of evidence. The primary goal of the participant discussion will be to identify the weaknesses and inadequacies of this type of evidence.

The presentation will then move into a more detailed consideration of the literature that illustrates intervention effects with more rigorous methodology and higher quality data. This type of evidence will be related back to the strategic logic model. Participants will be lead in a discussion of the degree to which the evidence addresses each component of the logic model.

An important part of evaluating the adequacy of the evidence is attention to whether or not those who are most at risk have been impacted by the intervention. All too frequently, public health education can be a version of “preaching to the choir.” The strategy of “audience segmentation”, first introduced in the 5 Step Model as part of the first step, “Understand Your Audience” will be re-introduced here in the context of evaluation of effectiveness. Two schemes for segmenting audiences for evaluation purposes will be reviewed. The first is generic to public health interventions and consisting in identifying and collecting the appropriate data to stratify by risk level. A simple scheme for stratifying into low, moderate and high risk, by status on two variables will be introduced. The variables are frequency of engaging in the risk behavior and quantification of the risk behavior (e.g., days smoking and number of cigarettes per day; days drinking and drinks per episode). A second scheme for audience segmentation is particular to social norms interventions. This will be introduced and explained with the aid of animated graphics. This is segmentation according to accuracy of perception and personal behavior. Each approach will be illustrated with relevant data.

The highest form of evidence is from a randomized control trial (RCT) and careful attention will be paid to the best example for social norms marketing: deJong's NIAAA-funded RCT, and its replication, using a social marketing intervention to address high risk drinking in college students. This is particularly instructive as the first study demonstrated effectiveness while the replication failed to demonstrate effectiveness. In subsequent analyzes, the failure seemed to be associated with environmental differences (i.e., saturation of alcohol outlets in close proximity to the school). Consideration of these studies thus facilitates discussion with participants of a number of important issues, including the limitations of even the best evidence and the importance of context.

The presentation will conclude with the use of the social-ecological model to discuss the context of public health interventions and the place of social norms interventions within that context.