Evaluation in the context of strong ideology: Ethical challenges for abstinence education evaluators

Twenty years since funding for abstinence-until-marriage programs began, there remains limited evidence of their effectiveness. Researchers who evaluate programs that are deeply rooted in ideology, however, are often aligned with particular types of programs, and rarely cross the ideological divide. Two questions that should be asked when conducting evaluation of programs that are ideologically driven are: 1) if a program did not achieve the desired result, would those who were ideologically motivated want to know the answer and 2) if the answer was not what they sought, what would they be prepared to do about it? One popular abstinence education program sought to demonstrate evidence of its effectiveness, believing that working with an evaluator who was not seen as an “abstinence evaluator” would give credence to the findings. A study was conducted in 6 high schools, randomly assigned to the curriculum and a standard text-book based health program. When the evaluation demonstrated a significant short-term behavioral impact, but not a longer-term impact, the organization agreed to publish the data with a peer review stamp of approval, including the study's limitations, both in the methods and the implications of the findings. Subsequently, however, the organization and a national clearinghouse published a press release that exaggerated and incorrectly reflected the reported findings, creating both ethical and practical concerns for the evaluator. The presentation will describe the evaluation, its actual implications, the ethical and practical considerations in evaluating ideologically-driven programs, and the implications of such evaluation work for policy advocacy.