Recommendations for conducting evaluations of scaled-up program delivery: Outcomes of scaling-up an evidence-based walking program across Oregon
The Re-AIM model was used to guide the evaluation approach. Process and outcome evaluation tools were developed from existing program resources and data collection was integrated into implementation protocols. Process data were collected by researchers through interviews with leaders (n=39) and class observations (n=33). Participant (n=598) pre/post health surveys were collected by volunteer leaders that delivered the program.
Sixty-six percent of sites successfully completed data collection. Best-practices identified in the process evaluation included training leaders for program delivery and data collection in-person vs. online, and developing partnerships across multiple sectors to share resources and increase reach. Fidelity scores varied across sites. Outcomes showed that most participants fit the programs’ target audience: 71% had arthritis; 38% did not meet physical activity guidelines pre-program. Mean class attendance was 8.5(SD=2.4). Self-reported pain and fatigue were significantly reduced post-intervention (p<.01; p<.05, respectively)
Using multiple methods for evaluation yielded important insights into factors facilitating evaluation, scale-up and delivery. Outcome evaluation, however, was challenged by incomplete data collection and poor participant retention. We describe our approach, methods, challenges and findings in conducting this large-scale evaluation and share key tools we used. We also offer practical methodological recommendations for future efforts.
Learning Areas:Conduct evaluation related to programs, research, and other areas of practice
Implementation of health education strategies, interventions and programs
Public health or related research
Systems thinking models (conceptual and theoretical models), applications related to public health
Define scaling up and explain importance of evaluation when scaling up programs Discuss approaches to reducing costs while maximizing impact of large-scale evaluations
Keyword(s): Evaluation, Outcomes Research
Qualified on the content I am responsible for because: I am designed, implemented, and evaluated the scale up of the Walk With Ease program via the OSU Extension Service. I am a PhD student in Public Health, with experience in program implementation and evaluation of evidence-based programs
Any relevant financial relationships? No
I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.