335371
Challenges and rigor in evaluating early childcare centers' adoption of nutrition policies and practices: Lessons learned from Transforming Our Community's Health (TOUCH) Community Transformation Grant Evaluation
Methods: To determine the extent to which the county-wide initiative improved the nutrition environments of ECCs, we had to respond to evaluation resource constraints and target population access issues. We balanced the costs of a resource-intensive data collection protocol with available organizational resources, and we employed valuable community partners as liaisons between the target population and evaluation team.
Results: To ration resources, while maintaining scientific rigor, we utilized nutrition-specific sections of a validated observation protocol, the Environment and Policy Assessment and Observation (EPAO) protocol, by way of a trained team of EPAO observers that went out in pairs to establish inter-rater reliability, before going out individually, during two waves of data collection, Spring 2013 and Spring 2014.
Discussion: The challenges we faced in evaluating nutrition policy and practice changes in this intervention entailed ECCs resisting participation in evaluation activities and time/person-power spent collecting pre/post data. Nevertheless, our approach ensured scientific rigor through high inter-rater reliability utilizing a truncated form of a validated instrument. Accordingly, we had confidence in our conclusions and credible feedback for our partners.
Learning Areas:
Conduct evaluation related to programs, research, and other areas of practicePlanning of health education strategies, interventions, and programs
Learning Objectives:
Explain potential barriers involved in evaluating programs within early childcare centers.
Design evaluation plans that ensure methodological rigor despite systemic, organizational, and personal barriers and challenges to evaluation activities.
Discuss the potential trade-offs between feasibility and scientific rigor within frontline, community-involved evaluation.
Keyword(s): Evaluation, Child Health Promotion
Qualified on the content I am responsible for because: I am an Assistant Professor of Public Health, and I practice and teach public health program planning and evaluation. I was an evaluator for a Centers for Disease Control and Prevention Community Transformation Grant received by the Broward Regional Health Planning Council in Broward County, Florida, in which I led the evaluation of an early childcare center nutrition intervention. Additionally, I am a coauthor on three peer reviewed journal articles reporting childhood obesity research.
Any relevant financial relationships? No
I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.