Online Program

Challenges and rigor in evaluating early childcare centers' adoption of nutrition policies and practices: Lessons learned from Transforming Our Community's Health (TOUCH) Community Transformation Grant Evaluation

Monday, November 2, 2015 : 8:30 a.m. - 8:50 a.m.

T. Lucas Hollar, PhD, Master of Public Health Program, College of Osteopathic Medicine, Nova Southeastern University, Fort Lauderdale, FL
Nicole Cook, PhD, MPA, Master of Public Health Program, College of Osteopathic Medicine, Nova Southeastern University, Fort Lauderdale, FL
David Quinn, BS, MPH, Master of Public Health Program College of Osteopathic Medicine, Nova Southeastern University, Fort Lauderdale-Davie, FL
Teina Phillips, MPA, TOUCH Program Director, Broward Regional Health Planning Council, Hollywood, FL

Michael De Lucca, MHM, Broward Regional Health Planning Council, Inc., Hollywood, FL
Background: Early childcare centers (ECC) are gaining attention as valuable public health arenas for addressing childhood obesity. However, ECCs involve a number of challenges to conducting rigorous evaluation activities. In the midst of resource constraints, inter-organizational challenges, and methodological necessities, we sought to evaluate the extent to which training early childcare providers in the implementation of evidence-based nutrition practices improved the nutrition policies and practices of ECCs serving racially and ethnically diverse, low-income children in Broward County, Florida.

Methods: To determine the extent to which the county-wide initiative improved the nutrition environments of ECCs, we had to respond to evaluation resource constraints and target population access issues. We balanced the costs of a resource-intensive data collection protocol with available organizational resources, and we employed valuable community partners as liaisons between the target population and evaluation team.

Results: To ration resources, while maintaining scientific rigor, we utilized nutrition-specific sections of a validated observation protocol, the Environment and Policy Assessment and Observation (EPAO) protocol, by way of a trained team of EPAO observers that went out in pairs to establish inter-rater reliability, before going out individually, during two waves of data collection, Spring 2013 and Spring 2014.

Discussion: The challenges we faced in evaluating nutrition policy and practice changes in this intervention entailed ECCs resisting participation in evaluation activities and time/person-power spent collecting pre/post data. Nevertheless, our approach ensured scientific rigor through high inter-rater reliability utilizing a truncated form of a validated instrument. Accordingly, we had confidence in our conclusions and credible feedback for our partners.

Learning Areas:

Conduct evaluation related to programs, research, and other areas of practice
Planning of health education strategies, interventions, and programs

Learning Objectives:
Explain potential barriers involved in evaluating programs within early childcare centers. Design evaluation plans that ensure methodological rigor despite systemic, organizational, and personal barriers and challenges to evaluation activities. Discuss the potential trade-offs between feasibility and scientific rigor within frontline, community-involved evaluation.

Keyword(s): Evaluation, Child Health Promotion

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am an Assistant Professor of Public Health, and I practice and teach public health program planning and evaluation. I was an evaluator for a Centers for Disease Control and Prevention Community Transformation Grant received by the Broward Regional Health Planning Council in Broward County, Florida, in which I led the evaluation of an early childcare center nutrition intervention. Additionally, I am a coauthor on three peer reviewed journal articles reporting childhood obesity research.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.