Challenges and triumphs in implementation science: Measuring the impact of a low-tech, low cost obstetric and neonatal emergency simulation training program (PRONTO) in Mexico, Guatemala, and Kenya

Background: A skilled birth attendant during delivery saves lives, but there is little evidence for the effectiveness of training programs teaching emergency obstetric and perinatal care. PRONTO is an innovative approach to training using low-tech simulation and team training for low-resource settings. During implementation we aimed to rigorously test its impact in Mexico, Kenya, and Guatemala. Methods: We used matched pair cluster randomized designs in Mexico and Guatemala. Hospitals and clinics were matched by infrastructure and number of providers and deliveries attended, then randomized to intervention or control. In Kenya we are using a quasi-experimental design with 27 intervention clinics and 18 comparison clinics. Process and outcome measures for maternal/perinatal complications and mortality rates are collected in intervention and control sites. Results: As of early 2013 we have trained 757 providers (217 in Guatemala, 436 in Mexico, and 104 in Kenya). Significant improvements have been documented in providers' knowledge and self-efficacy. Provider teams have achieved system change goals in all sites. Challenges in Mexico in measuring rates of childbirth complications have informed data collection strategies in Guatemala and Kenya, including adaptation of the WHO “near miss” approach to measuring maternal and perinatal morbidity and mortality. Conclusions: Efficacy of training programs for obstetric and neonatal emergencies cannot be taken for granted. Rigorous implementation evaluation is critical to guide policy. PRONTO offers an innovative training approach with growing evidence. Our reflections and recommendations can help other implementation scientists in maternal health worldwide to plan or refine study design and data collection strategies.