A novel approach to workshop evaluation: An enhanced role for observers

Background: Evaluations of public health trainings are usually based on trainee reactions post-event and rarely on their own actions or behavior during the training; however, this information may be equally valuable and is often overlooked. In the US-Mexico border region, binational communication is critical to effective public health practice; we explored whether observers recording proceedings of a three-day bilingual training to build capacity among US-Mexico teams of health professionals could also document participant behavior that would be useful in improving future events. Methods: Included on the form capturing workshop proceedings were seventeen open-ended questions which prompted observers to document in a semi-structured manner trainee/trainer attentiveness, interaction, and level of involvement during each workshop session. Three bilingual observers recorded trainee/trainer behaviors in real time. This information was later added to trainee/trainer post-workshop feedback to obtain a more comprehensive evaluation from multiple perspectives. Results: Observers identified variations in attentiveness, interaction and involvement among trainees/trainers not noted in trainee/trainer feedback which may be related to factors such as bilingual ability. For example, the least interactive and attentive team lacked one common language. Further analysis and interpretation of behavioral observations were complicated by different observer reporting styles, limited practice prior to the workshop, and unstructured format of the instrument. Conclusions: Simultaneous recording of trainee/trainer behaviors and workshop proceedings was manageable and helpful behavioral information was collected. Usefulness of this approach to improve participant communication during training may be increased by using a more systematic data collection format, e.g. a checklist or closed-ended questions.