220759 A novel approach to workshop evaluation: An enhanced role for observers

Tuesday, November 9, 2010

Lorraine Navarrete, MPH , US-Mexico Border Health Commission, El Paso, TX
JoEllen Welter, MPH , Independent Consultant, Mauren, Switzerland
Melissa Denn, BS , College of Medicine, University of Arizona, Tucson, AZ
Teresa Gomez-Muņoz, BCH, MPH , New Mexico State University, Las Cruces, NM
Jill A. McDonald, PhD , Division of Reproductive Health/NCCDPHP, CDC, El Paso, TX
Background: Evaluations of public health trainings are usually based on trainee reactions post-event and rarely on their own actions or behavior during the training; however, this information may be equally valuable and is often overlooked. In the US-Mexico border region, binational communication is critical to effective public health practice; we explored whether observers recording proceedings of a three-day bilingual training to build capacity among US-Mexico teams of health professionals could also document participant behavior that would be useful in improving future events. Methods: Included on the form capturing workshop proceedings were seventeen open-ended questions which prompted observers to document in a semi-structured manner trainee/trainer attentiveness, interaction, and level of involvement during each workshop session. Three bilingual observers recorded trainee/trainer behaviors in real time. This information was later added to trainee/trainer post-workshop feedback to obtain a more comprehensive evaluation from multiple perspectives. Results: Observers identified variations in attentiveness, interaction and involvement among trainees/trainers not noted in trainee/trainer feedback which may be related to factors such as bilingual ability. For example, the least interactive and attentive team lacked one common language. Further analysis and interpretation of behavioral observations were complicated by different observer reporting styles, limited practice prior to the workshop, and unstructured format of the instrument. Conclusions: Simultaneous recording of trainee/trainer behaviors and workshop proceedings was manageable and helpful behavioral information was collected. Usefulness of this approach to improve participant communication during training may be increased by using a more systematic data collection format, e.g. a checklist or closed-ended questions.

Learning Areas:
Communication and informatics
Conduct evaluation related to programs, research, and other areas of practice
Diversity and culture
Planning of health education strategies, interventions, and programs
Public health or related education

Learning Objectives:
1.Describe two roles for an informed observer at a workshop/training. 2.Describe at least two constructive uses of documenting participant behavior during a workshop/training.

Keywords: Evaluation, Cultural Competency

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I acted as an observer for the training in which the abstract refers and implmented the innovative methods discussed.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.