The 131st Annual Meeting (November 15-19, 2003) of APHA

Abstract #69118

Aimee Reedy, EdD, MPH, Rocio Gonzalez-Luna, MPH, Guadalupe S Olivas, PhD, and Anandi Sujeer, MPH. Santa Clara County Public Health Department, 3003 Moorpark Avenue, San Jose, CA 95128, (408) 423-0724, Aimee.Reedy@hhs.co.santa-clara.ca.us

The Santa Clara County Public Health Department is in its third year of a department-wide evaluation initiative. In year one, an evaluation framework was created to guide the development and implementation of performance measurement systems across over 50 health promotion, prevention and direct service programs using the 10 Essential Public Health Services. The framework has been operationalized and implemented during the past two years.

Implementation of the framework occurred in three phases. During Phase I, department personnel were offered workshops on the basics of evaluation, the details of the evaluation framework, and management of complex evaluations. During Phase II, technical assistance was offered to each program manager and staff with the goal of developing evaluation plans, including logic models, goals, objectives, problem statements, program theory and components. Consistent with the framework, each objective was linked to standard indicators across four performance domains: quality, effectiveness, efficiency and accessibility. Phase III focused on the design and operationalization of protocols for data collection at the program, essential service, and department levels.

Strategies have been developed to address the following challenges: (1) staff anxiety resulting from negative perceptions or experiences with evaluation, (2) appropriate level of staff and program involvement in the design of evaluation plans, (3) reconciling evaluation plans with statewide program goals (4) insufficient data expertise, (5) emphasis on process versus outcome objectives, (6) lack of sufficient electronic data systems, and (6) the importance of continuous feedback loops across all levels of the organization.

Learning Objectives:

At the conclusion of the session, the participant will be able to:
• Describe strategies used in implementing an evaluation framework to strengthen local public health department accountability and performance.
• Identify strategies to address barriers to department-wide evaluation.
• Identify strategies for managing staff anxiety and systems change associated with a large-scale evaluation initiative.

Presenting author's disclosure statement:
I do not have any significant financial interest/arrangement or affiliation with any organization/institution whose products or services are being discussed in this session.