Online Program

Measuring the impact of capacity development on health: Pact's organizational performance index

Monday, November 4, 2013 : 8:30 a.m. - 8:45 a.m.

Jennifer Mulik, MPH, MA, Pact, Washington, DC
Kerry Bruce, DrPH, Pact, Washington, DC
The impact of capacity development work is notoriously difficult to measure. Many capacity developers analyze the results of their work by measuring changes in organizational systems, practices, policies and staff skills. Capacity development practitioners must go further, connecting internal organizational change to improvements in the health of beneficiaries and communities. We developed an Organizational Performance Index (OPI) that is designed to capture an organization's performance based on a scale of Level 1 (low performing) to Level 4 (high performing) under four domains of performance: effectiveness, efficiency, relevance, and sustainability and tested its reliability. Methods: We evaluated the reliability of the OPI with programs in Ethiopia, Nigeria, South Sudan, Swaziland, Vietnam, and Zimbabwe. In total, 40 partners were assessed, with each partner assessed by two staff as well as self-assessed by the organization's own staff. Cronbach's Alpha was used to assess the inter-rater reliability of the scoring. Reliability of .70 or higher was sought in the research. Results: The reliability of the OPI tool was on average .85 for all countries with only Nigeria and Ethiopia reporting scores under .70. Discussion: The OPI proved reliable and can be utilized by capacity developers to independently evaluate capacity development outcomes of their health projects. Further, the OPI measured change in organizational performance and correlated capacity development inputs and community level impact for health projects. It is a revolutionary yet user-friendly approach that looks beyond the development of organizational systems or skills and analyzes the actual value added through capacity development efforts.

Learning Areas:

Program planning
Public health or related research

Learning Objectives:
Explain how to measure capacity development outcomes for health programs. Demonstrate a new capacity development measurement index and show it's application in health programs.

Keyword(s): Outcome Measures, Organizational Change

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am responsible for Pact's Global Health programming and we use this tool for measuring outcomes for all of our projects.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.