142nd APHA Annual Meeting and Exposition

Annual Meeting Recordings are now available for purchase

Portfolio Capabilities Assessment: Using a portfolio management strategy to improve Army community and behavioral health programs' evaluation practice

142nd APHA Annual Meeting and Exposition (November 15 - November 19, 2014): http://www.apha.org/events-and-meetings/annual
Tuesday, November 18, 2014 : 10:45 AM - 11:00 AM

Moira Rivera, PhD , Public Health Assessment Program, U.S. Army Institute of Public Health, Aberdeen Proving Ground - Edgewood Area, MD
Amy Cowell , USAPHC, Aberdeen Proving Grounds-Edgewood Area
William Ford , Army Institue of Public Health, Health Promotion and Wellness, USAPHC, Aberdeen Proving Grounds, Edgewood Area
Background: Unfavorable trends in community and behavioral health indicators persist in the Army despite the existence of many programs designed to address such issues.  Modeled after a combination of program evaluation frameworks, the Portfolio Capabilities Assessment (PCA) was developed to identify programs that produce evidence of a favorable effect on community and behavioral health outcomes and create a culture of accountability and effectiveness among Army community and behavioral health programs. 

Methods: Each year, portfolio programs submit the results of program evaluation studies to the Army Institute of Public Health (AIPH).  AIPH (1) reviews the materials, (2) assigns a score describing how well programs demonstrate effectiveness through program evaluation, (3) provides program staff scores with recommendations for improving evaluation practice, and (4) presents the results to Army Senior Leaders to inform programmatic decision-making.

Results: Of 57 Army programs reviewed, 7 (12%) provided peer-reviewed evidence of effectiveness. Sixteen (28%) showed preliminary evidence of effectiveness, and 34 (60%) showed no clear evidence of effectiveness.  Also, 55 programs (96%) used research to demonstrate need for their services in the Army, and 54 programs (95%) used literature reviews to justify their approach. Each of these results marks an improvement from the 2012 PCA.

Discussion: With continued support and process improvement, the PCA may strengthen Army community and behavioral health programs’ commitment to ensuring effectiveness, serve as a model for promoting evaluation within complex organizations, and improve the Army’s programmatic strategy to safeguard the health of Army Soldiers, civilians, and their families.

Learning Areas:

Conduct evaluation related to programs, research, and other areas of practice
Public health or related laws, regulations, standards, or guidelines
Public health or related research
Systems thinking models (conceptual and theoretical models), applications related to public health

Learning Objectives:
Describe the Army’s Portfolio Capabilities Assessment approach to promoting evaluation among Army community and behavioral health programs. Describe the effect the Portfolio Capabilities Assessment had on evaluation practice in the Army. Identify key lessons learned from using portfolio management to promote Army community and behavioral health program evaluation that can be applied to other complex organizations.

Keyword(s): Evaluation, Health Systems Transformation

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: As a Program Evaluation Consultant, I advocate the use of theory and evaluation to ensure the effectiveness of a portfolio of Army health promotion programs. I made a significant contribution to this abstract and associated work by supporting the development of the evaluation framework, serving as a reviewer for the 2013 and 2014 assessments, and reporting assessment results in briefings to Army leaders and a manuscript submitted to the American Journal of Public Health.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.