305579
Portfolio Capabilities Assessment: Using a portfolio management strategy to improve Army community and behavioral health programs' evaluation practice
Methods: Each year, portfolio programs submit the results of program evaluation studies to the Army Institute of Public Health (AIPH). AIPH (1) reviews the materials, (2) assigns a score describing how well programs demonstrate effectiveness through program evaluation, (3) provides program staff scores with recommendations for improving evaluation practice, and (4) presents the results to Army Senior Leaders to inform programmatic decision-making.
Results: Of 57 Army programs reviewed, 7 (12%) provided peer-reviewed evidence of effectiveness. Sixteen (28%) showed preliminary evidence of effectiveness, and 34 (60%) showed no clear evidence of effectiveness. Also, 55 programs (96%) used research to demonstrate need for their services in the Army, and 54 programs (95%) used literature reviews to justify their approach. Each of these results marks an improvement from the 2012 PCA.
Discussion: With continued support and process improvement, the PCA may strengthen Army community and behavioral health programs’ commitment to ensuring effectiveness, serve as a model for promoting evaluation within complex organizations, and improve the Army’s programmatic strategy to safeguard the health of Army Soldiers, civilians, and their families.
Learning Areas:
Conduct evaluation related to programs, research, and other areas of practicePublic health or related laws, regulations, standards, or guidelines
Public health or related research
Systems thinking models (conceptual and theoretical models), applications related to public health
Learning Objectives:
Describe the Army’s Portfolio Capabilities Assessment approach to promoting evaluation among Army community and behavioral health programs.
Describe the effect the Portfolio Capabilities Assessment had on evaluation practice in the Army.
Identify key lessons learned from using portfolio management to promote Army community and behavioral health program evaluation that can be applied to other complex organizations.
Keyword(s): Evaluation, Health Systems Transformation
Qualified on the content I am responsible for because: As a Program Evaluation Consultant, I advocate the use of theory and evaluation to ensure the effectiveness of a portfolio of Army health promotion programs. I made a significant contribution to this abstract and associated work by supporting the development of the evaluation framework, serving as a reviewer for the 2013 and 2014 assessments, and reporting assessment results in briefings to Army leaders and a manuscript submitted to the American Journal of Public Health.
Any relevant financial relationships? No
I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.