Online Program

Quality Improvement in Public Health: “Real World” evaluation to meet local program needs

Wednesday, November 4, 2015 : 11:30 a.m. - 11:45 a.m.

Maria Gilson deValpine, RN MSN PhD, Department of Nursing, James Madison University, Harrisonburg, VA
Background/issue: Public health quality improvement suffers from evaluations that are often poorly conceived, have sub-optimal study designs, and lack good quality data for program outcomes.

Description: A program evaluation was designed for a federally grant funded MCH home visiting program targeted at families affected by substance abuse. The program to be evaluated was already in progress and had been designed without adequate attention to effective study design, data for outcomes evaluation, or analyzable data for program implementation. Federal technical assistance was inflexible and demanding beyond the means of local program capacity. The new program evaluation plan made ready use of Bamberger’s “Real World” evaluation approach incorporating constraints associated with budget, stakeholder expectations, data limitations, and political considerations.

Lessons learned: Beyond the obvious lesson—that program evaluations must be planned prior to implementation, program managers and evaluators learned to use Bamberger’s constraints to evaluate the program for local program improvement. Failing to consistently satisfy the federal funder, the local program was able to expand capacity and illuminate which programs were most likely to be effective in this small rural setting. 

Implications/recommendations: Federally funded public health programs often come with the expectation that public health departments have evaluation capacities far beyond what is realistic for small agencies in rural settings with limited resources. Contracting with “expert” program evaluators is often not possible given local resources, and may consume much of program budgets. To be successful, health departments may need to lay realistic expectations of program evaluability and retain a focus on local needs.

Learning Areas:

Program planning

Learning Objectives:
Describe "Real World" evaluation. Discuss application of program evaluation planning to local public health programs.

Keyword(s): Quality Improvement, Maternal and Child Health

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am the program evaluator for the subject program.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.