Online Program

Integration of public health and health services evaluation measures: The case for standardizing reach and dose

Wednesday, November 4, 2015

Debra Holden, PhD, Community Health Promotion Research Program, RTI International, Research Triangle Park, NC
Jeanette Renaud, PhD, Community Health Promotion Research, RTI International, Durham, NC
Lucia Rojas Smith, DrPH, Social, Health, and Organizational Research and Evaluation, RTI International, Washington, DC
Peter Amico, PhD, RTI International, Waltham, MA
Sara Jacobs, PhD, Social & Health Organizational Research and Evaluation, RTI International, Atlanta, GA
Sonya Goode, MPH, BA, Community Health Promotion Research Program, RTI International, RTP, NC
Background: The concepts of the ‘reach’ (i.e., the proportion of a population exposed to an intervention) and ‘dose’ (i.e., the intensity of a participant’s exposure to the intervention) of an intervention are important evaluation criteria in the field of public health but are less prominent in health services research. The target population of an intervention can be at the organization, setting, or population level but is often not specified as well as needed for the evaluation design. There is no standard or widely accepted definition of treatment intensity in the health services field that helps to quantify the degree to which increasing exposure impacts specific outcomes. Instead, exposure to the intervention is typically dichotomous (i.e., treatment group or not). This treatment exposure measurement can lead to ambiguous conclusions about the effectiveness of an intervention. This challenge was encountered in designing the evaluation of 24 Health Care Innovation Awardees of the Centers for Medicare and Medicaid Innovation and solutions were implemented that will be described.

Methods: Qualitative data from awardees are compared to aggregated quantitative data provided by awardees via their electronic medical records or administrative data.

Results: Few of the 24 awardees provided data on the potential reach of their interventions and had difficulty defining their target populations. Even fewer awardees were collecting measures of dose that could be used to assess exposure to the interventions.

Conclusions: We present an assessment of the reach and dose measures used across the 24 awardees, the challenges of conceptualizing these measures and offer recommendations for standardizing these measures.

Learning Areas:

Conduct evaluation related to programs, research, and other areas of practice
Program planning
Provision of health care to the public

Learning Objectives:
Define how reach and dose apply to health services interventions Discuss the challenges in establishing a standardized measure for reach or dose in health care settings Describe how reach and dose measures were adapted for program evaluation

Keyword(s): Evaluation, Outcomes Research

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I have been a program evaluator for nearly 25 years of national public health programs addressing chronic disease prevention and health promotion. I am lead author of a book published by Sage on program evaluation planning and have specific expertise in aligning evaluation measures with key domains of interest and implementing mixed methods evaluations that capture outcomes at multiple levels.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.