Proposal
Introduction to the course
Michael Kogan, PhD
Office of Epidemiology and Research, HRSA, Rockville, MD
APHA 2024 Annual Meeting and Expo
We will introduce the presenters and outline the course. The purpose of this course is to teach public health professionals, particularly at the community, local or state levels how to evaluate public health programs.
Conduct evaluation related to programs, research, and other areas of practice Planning of health education strategies, interventions, and programs Program planning Public health or related research
Proposal
Definitions of evaluation and why evaluate
Michael Kogan, PhD
Office of Epidemiology and Research, HRSA, Rockville, MD
APHA 2024 Annual Meeting and Expo
This presentation will develop the background for the rest of the workshop. We will discuss the importance of conducting program evaluations. We will begin by discussing broad topics such as the increasing complexity of addressing public health problems, the heightened need to address program effectiveness in an era of budgetary constraints, and the need to develop programs using evidence-based measures. We will present questions that applied program evaluation should address, such as: Is the program meeting the intended needs? Who does the program serve? Is the program cost-effective? Has the program achieved its expected outcomes? What are people doing differently as a result of the program? What is the long-term impact of the program? What are the strengths and weaknesses of the program? Were there unintended consequences of the program?
Conduct evaluation related to programs, research, and other areas of practice Program planning Public health or related research
Proposal
CDC program evaluation framework overview
Julie Preskitt, MSOT, MPH, PhD
University of Alabama at Birmingham School of Public Health, Birmingham, AL
APHA 2024 Annual Meeting and Expo
It is important to have a framework when planning and conducting a program evaluation. Major reasons to have a framework is to: 1) build cohesion and collaboration among stakeholders involved with the program development, implementation, and evaluation and 2) provide a 'map' to implement an evaluation. The CDC Evaluation Framework is an evidence-based, practical approach for applied program evaluation in the field. The framework consists of 6 steps and the steps will be reviewed and discussed during this presentation.
Conduct evaluation related to programs, research, and other areas of practice Program planning Public health or related nursing
Proposal
Case studies overview
Julie Preskitt, MSOT, MPH, PhD
University of Alabama at Birmingham School of Public Health, Birmingham, AL
APHA 2024 Annual Meeting and Expo
Three case studies, in different areas of public health, will be presented to highlight major aspects of process and outcome evaluations and to review the steps of the CDC Program Evaluation Framework. The students will have the choice which case study group they'd like to be in throughout the day. Participants will receive copies of the case studies prior to the Learning Institute. This session will review pertinent aspects of the process and outcome evaluations. The case studies should appeal to a wide range of participants in the learning Institute, regardless of their area of focus in public health.
Assessment of individual and community needs for health education Conduct evaluation related to programs, research, and other areas of practice Public health or related research
Proposal
Step 1: Engage stakeholders and needs assessment then practice in groups
Michael Kogan, PhD
Office of Epidemiology and Research, HRSA, Rockville, MD
APHA 2024 Annual Meeting and Expo
This session is one part of a full-day learning session: Evaluating Public Health Programs. This specific part will focus first on how to engage stakeholders, and why they are important to the success of any evaluation, then we will focus on assessing needs as a critical first step in the public health planning cycle. The session will include an overview of the process, as well as detailed description of activities that occur at each of the three phases of needs assessment. The students will then meet in their workgroups for practicing skills learned in the previous session: assessing needs as a critical first step in the public health planning cycle, which includes an overview of the process, as well as detailed description of activities that occur at each of the three phases of needs assessment.
Assessment of individual and community needs for health education Conduct evaluation related to programs, research, and other areas of practice Program planning Public health or related research
Proposal
Program evaluation step 2: Creating goals, objectives, and logic models
Kim Baskette, Ph.D. CHES
Radford University Carilion, Roanoke, VA
APHA 2024 Annual Meeting and Expo
Step 2 of the CDC Evaluation Framework is "Describe the Program". Key elements of this step include 1) Establishing program goals and objectives, and (2) Creating a logic model which " depicts the relationship between your program’s activities and its intended effects" in both process and outcome evaluations. The session will start with an overview of program goals which are broad, general statements describing the overall aim and/or purpose of the public health intervention. Next, program objectives will be addressed, including differentiating between goals and objectives and understanding how to link the program activities and outcomes to the goal(s) and objective(s). The content on objectives will be organized around SMARTIE objectives - meaning each objective is Specific/Strategic, Measurable, Achievable or Attainable, Relevant, Timely or Time-bound, and Inclusive and Equitable. The session will cover major components of a logic model and the rationale on utilizing a logic model when developing a program evaluation. Content will include where goals and objectives are placed in a logic model and how the components of the logic model (resources, activities, outputs, and outcomes/impacts) coincide with the process and outcome evaluations. Participants will then have the necessary content to start to develop a logic model when they go into their small groups.
Conduct evaluation related to programs, research, and other areas of practice Public health or related research
Proposal
Program evaluation steps 3&4: Focusing the evaluation design and gathering credible evidence
Kim Baskette, Ph.D. CHES
Radford University Carilion, Roanoke, VA
APHA 2024 Annual Meeting and Expo
This session is one part of a full-day Learning Institute: Evaluating Public Health Programs. This specific session will focus on Step 3: Focus the Evaluation Design and Step 4: Gather Credible Evidence of the CDC Evaluation Framework. The session will include understanding how to develop questions that can guide the practitioner to establish the scope, purpose, and feasibility of process and outcome evaluations. The session will also delve into how to align process and outcome evaluation data with the questions, data sources, and factors that influence the quantity and quality of the evidence collected as part of the evaluation. The information presented in this session will set the foundation for the sessions that follow.
Assessment of individual and community needs for health education Conduct evaluation related to programs, research, and other areas of practice Program planning Public health or related research
Proposal
Step 4b: Data collection and analysis
Julie Preskitt, MSOT, MPH, PhD
University of Alabama at Birmingham School of Public Health, Birmingham, AL
APHA 2024 Annual Meeting and Expo
Data analysis is sometimes perceived as a daunting activity with vague memories of biostatistics classes in graduate school. What data analysis can be accomplished in applied program evaluation? This session will start with the importance of quality data collection and ethical considerations. Next, the two types of statistics will be briefly reviewed: descriptive and inferential. Community-based resources will be discussed and identified for more advanced analyses, such as inferential statistics. The resources include partnerships with local colleges and universities to assist with advanced analysis. The session will focus on how one can employ principles of descriptive statistics to summarize data (tables and visual displays of data) and basic pre and post intervention comparisons. Web-based resources will be included as a future resource. Students will then meet in their groups to prepare design options, data sources, strengths/limitations, and provide an outline for data analysis based on what they learned in Steps 3 and 4.
Assessment of individual and community needs for health education Conduct evaluation related to programs, research, and other areas of practice Program planning Public health or related nursing
Proposal
Steps 5 and 6: Justify conclusions and dissemination
Michael Kogan, PhD
Office of Epidemiology and Research, HRSA, Rockville, MD
APHA 2024 Annual Meeting and Expo
This session will address the following two steps of the CDC Evaluation Framework: Step 5 - Justify Conclusions and Step 6 - Ensure Use and Share Lessons Learned. During the presentation of Step 5, participants will learn how to move from data analyses results to writing recommendations. The important role of stakeholders will be emphasized again, in addition to including strengths of the program. The steps continue the content of not only the CDC Evaluation Framework but also an approach that supports community-based participatory research. The presentation will also address political aspects that may impact the development of recommendations from a positive (supporting) perspective and a negative perspective (attempts to censure recommendations) and highlight possible strategies for each situation. During the presentation of Step 6, participants will learn how to disseminate the results of the evaluation based on the audience. A dissemination plan will be reviewed as an approach to develop a strategy to share findings. Resources will be reviewed on how to develop a health policy brief. The students will then work in their groups developing a dissemination plan for their proposed public health program. Each individual will receive a dissemination plan handout. Near the end of the session, each group will present key aspects of their dissemination plan to the participants of this LI.
Assessment of individual and community needs for health education Communication and informatics Conduct evaluation related to programs, research, and other areas of practice Program planning Public health or related research