260707 Assessing the psychometric properties of a Positive Youth Development/Teen Pregnancy Prevention program evaluation instrument

Sunday, October 28, 2012

Heather D. Blunt, PhD, MPH, CPH, CHES , Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Charlotte Noble , College of Arts and Sciences, Department of Anthropology, University of South Florida, Tampa, FL
Natalie Klinkenberger, BS , Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Sarah Maness, MPH , Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Kara McGinnis, BA , Dept. of Anthropology and College of Public Health, University of South Florida, Tampa, FL
Rita DeBate, PhD, MPH, CHES , Center for Transdisciplinary Research on Women's Health, Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Karen Perrin, PhD , Community and Family Health, University of South Florida, College of Public Health, Tampa, FL
Ellen Daley, PhD , Center for Transdisciplinary Research on Women's Health, Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Stephanie L. Marhefka, PhD , Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Wei Wang, PhD , Department of Epidemiology and Biostatistics, College of Public Health, University of South Florida, Tampa, FL
Eric R. Buhi, MPH, PhD , Department of Community and Family Health, College of Public Health, University of South Florida, Tampa, FL
Background: A comprehensive evaluation of the Teen Outreach Program (TOP) is being conducted in non-metropolitan counties in Florida.

Purpose: The purpose of this cross-sectional study was to pilot test a survey instrument developed to evaluate a federally-funded teen pregnancy prevention project. This analysis reports the psychometric characteristics of 4 scales used in this study.

Significance: The collection of reliable and valid data is vital to the integrity of research and evaluation projects.

Methodology: A pilot survey was administered during five class periods in June 2011 in a rural Florida high school. The survey included measures of Positive Youth Development (PYD), with subscales assessing character, caring, connection and confidence; self-regulation (SR), with subscales assessing emotion, cognitions, and behaviors; expectations/aspirations; and community service (CS). Analyses include Cronbach's alpha and confirmatory factor analysis with Varimax rotation.

Findings/Results: Fifty-eight youth are included in these analyses. Reliability and validity statistics were calculated for the scaled items. Results show the scaled data are sufficiently reliable (Cronbach's alpha: Character=.877, Caring=.774, Connection=.908, Confidence=.819, CS=.970, Expectations/Aspirations=.860). Only SR (Cronbach's alpha=.641) resulted in debatable score reliability. Factor analysis found theoretically sound results that reflect previous research findings. Validity statistics are acceptable with variance explained ranging from 54% (emotions subscale of SR) to 79% (CS).

Conclusions/Recommendations: Detecting meaningful effects depends on reliable and valid data/scores. Despite a small sample size, the data in this pilot study appear to be reliable and valid. Assessing psychometric characteristics of survey data is an important step to explaining the influence of measurement error on results.

Learning Areas:
Conduct evaluation related to programs, research, and other areas of practice
Public health or related research

Learning Objectives:
1. Describe the importance of valid and reliable data from evaluation measurement instruments for conducting research and evaluation 2. Evaluate the psychometric characteristics of Positive Youth Development/Teen Pregnancy Prevention program evaluation instrument.

Keywords: Survey, Adolescent Health

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am senio senior doctoral student on the project which generated this data. I have been involved with this project from the grant development stage and have contributed to the development of all evaluation materials. I am Student Involvement Co-Chair at The Society for Scientific Study of Sexuality Graduate Research Associate at USF, served as Editorial Associate, Journal of Behavioral Health Services & Research. I am published on this topic in peer reviewed professional journals.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

Back to: 2054.0: Adolescents and School Health