Back to Annual Meeting Page		American Public Health Association 133rd Annual Meeting & Exposition December 10-14, 2005 Philadelphia, PA

Abstract #108205

Tod Mijanovich, MPA¹, Winston Lin, MS¹, and Beth Weitzman, PhD². (1) Center for Health and Public Service Research, New York University, Wagner Graduate School of Public Service, The Puck Building, 295 Lafayette St., 2nd Floor, New York, NY 10012, 212-998-7467, tm11@nyu.edu, (2) Wagner Graduate School of Public Service, New York University, Center for Health and Public Service Research, 726 Broadway, 5th Floor, New York, NY 10003

Community-level interventions known as Comprehensive Community Initiatives (CCIs) pose major challenges for evaluators, among them that CCIs are often thought to affect multiple outcomes. Researchers have taken various approaches to the multiple outcome problem, such as developing indices or extracting "factors" that combine multiple outcomes into a single measure, or analyzing the impact on each outcome while statistically adjusting for multiple comparisons. However, single indices or factors may miss important dimensions of change that are unique to individual measures, and statistical adjustments (such as the Bonferroni adjustment) may be too conservative in the presence of correlated outcomes. We explore several approaches to the problem of measuring change in multiple outcomes using data collected for the evaluation of the Urban Health Initiative (UHI), a ten-year effort to improve the well-being of youth in five economically distressed U.S. cities. We will present multiple methods of analyzing the Survey of Adults and Youth (SAY), a telephone survey fielded in 1998-9 and 2004-5 in the five UHI program cities and in nine comparison cities. Pre- and post-program survey data collected in both program and comparison cities allow for a difference-in-difference design, comparing program city changes over time to comparison city changes. Analyses will include: analysis of aggregate-level indicators using cities as analytical units; respondent-level analysis of individual indicators using robust standard errors; combining indicators via MANOVA and factor analysis; converting indicators to effect sizes and comparing changes in average effect sizes; and multi-level analysis of individual indicators.

Learning Objectives: At the conclusion of this session, participants will be able to

• Describe and distinguish between the various types of multiple indicator problems that complicate the evaluation of large-scale program initiatives.
• Describe the range of possible ways to analyze multiple program outcomes.
• Evaluate the appropriateness and robustness of individual methods of analyzing multiple indicators.

Keywords: Outcome Measures, Community Research

Presenting author's disclosure statement:

I wish to disclose that I have NO financial interests or other relationship with the manufactures of commercial products, suppliers of commercial services or commercial supporters.