In this Section |
![]() 258226 Strengthening causal inference using Optimal Full Matching: An Empirical Demonstration in a Program EvaluationWednesday, October 31, 2012
: 12:50 PM - 1:10 PM
Program evaluations are often limited by confounding due to differences in treatment and control groups. Optimal full matching, a recent innovation in propensity score matching, can minimize these differences and strengthen causal inferences. However, optimal full matching makes computing the treatment effect and statistical significance challenging because matched sets, not pairs, are created. Parametric methods require a high degree of homogeneity of outcomes within matched sets. Using data from the New York/New York III supportive housing program evaluation we assessed the usefulness of optimal full matching, stratification, and one-to-one greedy matching in minimizing differences between people eligible for the program who were placed versus not placed in housing. For example, among 201 young adults who aged out of foster care, were eligible for the program (2007-2008), and had one year of follow-up time, 68 were placed and 133 were not placed. On average there was 65% of a standard deviation difference between baseline characteristics of treatment and control groups before matching. Optimal full matching produced the largest reduction, with groups having an average of 4% of a standard deviation difference post-matching, compared to stratification (13%) and one-to-one greedy matching (36%). In our dataset the cost data for the main outcomes were heavily skewed and weakly correlated within matched sets (the intraclass correlation coefficient for total cost of services and benefit post-treatment was 0.004). Nonparametric permutation-based methods (Hodges-Lehmann aligned rank test and bootstrapping) addressed this challenge. Optimal full matching can strengthen causal inferences when randomization is not feasible.
Learning Areas:
Biostatistics, economicsConduct evaluation related to programs, research, and other areas of practice Epidemiology Learning Objectives: Keywords: Statistics, Evaluation
Presenting author's disclosure statement:
Qualified on the content I am responsible for because: I have been responsible for developing statistical methods for NY/NY III program evaluation. I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.
Back to: 5214.0: Statistical Applications: General Methods
|