Advancing the state of research designs used in injury control policy evaluation

Empirical evaluation of injury control laws has supported dramatic prevention gains in the past decades. Studies have demonstrated beneficial effects in reduced injury, and facilitated diffusion of effective policies. Nevertheless, evaluations continue to use research designs characterized by limitations on internal validity—the strength of confidence in a causal interpretation of observed effects. Drawing on a new book on Public Health Law Research: Theory and Methods, this presentation describes and illustrates the use of experimental and quasi-experimental designs in the evaluation of injury control laws. Few policy evaluators consider random assignment to treatment conditions, since scientists never control passage and implementation of laws and regulations. In fact, there are instances in which new rules may be implemented in a way that allows experimental evaluation. Moreover, experimental methods may be used in the “laboratory” to experimentally test the mechanisms of a law's effects. In natural experiments where random assignment is not possible, there are many design elements that, when combined in a single study, produce study designs that match or sometimes exceed the validity of RCTs. Such elements include: long time-series spanning decades; high time-resolution measures (daily/weekly/monthly); multiple comparison groups, sites, measures; hierarchically nested levels of comparisons; and dose-response studies. Renewed calls for evidenced-based policy making are presenting new opportunities for advancing the use of law to advance injury prevention. Improving the science of policy evaluation by strengthening research designs will help advance public health policy's effectiveness.