Against statistical inference: A commentary on the role of statistics in public health research

The distinction is made between statistical inference and scientific inference. Statistical inference is defined here (admittedly narrowly) as the acceptance or rejection of a hypothesis based on a statistical test designed to have known error probabilities under specified assumptions. It is instilled in students with the mantra: “p<.05, therefore we can reject the null hypothesis”. In contrast, scientific inference is defined as making a judgment about a hypothesis based on the evidence in a particular data set along with evidence from previous studies and considerations of biologic plausibility. The key role of a statistical analysis in scientific interference is to quantify the evidence in the data so that this evidence can be properly weighed with other information in making scientific judgements. Few statisticians or scientists would argue against giving priority to scientific inference over statistical inference in the conduct of science. Despite this fact, the language of statistical inference (as narrowly defined) pervades the scientific literature. While this language is often harmless, it is argued here that it often has a detrimental impact on the reasonableness of scientific inference and the conduct of research. This commentary will describe the historical background of the distinction between statistical and scientific inference, and provide examples of how the narrow notion of statistical inference can adversely affect public health research and policy.