Arguments for and against the use of systematic reviews in healthcare decision making: Cui bono?

Background: Systematic reviews of pharmaceuticals are used to inform practice guidelines and public- and private-sector drug coverage decisions. Arguments for and against the use of systematic reviews in health policy may be related to authors' affiliations, including income sources and ideological backgrounds. For example, when meta-analyses concluded that exposure to second-hand smoke is harmful, the tobacco industry attacked the basic methodology, and funded research to refute the findings. Objectives: In order to inform public- and private- sector policymakers about the forms of bias that may fuel the discourse surrounding the policy uses of systematic reviews, we describe the basic arguments. We identify possible financial, ideological, organizational and other affiliation biases of the arguments and rebuttals. Methods: We perform a critical review of articles that evaluate the scope, methods, or process of systematic reviews of pharmaceuticals. We determine what arguments are being made, and the affiliations of those making the arguments. We also explore the genesis of the arguments by examining possible affiliation biases in seminal papers cited by authors making the arguments. Results: Arguments against the use of systematic reviews of pharmaceuticals for health policy decisions are often but not always made by industry-affiliated critics. Arguments, which sometimes resemble tactics used by Big Tobacco to instill doubts about findings unfavorable to the tobacco industry, criticize methods and question interpretation of results. Other arguments focus on inappropriate cost containment, failure of the evidence to account for differential response to treatment by patients who differ from the average, and raise questions about the use of systematic reviews to develop of drug formularies for programs like Medicaid and others that serve diverse or “non-standard” populations. Arguments supporting policy uses of systematic reviews are most often made by proponents of evidence-based medicine, academic researchers, and payers. Supporters affirm that systematic reviews provide the best available evidence, and cite reviewers' lack of financial ties to the pharmaceutical industry. They cite the reviews' standardized study designs and wide acceptance, as well as their usefulness in developing policy that ensures access to effective medicines while limiting wasteful spending on ineffective and expensive treatments. Discussion: Values play a role in health policy and that evidence informs it. Biases, especially financial conflicts of interest, need to be transparent and taken into account when policy makers evaluate arguments for and against using systematic reviews in health policy and drug coverage decisions.