Online Program

Systematic review and rapid evidence assessment of the literature methodology: The field, logistics, and challenges

Monday, November 4, 2013 : 5:10 p.m. - 5:20 p.m.

Raheleh Khorsan, MA, Military Medical Research and Integrative Medicine, Samueli Institute, Corona del Mar, CA
Cindy Crawford, BA, Military Medical Research, Samueli Institute, Alexandria, VA
Objective: At present, evidence rankings do not consider equally internal (IV), external (EV), and model validity (MV) for CAM studies. Therefore, we have developed an EV/MV-REAL-PLUS-tool for evaluating EV and MV criteria. The purpose of this presentation is to describe this model. Methods: A systematic review methodology was used to evaluate the literature on EV/MV. Pubmed search for keywords external validity, model validity, and bias-scoring from inception to Jan 2013. The research team developed the EV/MV-REAL-PLUS-tool scale based on other published EV/MV scales i.e the GAP-Scale. Results: 1131 abstracts were reviewed. We created the EV/MV-REAL-PLUS-tool scale to measure: 1) how closely the study population; 2) the institution types in the study; 3) the types of physicians in the study; 4) the role of clinician decision-making in the study; and 5) the role of patient preferences in the study resemble those in actual practice. Conclusion: Improved reporting on EV can help produce and provide information that will help guide policy makers, public health researchers, and other scientists, especially in the field of Integrative Health Care, in their selection, development, and improvement in their research-tested intervention. Overall, RCTs and pragmatic-studies with high EV have the potential to provide the most useful information about ‘real-world' consequences of health interventions. It is hoped that this novel tool which considers internal and external validity on equal footing, will better guide clinical decision making.

Learning Areas:

Conduct evaluation related to programs, research, and other areas of practice
Systems thinking models (conceptual and theoretical models), applications related to public health

Learning Objectives:
Define the strengths & limits of systematic reviews and REALs. Identify the differences between Internal Validity, External Validity and Model Validity. Describe the role, benefits, challenges and appropriate use of systematic review methods in the basic and applied social and clinical sciences.

Keyword(s): Alternative Medicine/Therapies, Research

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am a Research Associate of the Systematic Review Program at Samueli Institute. I co-developed the systematic review, meta-analysis and rapid evidence assessment of the literature (REAL) process that is implemented and taught at Samueli Institute.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.