261513
Use of technology to improve inter-rater reliability in medical record review: The added-value of mobile devices
Monday, October 29, 2012
: 2:30 PM - 2:50 PM
Leslie A. Watts, MS
,
Bixby Center for Global Reproductive Health, University of California, San Francisco, Sacramento, CA
Mary Menz, PHN, BSN
,
Bixby Center for Global Reproductive Health, University of California, San Francisco, Sacramento, CA
Shantha Rao
,
Bixby Center for Global Reproductive Health, University of California, San Francisco, Sacramento, CA
Michael Howell, MA
,
Bixby Center for Global Reproductive Health, University of California, San Francisco, Sacramento
Philip Darney, MD, MSc
,
Bixby Center for Global Reproductive Health, University of California, San Francisco, San Francisco, CA
Heike Thiel de Bocanegra, PhD, MPH
,
Assistant Professor and Director, UCSF Family PACT Evaluation, Bixby Center for Global Reproductive Health, University of California, San Francisco, Sacramento, CA
Medical record review is a widely used methodology for clinical research, for health care program evaluation and for collecting data to assess quality of care. Measuring inter-rater reliability among independent chart abstractors (raters) is an essential component in assuring data collection has high correlation of agreement among abstractors across responses. In 2011/12, nine nurse abstractors from the University of California, San Francisco's Bixby Center for Global Reproductive Health collected data from nearly 5,000 medical records in 216 provider sites in 18 counties, for the California Department of Public Health Office of Family Planning's Family PACT (Planning, Access, Care, and Treatment) Program. The abstraction tool was developed in Microsoft Excel with software written for a wireless tablet. The tool was designed to minimize response variation by using yes/no and multiple choice responses as well as date and number dials. We identified a subset of key responses among the more than 150 data points to compare with responses of each of the abstractors. Ten charts at three different provider sites were re-abstracted for each of the abstractors by the quality assurance nurse, considered to be the true score. In this presentation, we will discuss findings suggesting that the combination of an intensive abstractor training with baseline homework, the ease of use of the wireless tablet, and an abstraction tool that supports standardized responses serves to minimize the effect of abstractor variability on data quality results. These strategies are cost efficient, maximize available resources, and are reproducible on a large scale.
Learning Areas:
Conduct evaluation related to programs, research, and other areas of practice
Learning Objectives: 1. Discuss how data collection using a mobile device can improve correlation of agreement among abstractors across responses.
2. Explain how a defined set of comparison data points can be efficient in assessing inter-rater reliability of data collected on a mobile device.
3. Describe how a chart abstraction tool with standardized responses tends to minimize variability of data collected by multiple abstractors.
Keywords: Data Collection, Quality Assurance
Presenting author's disclosure statement:Qualified on the content I am responsible for because: Leslie Watts is an Analyst with the UCSF Bixby Center for Global Reproductive Health and with the Family PACT contract for more than fourteen years. She is currently the project coordinator for the statewide Family PACT medical record review. Her activities include participation in quality improvement activities such as Clinical Practice Alerts, Provider Profiles, and support to the Clinical Practice Committee. Leslie has a Master of Science degree from U.C. Davis.
Any relevant financial relationships? No
I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines,
and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed
in my presentation.
|