The 131st Annual Meeting (November 15-19, 2003) of APHA

Abstract #69957

Patrick S. Romano, MD, MPH¹, Jeffrey Geppert, JD², Sheryl M Davies, MA³, Kathryn M. McDonald, MM³, Anne Elixhauser, PhD⁴, and Marlene R. Miller, MD, MSc⁵. (1) Center for Health Services Research in Primary Care, University of California Davis, 4150 V Street; PSSB Suite 2400, Sacramento, CA 95817, 916-734-7004, psromano@ucdavis.edu, (2) National Bureau of Economic Research, 30 Alta Road, Stanford, CA 94305-6019, (3) Center for Primary Care and Outcomes Research, Stanford University, 117 Encina Commons, Stanford, CA 94305, (4) Center for Organization and Delivery Studies, Agency for Healthcare Research and Quality, 2101 East Jefferson St., Suite 605, Rockville, MD 20852, (5) Johns Hopkins Childrens Center, 600 North Wolfe Street, CMSC 2-125A, Baltimore, MD 21287

BACKGROUND: Recent reports have focused national attention on preventable medical errors. However, researchers and policy-makers currently have few tools with which to study the epidemiology of patient safety. The UC-Stanford Evidence-based Practice Center worked with AHRQ to refine and validate a pilot set of Patient Safety Indicators (PSIs) based on routinely collected hospital administrative data. METHODS: The project consisted of five phases. First, we systematically reviewed the research literature to identify candidate indicators of iatrogenic complications and to collect information about their coding and construct validity. Second, we evaluated the face validity of candidate indicators by convening 11 panels of 5-9 expert clinicians nominated by 28 professional organizations, and applying an adaptation of the RAND/UCLA Appropriateness Method. Third, we consulted coding experts to confirm the ICD-9-CM definition of each indicator. Fourth, we empirically evaluated promising indicators using the Healthcare Cost and Utilization Project (HCUP) 1995-97 State Inpatient Databases, adjusting for age, gender, comorbidities, DRG clusters, and selected interactions. Multivariate signal extraction and factor analytic methods were used to understand hospital-level variation. Fifth, we wrote and tested SAS code that researchers can use to estimate PSI rates based on their own data. RESULTS: We identified 34 candidate indicators, after excluding those previously shown to have questionable coding or construct validity. Based on second-round panel ratings, 22 indicators had acceptable face validity. We dropped 2 indicators due to operational concerns and empirically evaluated the remaining 20 (plus 17 lower-rated "experimental" indicators). Of these 20 indicators, 2 (transfusion reaction, foreign body left during procedure) had minimal variation in hospital-level rates. All but 5 of the 18 remaining indicators had signal-to-noise ratios exceeding 0.5 in 1997, meaning that hospital-level signal is not overwhelmed by random noise. Year-to-year hospital-level correlations exceeded 0.6 for 4 indicators (decubitus ulcer, accidental puncture or laceration, infection due to medical care, and obstetric trauma/vaginal delivery). Factor analysis suggested 2 underlying constructs: obstetric, catheter-related, and technical complications versus postoperative complications. CONCLUSION: To evaluate patient safety interventions and identify best practices, researchers need reliable and valid outcome measures. Through literature review, expert panel ratings, and empircal analyses, we identified and refined 20 indicators that hospitals, state data organizations and others can use to screen for potential patient safety problems and enhance surveillance and quality improvement activities.

Learning Objectives:

At the conclusion of this session, each participant will be able to:
• Describe the AHRQ Quality Indicators, including the three modules available for public use (Inpatient Quality Indicators, Prevention Quality Indicators, Patient Safety Indicators).
• Describe the process through which the Patient Safety Indicators were developed and tested.
• Discuss potential applications of the Patient Safety Indicators in provider organizations, public health agencies, and health data organizations.

Keywords: Quality of Care, Outcome Measures

Related Web page: www.qualityindicators.ahrq.gov

Presenting author's disclosure statement:
Organization/institution whose products or services will be discussed: US Agency for Healthcare Research and Quality UC-Stanford Evidence-based Practice Center
I have a significant financial interest/arrangement or affiliation with any organization/institution whose products or services are being discussed in this session.
Relationship: Recipient of research grants and contracts from AHRQ.

Handout (.ppt format, 282.0 kb)