223868 Methods for detecting suicidality outcomes in observational comparative effectiveness and safety studies

Tuesday, November 9, 2010 : 3:15 PM - 3:30 PM

Heather Orton Anderson, PhD , Colorado School of Public Health, University of Colorado Denver, Aurora, CO
Anne Libby, PhD , School of Pharmacy, University of Colorado Denver, Aurora, CO
Robert Valuck, PhD , School of Pharmacy, University of Colorado-Denver, Aurora, CO
In observational comparative effectiveness and safety research, detection of an outcome is often flawed because researchers must rely on incomplete data sources such as claims data and electronic health record (EHR) data. Suicidality--an outcome used by the U.S. Food and Drug Administration (FDA) in making regulatory policy, encompassing suicidal thoughts (ideation), preparatory behaviors, suicide attempts, and completed suicide--is one particular outcome that can be under-detected, particularly in claims data. A combination of assessment methods in observational studies could increase the sensitivity and specificity of detection, thereby improving the overall quality of such studies. The Distributed Ambulatory Research in Therapeutics Network (DARTNet) is a federated EHR data network comprised of fifteen organizations representing over 800 clinicians and over 1.5 million patients. DARTNet was formed through a grant from the Agency for Healthcare Research and Quality (AHRQ) as part of its Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network. A currently funded AHRQ task order is using DARTNet to conduct a comparative effectiveness and safety study on major depression. Suicidality is one of several safety outcomes of interest for this study. Currently, the only way to detect suicidality in DARTNet patients is by identifying a diagnosis or medical problem indicating suicide attempt or ideation in their EHR. A second method of detection is being tested, whereby paid medical claims records are being linked to DARTNet patients' EHR, allowing their medical claims to be searched for suicide-related ICD-9 diagnosis codes. A third, novel method will also be tested, using natural language processing (NLP) to search DARTNet patient EHRs for character strings that would indicate suicidal behavior, suicide ideation, attempt and/or completion (for example, “suicid*”). Therefore, suicidality among DARTNet patients will be detected using three different methods: a diagnosis of suicidal ideation or attempt in the EHR; a diagnosis code of suicidal ideation or attempt on a paid medical claim; and a mention of suicidality in the EHR notes (detected using NLP). A cohort of patients with episodes of major depressive disorder will be identified using EHR data available from DARTNet practices. For this presentation, crude rates of suicidality outcomes will be estimated using the three methods of detection described above. Overlap across the three methods will be examined. Finally, using the measurements of suicidality obtained from NLP as the “gold standard”, sensitivity and specificity of the other two methods will be estimated.

Learning Areas:
Provision of health care to the public
Public health or related public policy
Public health or related research

Learning Objectives:
1. Describe the use of Natural Language Processing (NLP) to detect measurable suicidality outcomes from electronic health record (EHR) data. 2. Compare the detection of suicidality outcomes across three detection methods: (1) NLP of EHR data, (2) diagnosis codes in claims data, (3) diagnosis indicators in EHR data. 3. Discuss the strengths of using a combination of detection methods and the bias introduced by using an inadaquate method.

Keywords: Suicide, Data Collection

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am qualified to present because I am a collaborating researcher with a team specializing in comparative effectiveness and safety research, and am the project lead for the submitted study.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.