Methods for detecting suicidality outcomes in observational comparative effectiveness and safety studies

In observational comparative effectiveness and safety research, detection of an outcome is often flawed because researchers must rely on incomplete data sources such as claims data and electronic health record (EHR) data. Suicidality--an outcome used by the U.S. Food and Drug Administration (FDA) in making regulatory policy, encompassing suicidal thoughts (ideation), preparatory behaviors, suicide attempts, and completed suicide--is one particular outcome that can be under-detected, particularly in claims data. A combination of assessment methods in observational studies could increase the sensitivity and specificity of detection, thereby improving the overall quality of such studies. The Distributed Ambulatory Research in Therapeutics Network (DARTNet) is a federated EHR data network comprised of fifteen organizations representing over 800 clinicians and over 1.5 million patients. DARTNet was formed through a grant from the Agency for Healthcare Research and Quality (AHRQ) as part of its Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network. A currently funded AHRQ task order is using DARTNet to conduct a comparative effectiveness and safety study on major depression. Suicidality is one of several safety outcomes of interest for this study. Currently, the only way to detect suicidality in DARTNet patients is by identifying a diagnosis or medical problem indicating suicide attempt or ideation in their EHR. A second method of detection is being tested, whereby paid medical claims records are being linked to DARTNet patients' EHR, allowing their medical claims to be searched for suicide-related ICD-9 diagnosis codes. A third, novel method will also be tested, using natural language processing (NLP) to search DARTNet patient EHRs for character strings that would indicate suicidal behavior, suicide ideation, attempt and/or completion (for example, “suicid*”). Therefore, suicidality among DARTNet patients will be detected using three different methods: a diagnosis of suicidal ideation or attempt in the EHR; a diagnosis code of suicidal ideation or attempt on a paid medical claim; and a mention of suicidality in the EHR notes (detected using NLP). A cohort of patients with episodes of major depressive disorder will be identified using EHR data available from DARTNet practices. For this presentation, crude rates of suicidality outcomes will be estimated using the three methods of detection described above. Overlap across the three methods will be examined. Finally, using the measurements of suicidality obtained from NLP as the “gold standard”, sensitivity and specificity of the other two methods will be estimated.