Session
Applied Public Health Statistics Roundtable
APHA 2022 Annual Meeting and Expo
Abstract
Quality of causes of death data in Minnesota from 2011 to 2022 – A descriptive analysis of the usage and correlates of garbage codes
APHA 2022 Annual Meeting and Expo
An important function of death statistics is to estimate the burden of various diseases. Accurate cause-of-death information on death certificates is thus critical to inform public health policies. In this paper, we examine the quality of cause-of-death data on death certificates in Minnesota from 2011 onwards and identify the correlates of death certificates assigned unsuitable causes of death (garbage codes). Our key data source is death certificates from the Minnesota Department of Health
We classified all ICD-10 codes in our data into five types of garbage codes and four levels of garbage codes using classification criteria developed by Analysis of National Causes of Death for Action (ANACONDA) software and analyzed the levels, and patterns associated with the usage of grange codes in Minnesota.
We found over 500 different garbage codes used in our data with around 23% of the total death certificates between 2011 and 2021 ving garbage codes applied to them. The most frequently used codes included R99 (ill-defined and unspecified causes of death), I64 (unspecified stroke), I500 (heart failure), C80 (malignant neoplasm) and J189 (unspecified pneumonia). We also found some counties to have consistently high levels of usage of garbage codes. Further analysis will examine socio-economic correlates and temporal patterns to understand who gets assigned garbage codes and whether the usage of garbage codes increased during the COVID-19 pandemic.
There is opportunity to improve the causes of death data in Minnesota and consequently improve our understanding of the burden of different diseases, particularly in underrepresented communities. The main policy implications of our findings are that it is important to spread awareness of the importance of accurate mortality data and provide training to doctors and other professionals to certify deaths.
Abstract
A novel method for public health statistical analysis: Application of max-p regionalization in analysis of cancer surgery across a midwest state
APHA 2022 Annual Meeting and Expo
Title: A novel method for public health statistical analysis: Application of max-p regionalization in analysis of cancer surgery across a midwest state
Background/purpose: Spatial analysis of health data typically requires either suppressing areas that have small numbers or resorting to larger geographies to generalize data. While other methods of spatial analysis exist to visualize small numbers, they do not allow for statistical comparison of data aggregated to small geographies (census tracts) and generalizing to larger geographies can mask health disparities by smoothing out spaces. The purpose of this study was to use regionalization methods to combine geographies with small numbers and avoid geographic suppression; to use specific and granular geographies to generalize space; and to evaluate regional variation in surgical management of gastrointestinal (GI) cancers.
Methods: Patients with potentially resectable GI cancers from 2009–2018 were identified from the Ohio Cancer Incidence Surveillance System (OCISS), coded according to receipt of surgery, and aggregated to Ohio census tracts. The max-p regionalization algorithm was used to create homogenous clusters that met the small-number threshold for the number of surgeries. Spatial homogeneity was informed by the area deprivation index (ADI).
Results: 2,948 Ohio census tracts were homogenized into 576 MaxTracts. Surgery rates differed between urban and rural areas: urban centers and rural areas had low rates, while suburban peripheries had high rates.
Conclusion: Max-p regionalization is a viable option for making geographic associations with census data while avoiding small number issues. Although max-p generalizes space, it is a data informed process that reduces geographic smoothing and the reliance on universal larger geographies.
Abstract
“Quarantine exempt”: Impacts of pooled testing in Maine’s K-12 test-to-stay approach to COVID-19 prevention
APHA 2022 Annual Meeting and Expo
The U.S. CDC recommends testing, vaccination, and masking to minimize COVID-19 spread, but few studies have tested their efficacy in public K–12 schools. During the fall of 2021, Maine schools implemented a SARS-CoV-2 pooled test-to-stay strategy amid unprecedented COVID-19 “Delta surge infection rates. At this time, 189 of Maine’s 307 known public school administrative units (SAUs) and private school entities opted into pooled testing, which offered modified quarantine allowing pooled testing participants to remain in school if identified as close contacts. Maine Center for Disease Control (Maine CDC), in collaboration with Maine Department of Education (Maine DOE) and U.S. CDC, evaluated Maine’s SARS-CoV-2 pooled test-to-stay strategy on four outcomes: number of outbreaks, number of close contacts, hospitalization rates, and case rates in pooled testing and comparison SAUs. Data sources for the quasi-experimental design included Maine CDC case, close contact, immunization, and outbreaks reporting systems, Maine DOE attendance and demographic records, school-level positive case reporting forms, pooled testing records, and American Community Survey census data from 2019. During the baseline 2019 and 2020 school years, pooled testing schools had a more racially diverse population than the schools that did not opt into pooled testing. To address selection bias, we generated balanced treatment (n=120) and comparison (n=75) samples using inverse probability of treatment weighting on propensity scores. Weighting characteristics included vaccination rates, demographic and geographic characteristics, and masking and other district-level policies related to COVID-19 prevention. We then conducted a difference-in-difference of pooled versus non-pooled testing on the four outcomes. Our findings show that among SAUs with pooled testing, fewer close contacts had to quarantine from school compared to SAUs non-pooled testing. Additionally, insignificant differences in COVID-19 case rates, hospitalization rates, and outbreaks suggest that the benefits of modified quarantine offset the risks. However, declines in disproportionately high non-White identifying incidence between baseline and endline were less in pooled testing schools than in non-pooled testing schools, and the observed COVID-19 mitigation strategies (vaccination, testing, and masking) were most influential among people who identified as White and who lived in rural/sparsely populated areas in Maine. Future school testing programs that build on this program’s successes will need to address barriers to testing programs and other COVID-19 prevention strategies, particularly among populations disproportionately affected by healthcare access inequities, by investing more funds in staffing for public health, community outreach, and translation; by adequately estimating the level of burden and staffing needed for public health initiatives in underserved schools; or, by more strictly prescribing the way public health policies and funds are administered within SAUs.
Abstract
Use of external information to improve precision in statistical estimation
APHA 2022 Annual Meeting and Expo
Combining external information from previously published results available in the form of means and standard errors with internal data is complicated due to possible bias associated with external data sources. MVAR and MMSE methods are evaluated for incorporating external information. MVAR minimizes variance and assumes external data sources are unbiased. MMSE minimizes mean squared error and relaxes the requirement of unbiasedness and allows for bias to exist. If bias is high, representing strong conflict between internal and external information, MMSE detects the bias and suppresses its impact. If bias is small, external information is used and statistical properties are improved. The methods were used in two applied projects (1) to evaluate the association between preterm birth and academic performance using previously published results and (2) to improve estimation based on the Wisconsin PRAMS dataset, with attention to potentially modifiable risk factors. In both projects, regression modeling of birth outcomes resulted in narrower confidence intervals suggesting more precise estimates, which can guide public health interventions to improve maternal-child health. To evaluate MVAR and MMSE estimators, we conducted a set of Monte-Carlo experiments with varied sample sizes (internal and external) and biases to evaluate MSEs, coverage probabilities, and type 1 and 2 errors. MVAR outperformed MMSE when no/low conflict (no/low bias) was present between the internal and external data. MMSE showed good control of coverage probability and type 1 error in all experiments and improved power properties in no/low conflict scenarios
Abstract
Using Machine Learning to Predict Opioid Overdose Using Electronic Health Records
APHA 2022 Annual Meeting and Expo
Background: Opioid overdose causes over 100 deaths in United States every day. Predicting which patients are at risk for opioid overdose can prevent overdose deaths. We sought to develop machine learning systems to predict opioid overdose risk based on data from patient electronic health records (EHR).
Methods: Our systems use both structural and unstructured data as features to predict patient-level overdose risk. Several popular machine learning models were used, including support vector machine, logistic regression, and gradient boosting. The models used features including lab results, medications, diagnoses, demographics, and case notes from social workers and clinicians. We developed a new detection tool to identify opioid-related aberrant behaviors (ORAB) and incorporated them as features. The predictive performances of the models were evaluated using data from a cohort of 1,958 patients prescribed opioids from Veterans Health Administration system (VHA).
Results: With the addition of ORAB features, the support vector machine (SVM) can predict patient overdose risk with an F-1 score of 0.82 and area under the receiver operating characteristic of 0.94.
Conclusion: Machine learning models leveraging structural and unstructured data and ORAB information in EHR as features can effectively predict the opioid overdose risk for patients. Performance is good across demographic groups. This study provides a novel method for identifying patients at risk for opioid overdose using machine learning.
Abstract
Human Annotated Texts: Monitoring Coder Behavior using Measurement Model
APHA 2022 Annual Meeting and Expo
Background: Technological advancements allow researchers to leverage machine learning (ML) procedures to classify qualitative texts, often first requiring a training sample via human annotation. Performance of Natural Language Processing (NLP) procedures depend on the coding accuracy of human-annotated texts. This study leverages the Rasch measurement (MFRM; Linacre, 1989) model to investigate coder behavior over time and how their behavior impacts coding quality.
Methods: Two human coders were provided 3000 tweets on e-cigarettes for annotation. Coders classified tweets into 11 topics, including commercial, political, and policy-based (anti, pro, neither) tweets. Coders coded 250 (weeks 1 and 2) or 500 tweets each week for seven weeks. Discrepancies were resolved using a third coder as adjudicator. Tweets were coded as correct if the coders matched one another or matched the adjudicator or incorrect if the coder did not match the adjudicator.
Results: We found commercial and THC topics were the easiest to identify for coders with adjudication rates of 0.3% and 2.1%, respectively. Classifying tweets about neither- and anti-policy topics were the most difficult with adjudication rates of 15.5% and 11.2%, respectively. The MFRM results suggest coders performed in similar fashion when coding the tweets. There were no significant differences in the difficulty levels of coding across the week, providing evidence of coder consistency over time.
Conclusions: This study provides a methodological framework for monitoring coder behavior when evaluating qualitative texts. The model employed provides a tool to further improve codebooks for qualitative research by making difficult topics easier to discern.
Abstract
Using Natural Language Processing to Detect Opioid-Related Aberrant Behaviors from Electronic Health Records
APHA 2022 Annual Meeting and Expo
Background: Opioid-related aberrant behaviors (ORABs) include non-medical use of prescription opioids and early refill requests. ORABs are important risk factors for opioid addiction and overdose. Most ORABs are not reflected in structural data but documented in electronic health record (EHR) notes. We report the development and evaluation of machine-learning-based natural language processing models for identifying ORABs from EHR notes among patients prescribed opioids.
Methods: We created a dataset by extracting relevant information from 1,742 EHR notes of 1,958 patients prescribed ≥90 days continuous supply of opioids in the Veterans Health Administration between July 1, 2005 and June 30, 2016. Domain experts labeled sentences in the EHR that contained ORABs and classified these sentences into pre-defined subcategories. Three state-of-the-art machine-learning models, including linear support vector machines (LinearSVM), denoising autoencoder for pretraining sequence-to-sequence models (BART), and a novel model, BART with Combined Features (BCF) are trained and evaluated on the annotated data. Detection accuracies of the three models are compared.
Results: The proposed BCF model, which was built on pre-trained embeddings and statistical features, performed the best, achieving a recall of 97.2%s (±0.4) for identifying whether a sentence contains an ORAB and an overall accuracy of 81.7% for classifying the ORAB into its subcategories.
Conclusions: The BCF model can effectively identify ORAB and its subcategories from the unstructured EHR notes. This work is the first step towards incorporating ORAB for opioid addiction and overdose surveillance.
Abstract
Using Observational Data to Emulate a Target Trial in Public Health Research
APHA 2022 Annual Meeting and Expo
Background: Randomized controlled trial (RCT) is a gold standard for causal inference. Yet RCT is one of the most expensive methods of data collection in terms of time and cost. Target trial approach refers to emulate a hypothetical RCT and answer causal questions using observational data, and it recently has been used in clinical research and to answer questions regarding comparative effectiveness. However, there is a scanty of studies in public health which applied target trial approach to answer causal questions. In this study, we used target trial approach to test whether HIV disclosure could improve social support as compared to non-disclosure. Methods: We specified and emulated a target trial using data derived from a prospective observational cohort among people living with HIV (PLWH) in Southwestern China. Data collection was conducted at baseline, 6-month, and 12-month follow-ups. We used inverse probability weighting (IPW) to control for the time-dependent confounding. Intention-to-treat (ITT) analysis was conducted after controlling for IPW and baseline confounders. Results: Among the 444 participants, the numbers of disclosure target increased across time, with 1 (1—2) at baseline and 2 (1—3) at 12-month follow-up. The adjusted ITT analysis showed that PLWH who disclosed their HIV status had better social support (β=9.9 [1.9—18.0]) as compared to those without disclosing. Conclusions: HIV disclosure benefits the access to social support among PLWH. Target trial approach and adjustment of time-dependent confounding can answer causal questions in public health research when RCT is not available.
Abstract
Geographic Disparities in Prevalence of Opioid Use Disorders in Veterans in the US (2009-2019)
APHA 2022 Annual Meeting and Expo
Background: In 2020, opioid overdose deaths continued to increase to 68,630 in United States. Over 1.6 million people had an opioid use disorder in the past year. Military Veterans are more prone to opioid misuse or abuse. Continued surveillance of opioid use disorder (OUD) among Veterans is of public health importance.
Methods: Using the national Veterans Health Administration (VHA) electronic health record (EHR) data, we analyzed the spatial and temporal trends in annual prevalence of OUD among Veterans from 2009 and 2019. Patients with OUD were identified based on ICD-10-CM diagnosis codes according to CDC recommendations. We calculated state-level prevalence estimates and examined their associations with state-level sociodemographic attributes according to the latest American Community Surveys (2020). Poisson models are used to analyze the state-level trends.
Results: Overall, the prevalence of OUD progressively declined from 3.42% in 2009 to 0.83% in 2019 among Veterans (p for trend<001). Census division-level prevalence ranged from 1.30% (Middle Atlantic) to 0.50% (South Atlantic). State-level prevalence ranged from 1.29% (PR) to 8.44% (WA) in 2009 and from 0.33% (MD) to 2.08% (UT) in 2019. Based on a multiple Poisson regression model, lower state-level prevalence rates were significantly associated with lower percent of being Black or Hispanic, higher per capita income, higher employment rate, higher percent without college education, and less population on food stamps.
Conclusions: While prevalence of OUD progressive decreases in the decade of 2009-2019, significant state-and census region-level disparities persisted, which are significantly associated with several key sociodemographic indicators.
Abstract
Racial differences in receipt of medications for opioid use disorder before and during the COVID-19 pandemic in the Veterans Health Administration
APHA 2022 Annual Meeting and Expo
Background: The COVID-19 pandemic exacerbated racial disparities in overdose death rates.4 Medications for opioid use disorder (OUD; MOUD) are effective for reducing overdose risk. We compared the receipt of MOUD between Black and White patients in the Veterans Health Administration (VHA) before and during the pandemic.
Methods: Using national VHA data, we compared 2 yearly cross-sections of Veterans with ≥1 OUD diagnosis: 3/1/19-2/28/20 (“2019 ) and 3/1/20-2/28/21 (“2020 ). We calculated proportions of patients with any MOUD (buprenorphine, naltrexone, methadone) receipt per year, overall and by race; and compared the proportions between Blacks and Whites using Chi-square tests.
Results: Overall, 68,241 and 63,909 veterans had any OUD diagnosis in 2019 and 2020, respectively. Proportion of veterans with OUD receiving MOUD increased from 47% in 2019 to 49% in 2020 (p<0.001) while the number decreased from 32,128 to 31,301. The proportion of Black Veterans with OUD receiving MOUD increased slightly across the years (47% in 2019 vs. 48% in 2020), as was the case for White Veterans (48% in 2019 vs. 50% in 2020). Black-to-White differences in proportion with MOUD receipt were -1.0% in 2019 and -1.7% in 2020.
Conclusions: Proportion with MOUD receipt slightly increased during the pandemic in both Black and White Veterans. However, in light of decreased numbers of Veterans with OUD diagnoses, this may be due to decreased engagement in care during the pandemic. Black-to-White difference in proportion treated slightly worsened from 2019 to 2020. Future work will incorporate additional social determinants of health and examine treatment retention.