Acute kidney injury (AKI) is common in hospitalized adults, with an incidence of ~20% and an associated 3-10-fold higher risk of death. An automated clinical decision support system for AKI could demonstrate predictive power comparable to that of human specialists. The aim of this study is to evaluate the concordance between AI and human recommendations in the setting of AKI. We hypothesize that AI recommendations will be highly concordant with those made by expert humans. Adults (≥18 years old) with AKI were enrolled across six hospitals within the Yale New Haven Health System, from the KAT-AKI Trial (NCT04040296), an ongoing clinical trial in which trained physicians and pharmacists make discreet recommendations for AKI evaluation and treatment in real time at the time of initial AKI diagnosis. With half of the data, we trained a neural network with two hidden layers of twelve neurons each to jointly predict all 43 possible recommendations. To control overfitting, we used L2 regularization of all network weights and early stopping using one sixth of the data as a validation set. Using the final third as a test set, we calculated individual AUCs and report their median across all recommendations, as well as individual AUCs across key performance indicators (KPIs) informed by global consensus guidelines. From November 2021 to May 2023, 1,831 participants were enrolled. The median(IQR) age was 73.4(62.2–82.8) years, 862(47.1%) were women, and 353(19.3%) self-identified as Black. For each patient, the neural network made 43 recommendations, for a total of 78,733 predictions. The median(IQR) AUC for all predictions was 0.74(0.67-0.83). KPIs across diagnostic and therapeutic domains include whether the following were recommended: nephrology consultation (AUC: 0.80), renal ultrasound (AUC: 0.80), urine chemistry (AUC: 0.81), echocardiogram (AUC: 0.70), telemetry (AUC: 0.99), and bicarbonate therapy (AUC: 0.90). This work demonstrates the ability to generate AI recommendations for AKI that are fairly concordant with expert recommendations.

Background:

Social determinants of health (SDOH) provide important context for understanding an individual’s healthcare challenges and inform public health actions to improve health equity. However, currently SDOH information is often incomplete, outdated, or missing from traditional data sources. Medical documents such as clinical notes, have historically been an underutilized source of SDOH data due to the effort of manual extraction. Recent advances in artificial intelligence (AI), specifically large language models (LLMs), can automate the extraction of SDOH from resources. To determine how to best utilize these advances, we assessed the ability of existing models to extract SDOH and propose a LLM SDOH software platform to be used in real-world data feeds.

Methods:

The assessment consisted of developing evaluation criteria for the LLMs, conducting a literature review, and designing a SDOH extraction software platform. The assessment criteria were applied to published LLMs to compare models and assess their ability to extract real-world data. To better utilize the high-scoring LLMs, a software platform was designed to facilitate the real-time extraction of SDOH from public health data sources.

Results:

Initial review of LLMs shows promising SDOH extraction capabilities. Publicly available LLMs can extract many types of SDOH data accurately including employment, substance use, living status, and relationship status. However, other SDOH categories, specifically housing and transportation insecurity proved to be more challenging. The identified LLMs can be readily adopted into a software platform as a stand-alone app or as part of an existing public health data pipeline to support real-time automated SDOH information captures within existing workflows.

Conclusion:

Automated SDOH extraction can reduce the burden on the public health and healthcare workforce, provide timely and more complete data, and be integrated into existing public health data exchange pipelines to improve public health’s ability to identify and address health inequities and enact context appropriate public health action.

Behavioral health epidemiologists use National Center of Health Statistics’ (NCHS) Multiple Cause of Death (MCOD) dataset to understand the prevalence and trends of drug-related mortality. With rapid advancements in the field of Machine Learning (ML) more people are turning to predictive analytics, a method that uses data, statistical algorithms, and machine learning techniques to identify the likelihood of future outcomes based on historical data. Predictive analytics can help behavioral health professionals identify at-risk populations and hotspots, take proactive action to align/allocate resources, and improve data-guided planning and decision making.

We applied PyCaret’s supervised ML module to 2021 all county restricted MCOD dataset from NCHS to test the accuracy and prediction of various models. We chose PyCaret over other open-source ML libraries, as it is low-code Python wrapper around several ML libraries that can be used to generate faster and efficient results with a few lines of codes that are relatively easy to understand and replicate. For preliminary analysis, we chose opioid-related mortality as the target outcome, and several other key demographic (e.g., age, race, sex) and geographic (e.g., state and county of residence) variables as covariates.

Our preliminary analysis results were promising. PyCaret was able to compare results from16 models (e.g., logistic regression, K nearest neighbor, CatBoost Classifier) in a manner of minutes for the 2021 dataset. Based on the preliminary analysis, CatBoost classifier was the best model with 0.71 precision, 0.51 recall, and a moderately high Kappa (0.62) and Matthew's Correlation Coefficient (0.63). We continue to refine and test the model with additional covariates and aim to present our updated findings, lessons learned, and limitations at this session.

Given the recent advances in ML, availability of the tool (i.e., Python’s PyCaret library), relative ease of coding, and enhanced speed/efficiency, ML can be a valuable tool for assisting behavioral health epidemiologists/methodologists in predicting drug-related mortality and inform decision making.

Background: By 2024, the U.S. government investment in health data and data accessibility will reach $19.9 billion. Despite the effort, health data are not entirely accessible and understood due to data sharing and visualization challenges. Visual communication challenges have created public health information gaps which are compounded in emergencies such as the COVID-19 pandemic, and potentially impacting poor health outcomes and increasing health inequities. Using the right data visualization techniques and tools for public health data communication has therefore become essential.

Objective: To examine visualization techniques and tools effective for public health visual data communication.

Methods: A scoping review was conducted to summarize the available evidence related to visualization, techniques, and tools for public health visual data communication as well as related principles and best practices. Original research articles from PubMed databases from 2020 -2024 were included.

Results: Twenty-eight published studies were included. Categorical data were mostly visualized using pie- and various variants of bar charts; continuous variables, mostly with line graphs, dot plots, box and whisker plots, and histograms; and the spread of disease as well as identifying spatial clustered patterns of incidence were visualized using geographic maps, e.g., choropleth and hotspot maps.

Conclusion: Most health data were presented visually with charts and figures, interactive dashboards, and geographic maps using ArcGIS, Tableau, D3.js, ggplot2, and plotly dash framework. The usage of the appropriate visualization techniques and tools for the right data improves comprehension, reduces public health information gaps, and potentially reduces poor health outcomes.