Sepsis, a life-threatening condition resulting from the body's overwhelming response to infection, claims millions of lives annually. Early recognition and intervention are crucial for improving patient outcomes. However, the heterogeneity of sepsis makes early detection challenging. Predictive analytics, particularly deep learning, offers a potential solution. The COMPOSER deep-learning model, which utilizes real-time data from electronic health records (EHRs), predicts sepsis onset before overt clinical manifestations. Unlike previous sepsis prediction algorithms, COMPOSER is designed to reduce false alarms by flagging uncertain cases as indeterminate. This study aimed to evaluate the real-world impact of integrating COMPOSER into clinical workflow via a nurse-facing Best Practice Advisory (BPA) in two emergency departments (EDs) within the UC San Diego Health System. The hypothesis was that the algorithm-based intervention would improve sepsis recognition and lead to better patient outcomes. A quasi-experimental before-and-after study design was employed to account for baseline differences and confounding factors. This approach compares outcomes before and after the deployment of COMPOSER, utilizing historical control data to address baseline acuity, comorbidities, seasonal influences, and secular trends.

Existing sepsis prediction algorithms within EHRs often suffer from poor positive predictive value (PPV), leading to provider mistrust and alarm fatigue. Studies implementing sophisticated machine learning models for sepsis prediction have shown promise in improving patient outcomes, such as reduced mortality and length of stay. However, many algorithms designed to predict sepsis fail to reach real-world clinical implementation. Models based on clinical criteria, such as the Systemic Inflammatory Response Syndrome (SIRS) criteria, often demonstrate limited success in improving patient-centered outcomes and have high false positive rates. While some recent studies using advanced models show positive impacts, limitations exist such as small sample sizes and lack of generalizability. The Epic Sepsis Score (ESS), a widely used predictive model, has not consistently shown improved patient-centered outcomes, with some studies revealing significant drops in diagnostic accuracy compared to initial reports and a high false positive rate.

This prospective before-and-after quasi-experimental study involved 6217 adult septic patients from January 1, 2021, to April 30, 2023, across two EDs. Sepsis was defined according to Sepsis-3 criteria. The intervention was the deployment of a nurse-facing BPA triggered by COMPOSER, a deep-learning model predicting sepsis within the next 4 hours. The study compared in-hospital mortality, sepsis bundle compliance, 72-h change in SOFA score, ICU-free days, and ICU encounters before and after COMPOSER implementation. Causal impact analysis using a Bayesian structural time-series approach was employed to account for confounders, assessing the intervention's significance at the 95% confidence level. Confounders such as emergency department volume, sex, baseline SOFA score, comorbidity burden (Elixhauser score), age, COVID-19 infection status, ED location, local trends, and seasonality were included in the analysis. The association between BPA acknowledgement and time to antibiotic administration was also evaluated using a two-sided t-test, adjusted for confounders. COMPOSER uses a feed-forward neural network, incorporating laboratory results, vital signs, demographics, comorbidities, and medications to generate a sepsis risk score. The conformal prediction method helps minimize false alarms by identifying uncertain cases. The implementation followed the EPIS (Exploration, Preparation, Implementation, Sustainment) framework, involving a multidisciplinary team, educational sessions, and iterative BPA adjustments based on feedback. A "silent mode trial" was conducted to evaluate algorithm accuracy and usefulness before live deployment. Model drift was monitored via a data quality dashboard and a predetermined change control plan was in place for retraining if necessary.

The deployment of COMPOSER was significantly associated with a 1.9% absolute reduction in in-hospital sepsis mortality (17% relative decrease; 95% CI, 0.3%-3.5%), a 5.0% absolute increase in sepsis bundle compliance (10% relative increase; 95% CI, 2.4%-8.0%), and a 4% (95% CI, 1.1%-7.1%) reduction in 72-h SOFA change. While not statistically significant, a downward trend in ICU admissions and an upward trend in ICU-free days were also observed. Analysis showed that when nurses indicated they would notify physicians immediately, there was a significant reduction in time to antibiotic administration (p=0.002). One ED showed a significant decrease in mortality post-intervention, while the other did not. Improvements were noted in various aspects of sepsis bundle compliance, such as antibiotic administration, repeat lactate measurements, and fluid administration. The Bayesian structural time-series model showed that the observed improvements in mortality and sepsis bundle compliance significantly exceeded the expected values without COMPOSER implementation.

This study demonstrates that the real-time implementation of a deep-learning model for sepsis prediction can be associated with significant improvements in patient outcomes and adherence to sepsis care protocols. The observed reduction in mortality and improved compliance suggest that COMPOSER effectively facilitates earlier sepsis recognition and intervention. The quicker administration of antibiotics when nurses actively notified physicians highlights a potential mechanism for the observed mortality benefit and reduction in organ dysfunction. The success of this implementation, which prioritized nurse-physician communication, is noteworthy given past challenges with algorithm adoption and the prevalence of alarm fatigue. While the potential for increased situational awareness among ED staff is not explicitly measured, it is a plausible contributing factor. The study's findings provide strong evidence for the potential of AI-driven interventions to enhance sepsis care.

This before-and-after study indicates that integrating the COMPOSER deep-learning model into clinical practice is associated with a significant reduction in sepsis mortality, improved sepsis bundle compliance, and faster time to antibiotic administration when nurses actively involve physicians. Further research, including multicenter randomized controlled trials, is needed to validate these findings across diverse healthcare settings and patient populations. Future studies should also investigate the long-term sustainability of the intervention and its cost-effectiveness. Exploring the impact of COMPOSER on patients without sepsis is also warranted.

The before-and-after quasi-experimental design limits the ability to make definitive causal inferences. The study was conducted in two EDs within a large academic medical center, limiting generalizability. While a large and diverse patient population was included, external validation in different healthcare settings is crucial. The relatively short post-intervention period raises concerns about long-term sustainability, and the lack of data on patients who did not develop sepsis prevents a comprehensive assessment of the intervention's overall impact.