The increasing use of artificial intelligence (AI) in healthcare, driven by the availability of large electronic health record (EHR) datasets like MIMIC-III, promises improved healthcare delivery. However, concerns regarding bias and fairness in these AI models are emerging. AI models trained on biased data reflecting systemic injustices in healthcare risk reinforcing existing inequalities, particularly for minority populations. While various methodologies and frameworks exist for assessing bias in AI models, a standard set of metrics for comprehensive evaluation remains elusive. The concept of fairness itself lacks a universal definition, leading to multiple interpretations. This study focuses on a MIMIC-III trained benchmarking model for in-hospital mortality (IHM) prediction, a widely used model in the research community. By replicating the original study and applying a comprehensive fairness and generalizability assessment framework, the authors aim to highlight the challenges of developing fair and generalizable models using open-source EHR data. The study uses a three-stage analytical framework: (1) internal model validation, (2) external validation, and (3) internal validation after retraining. Each stage involves descriptive cohort analysis, performance and fairness evaluations (including discrimination and calibration assessments), and comprehensive reporting.

The introduction extensively reviews existing literature highlighting the potential for AI bias in healthcare and the lack of standardized fairness assessment frameworks. It mentions studies demonstrating AI's potential to exacerbate existing health disparities and the various approaches to algorithmic auditing and bias mitigation. The authors discuss the challenges of defining fairness mathematically, classifying existing fairness definitions into three main categories: anti-classification, classification parity, and calibration. The importance of data sharing and transparency for building trust and improving model quality is also emphasized. MIMIC-III is presented as a significant resource, and the authors highlight the importance of understanding its limitations in fostering fair and equitable healthcare.

The study uses a three-stage analytical framework to assess the fairness and generalizability of a MIMIC-III trained in-hospital mortality (IHM) prediction model. **Data Sources:** The study utilizes two EHR datasets: MIMIC-III (v1.4) from Beth Israel Deaconess Medical Center and STARR from Stanford Health Care. Demographic factors considered include gender, insurance type (Medicare, Medicaid, private, other), and ethnicity (White, Asian, Black, Hispanic, Other). **Benchmark Cohort Creation:** The MIMIC benchmark cohort was recreated based on the original study's code, applying exclusion criteria to ensure data consistency. A similar process was used for the STARR data. **Benchmark Model:** The study focuses on a channel-wise long short-term memory (LSTM) model, chosen for its reported superior performance in the original study. **Three-Stage Analytical Framework:** 1. **Internal Validation (MIMIC):** The model's performance is evaluated on the MIMIC dataset. 2. **External Validation (STARR):** The MIMIC-trained model is tested on the STARR dataset. 3. **Internal Validation After Retraining (STARR):** The model is retrained on STARR data and then evaluated on the STARR test set. **Descriptive Cohort Analyses:** Demographic distributions and data missingness are analyzed for both datasets. **Performance Evaluations:** Model discrimination (AUROC, AUPRC, precision, recall, accuracy) and calibration (validation plots, calibration-in-the-large) are assessed. Bootstrapping is used to create confidence intervals for discrimination metrics. Calibration is assessed using Hosmer-Lemeshow (HL) tests and validation plots. **Fairness Evaluations:** Classification parity (equal performance across demographic groups, considering AUROC and AUPRC) and calibration fairness (outcome independence from protected attributes given risk estimates) are evaluated. Calibration-in-the-large and comorbidity-risk plots (relating algorithmic risk to Charlson comorbidity score) are used to assess calibration fairness. The challenges of interpreting fairness metrics in the context of class imbalance are explicitly discussed.

The study's key findings highlight several significant challenges related to bias, fairness, and generalizability in the MIMIC-III based IHM prediction model. **Class Imbalance:** The model demonstrates a significant class imbalance problem, with low recall for the minority class (IHM events). Only approximately 20-25% of high-risk patients are correctly identified. **Fairness Concerns:** * **Socioeconomic Status:** Medicaid patients consistently receive worse predictions than privately insured patients across all stages, indicating a violation of classification parity. * **Ethnicity:** Black patients show significantly lower model performance than non-Hispanic White patients in all stages, despite sometimes having lower event rates. This disparity is particularly concerning given that class imbalance should inherently benefit groups with lower event rates. * **Calibration Fairness:** The model shows significant decalibration, particularly underestimating the risk for Medicare patients, who have a higher baseline IHM rate. **Generalizability:** While the model shows some ability to generalize to the STARR dataset, its performance and fairness characteristics deteriorate in the external validation, indicating limitations in its generalizability. Retraining on STARR improves performance, but fairness concerns persist. **Comorbidity-Risk Analysis:** The analysis of comorbidity burden (using the Charlson score) revealed disparities across groups, even when controlling for the algorithmic risk score. This suggests a potential confounding influence of socioeconomic factors on both morbidity and mortality risk. The researchers provide detailed tables and figures to illustrate these findings, including demographic distributions, data missingness statistics, performance metrics, classification parity plots, calibration-in-the-large plots, and comorbidity-risk plots.

The findings demonstrate that even well-intentioned benchmark models developed using publicly available datasets like MIMIC-III can exhibit significant bias and lack generalizability. The class imbalance significantly affects performance and fairness assessment. The consistently worse predictions for Black patients and those with public insurance highlight the risk of perpetuating systemic health disparities. The observed disparities in comorbidity burden for similar risk scores across different socioeconomic groups suggest underlying inequities in access to and quality of healthcare. The study's limitations, including reliance on two similar datasets (both from academic settings) and the challenges associated with fairness evaluation in class-imbalanced data, are acknowledged. The study underscores the importance of more rigorous evaluations, which move beyond simplistic performance metrics (like AUROC) to include comprehensive fairness assessments and external validation.

This study highlights the critical need for more thorough and comprehensive evaluations of AI models used in healthcare, particularly those built upon open-source EHR data. The findings emphasize the dangers of class imbalance and the necessity of addressing fairness concerns to avoid perpetuating existing health disparities. The authors advocate for more rigorous reporting standards that incorporate fairness metrics and extensive validation, moving beyond simple accuracy or AUROC. Future research should focus on developing methods for mitigating bias in AI models and improving the generalizability of these models across diverse populations. Addressing these issues is essential to ensure that AI-driven healthcare solutions benefit all patients equitably.

The study acknowledges several limitations. Firstly, the fairness assessment is significantly constrained by the inherent class imbalance within the dataset. Secondly, the lack of multicenter data limits the generalizability of the findings, although two academic settings were evaluated. Finally, the similarity between the two datasets (both from academic medical centers) restricts the scope of generalizability assessment to some extent.