Deep learning based automatic detection algorithm for acute intracranial haemorrhage: a pivotal randomized clinical trial

Acute intracranial hemorrhage (AIH) is a life-threatening condition with high mortality rates. Brain CT scans are crucial for diagnosis, but prompt and accurate assessment remains challenging due to the high volume of data and potential for human error. Deep learning-based AI offers a potential solution to improve diagnostic accuracy and efficiency. Previous AI algorithms primarily used supervised learning with expert labeling, leading to potential discrepancies between experts. This study proposes a novel AI algorithm combining supervised hemorrhage detection and unsupervised anomaly detection to address these limitations. The algorithm uses a joint convolutional neural network (CNN)-recurrent neural network (RNN) architecture for hemorrhage detection, aiming for improved diagnostic performance compared to conventional CNN-based approaches.

Existing literature highlights the need for improved AIH detection methods due to the high mortality associated with delayed diagnosis. While MRI is highly accurate, its limitations (cost, availability, time) make CT the primary diagnostic tool. Deep learning has shown promise in medical image analysis, but previous AIH detection algorithms often relied on supervised learning methods, leading to inter-observer variability in labeling and training data inconsistencies. This study aimed to overcome these limitations by incorporating unsupervised anomaly detection in addition to supervised hemorrhage detection.

The study involved three phases: 1) Algorithm development using 104,466 slices from 3010 patients (104,666 slices from 3010 patients in the paper); 2) External validation using a large dataset of 1,855,465 slices from 49,841 patients; and 3) A retrospective, multi-reader, crossover, randomized reader study with 12,663 slices from 296 patients evaluated by nine reviewers (three non-radiologists, three radiologists, and three neuroradiologists) with and without AI assistance. The AI algorithm combined a supervised hemorrhage detection process (using a CNN-RNN architecture) and an unsupervised anomaly detection process. The reader study compared the diagnostic performance (sensitivity, specificity, accuracy) of AI-assisted and AI-unassisted interpretations using the chi-square test and generalized estimating equations (GEE).

The AI algorithm demonstrated high accuracy in both the external validation dataset (AUC of 0.992 patient-wise, 0.977 slice-wise) and the reader study. In the reader study, AI-assisted interpretation showed significantly higher diagnostic accuracy (0.9703 vs. 0.9471, p < 0.0001, patient-wise) compared to AI-unassisted interpretation. Non-radiologist physicians showed the most significant improvement in diagnostic accuracy with AI assistance. Sensitivity and specificity were also significantly improved in the AI-assisted group, particularly for patient-wise analysis. Standalone AI performance showed high AUROC values comparable to neuroradiologists with AI assistance.

The study's findings demonstrate the potential of the novel AI algorithm for improving AIH detection accuracy, particularly for non-radiologists. The combined supervised and unsupervised approach effectively addresses inter-observer variability issues associated with traditional supervised learning methods. The use of a CNN-RNN architecture further enhances the algorithm's ability to process 3D data and generate accurate patient-wise probability scores. The results highlight the potential of AI to improve diagnostic efficiency and reduce diagnostic errors, contributing to improved patient care.

The developed AI algorithm, combining supervised hemorrhage detection and unsupervised anomaly detection, significantly improves AIH detection accuracy on brain CT scans. AI assistance significantly benefits all clinicians, with non-radiologists demonstrating the largest gains. This algorithm shows promise for improving diagnostic efficiency and patient outcomes; however, further clinical validation and investigation of its role in managing AIH are warranted.

The study's retrospective design and potential selection bias are limitations. The reading environment in the reader study did not completely replicate real-world clinical practice, where additional clinical information might influence physician decisions. Class imbalance in the gold-standard review board could have also influenced the results. The lack of data on clinical outcomes related to morbidity and mortality limits a comprehensive assessment of the algorithm's clinical utility.