Breast cancer is a significant global health concern, being the most frequently diagnosed cancer and a leading cause of cancer-related deaths among women. Early detection is crucial for effective treatment and improved survival rates. Mammography, while widely used, has limitations, particularly in women with dense breast tissue. Ultrasound (US) serves as a valuable complementary and often primary modality in breast cancer diagnosis, offering advantages such as lower cost, lack of ionizing radiation, and real-time assessment. However, the interpretation of breast US images is challenging due to inter-observer variability and the high rate of false-positive findings. This variability leads to unnecessary recalls, further imaging, and biopsies, many of which are ultimately benign. Computer-aided diagnosis (CAD) systems using deep learning have shown promise in improving breast cancer diagnosis from US images. However, many prior systems rely on small datasets or exhaustive manual annotations, limiting their generalizability and real-world applicability. Existing studies have largely focused on differentiating benign and malignant lesions, neglecting the majority of negative (no lesion) exams. This work presents a novel AI system designed to reduce false positives in breast US interpretation, addressing the limitations of previous approaches by leveraging a large dataset and employing a weakly supervised learning paradigm for improved explainability and generalizability.

Existing literature highlights the challenges and opportunities in AI-assisted breast ultrasound interpretation. Early CAD systems often relied on handcrafted features, which lacked generalizability across different US units and protocols. Recent advances in deep learning have enabled more robust AI systems, but many rely on small, exhaustively labeled datasets or image-level/pixel-level labels, hindering their clinical translation. Previous research primarily focused on discriminating benign from malignant lesions, overlooking the large number of negative exams in clinical practice. The lack of explainability in many AI models also hinders trust and adoption by clinicians. Therefore, the current study aims to improve upon previous work by using a significantly larger, more diverse dataset, and implementing a weakly supervised learning approach which improves the explainability of the system's predictions.

This study used the NYU Breast Ultrasound Dataset, comprising 288,767 exams (5,442,907 images) from 14,203 patients. The dataset was split into training (60%), validation (10%), and internal test (30%) sets. Breast-level cancer labels were automatically extracted from pathology reports, utilizing a weakly supervised learning paradigm. The AI system is a deep convolutional neural network (CNN) that generates saliency maps, highlighting regions of interest for benign and malignant lesions. The architecture incorporated an attention module to weight the importance of different images within an exam. An external dataset, the BUSI dataset, was used for independent validation. A retrospective reader study was conducted, comparing the AI system's performance to that of ten board-certified breast radiologists on a subset of 663 exams. Radiologists assigned BI-RADS scores, and performance was assessed using AUROC, AUPRC, sensitivity, specificity, PPV, and biopsy rate. A hybrid model combining AI and radiologist predictions was also evaluated. Statistical analysis included calculating confidence intervals using bootstrap methods. Qualitative analysis of saliency maps was performed on selected cases to understand the AI's decision-making process. Finally, analysis was conducted to evaluate the potential of the AI system in triaging exams, identifying cancer-negative cases with high confidence. The detailed architecture of the deep learning model, including the saliency map generation, attention mechanism, and cancer diagnosis model, is described in the supplementary materials. This included the use of convolutional neural networks, global max pooling, and a gated attention mechanism. Specific implementation details such as image preprocessing, data augmentation techniques, and model training parameters are thoroughly described within the methods section of the paper. Various optimization techniques like L2 regularization and Adam optimizer were employed to improve model performance. The specific training and evaluation metrics are also defined. In addition, model ensembling was used by combining the predictions of the top three performing models.

The AI system achieved a high AUROC of 0.976 on the internal test set, significantly outperforming the average radiologist AUROC of 0.924 (p<0.001). The AI maintained high accuracy across different age groups, breast densities, and US machine manufacturers. In the reader study, the AI system achieved a higher AUROC (0.962) than the average radiologist (0.924). Compared to the average radiologist's specificity, the AI demonstrated improved sensitivity (94.5% vs 90.1%, p=0.0278). At the average radiologist's sensitivity, the AI showed improved specificity (85.6% vs 80.7%, p<0.001) and PPV (32.5% vs 27.1%, p<0.001) with a lower biopsy rate (19.8% vs 24.3%, p<0.001). The hybrid model, integrating AI and radiologist predictions, further improved performance, increasing specificity, PPV, and decreasing biopsy rates. The AI system reduced the radiologist's false-positive biopsy rate by 37.3%. Qualitative analysis of saliency maps provided insights into the AI's decision-making process, highlighting regions of interest in both correctly and incorrectly classified cases. The study also showed the potential of the AI for triaging exams, achieving an NPV of 98.6% at a specificity of 77.7% for cancer-negative case identification.

The findings demonstrate the potential of the AI system to improve the accuracy, consistency, and efficiency of breast ultrasound diagnosis. The system's superior performance compared to radiologists, particularly in reducing false positives, has significant clinical implications, leading to fewer unnecessary biopsies and patient anxiety. The hybrid model further highlights the benefits of combining AI with human expertise. The high performance across various patient subgroups indicates good generalizability. The weakly supervised learning paradigm allows for training on large datasets, which is crucial for achieving high accuracy in medical image analysis. The explainability feature, provided by the saliency maps, increases trust and facilitates better understanding and integration into clinical workflows. These results suggest the system's potential for improving patient care and reducing healthcare costs.

This study presents a high-performing AI system for breast ultrasound interpretation that surpasses the performance of experienced radiologists in reducing false positives while maintaining sensitivity. The integration of this AI system into clinical practice has the potential to significantly improve breast cancer diagnosis, reduce unnecessary biopsies, and enhance overall healthcare efficiency. Future research should focus on prospective clinical validation, integration with other imaging modalities (multimodal learning), and exploring the AI's role in risk stratification and personalized breast cancer screening.

The study's retrospective nature limits the generalizability of findings. The reader study didn't fully replicate real-world clinical practice, as radiologists lacked access to complete patient history and other imaging modalities. The size of the external validation dataset was relatively small, and images were acquired from a single US system. Further prospective studies are needed to confirm the AI's effectiveness in diverse clinical settings.