The unreliability of crackles: insights from a breath sound study using physicians and artificial intelligence

Background and Introduction: Respiratory auscultation suffers from inconsistency and inter-observer variability compared to other physical assessments. Objectives: To evaluate differences in the ability of physicians and artificial intelligence (AI) to identify different breath sounds. Methods: In a prospective study, 11,532 breath sounds from the Formosa Archive of Breath Sound were labeled by five physicians. Six AI models were trained: five models based on each physician’s labels and one All-data AI model trained on all labels. Discrepant labels between physicians and AI were deemed doubtful and re-assessed by two additional physicians; final labels were determined by majority vote. Performance of physicians and AI was assessed using sensitivity, specificity, and AUROC. Results: Of 11,532 files, 579 labels were doubtful; after relabeling and exclusion, 305 labels constituted the gold standard dataset. For wheezing, physicians and AI showed good sensitivity (89.5% vs. 86.6%) and specificity (96.4% vs. 95.2%). For crackles, both showed good sensitivity (93.9% vs. 80.3%) but poor specificity (56.6% vs. 65.9%). AUROC values were lower for crackles. Conclusion: Even with AI assistance, identifying crackles remains challenging compared to wheezing; crackles are unreliable for medical decision-making without further examination.