The COVID-19 pandemic has placed immense strain on global healthcare systems. Accurate and efficient screening for SARS-CoV-2 is crucial for mitigating this burden. While RT-PCR testing is the gold standard, its limited availability necessitates alternative screening methods. This study addresses this need by developing a machine-learning model to predict COVID-19 infection based on readily available patient information. Existing prediction models often rely on data from hospitalized patients, limiting their applicability to broader populations. The researchers leverage a unique dataset from the Israeli Ministry of Health, encompassing all individuals tested for SARS-CoV-2 during the initial months of the pandemic in Israel. This dataset includes demographic information, symptom reports, and testing results, offering a valuable resource for model development and evaluation, free of significant referral bias. The model aims to assist healthcare workers in triaging patients and prioritizing testing, especially in resource-constrained settings.

Several previous studies have explored the use of machine learning to predict COVID-19 diagnosis, employing various features such as CT scans, clinical symptoms, and laboratory tests. However, most previous models were based on data from hospitalized patients, which limits their generalizability. This study builds on existing research by using a more representative dataset encompassing the general population and using a set of simple, accessible features. This approach helps to improve efficiency and addresses the limitations of existing models. The researchers note studies that have identified other symptoms, such as anosmia (loss of smell) and ageusia (loss of taste), as highly predictive of COVID-19 but these symptoms were unavailable in their dataset.

The study utilized a dataset of 99,232 individuals tested for SARS-CoV-2 in Israel, publicly released by the Israeli Ministry of Health. The data included eight binary features: sex (male/female), age ≥60 years (true/false), known contact with an infected individual (true/false), and five symptoms (cough, fever, sore throat, shortness of breath, headache, each true/false). The data was split into a training-validation set (51,831 individuals, 4769 positive cases) and a prospective test set (47,401 individuals, 3624 positive cases). A gradient-boosting machine model was used, with LightGBM for implementation. Missing values were inherently handled by the model. The validation set was used for early stopping, optimizing for auROC. SHAP values were calculated to identify feature importance. The model was evaluated on the test set using auROC, precision-recall curves, and several other metrics including sensitivity, specificity, PPV, NPV and accuracy. Confidence intervals were derived using bootstrapping (1000 repetitions). The impact of biased symptom reporting was investigated by training and testing the model with and without biased features (headache, sore throat, shortness of breath), demonstrating the model's robustness even under biased conditions. Simulated test sets were created by randomly replacing negative symptom reports with blank values. The Tel Aviv University IRB deemed the use of the public dataset exempt from approval.

The model achieved an auROC of 0.90 (95% CI: 0.892-0.905) on the prospective test set. Possible operating points included 87.30% sensitivity and 71.98% specificity, or 85.76% sensitivity and 79.18% specificity. The auPRC was 0.66 (95% CI: 0.647-0.678). SHAP value analysis revealed that cough, fever, and known contact with an infected individual were the most important features. Male sex was also identified as a predictive factor. Analysis of biased symptom reporting indicated that features like headache, sore throat, and shortness of breath showed imbalanced reporting. The model's performance was evaluated, even with removing features with high reporting bias. When trained and tested with only balanced features, an auROC of 0.862 (95% CI: 0.855-0.870) was achieved. The model maintained promising results even when simulated test sets were used, substituting negative values with blank values for all five symptoms. The study notes a limitation that the symptoms were self-reported, resulting in potential underreporting or misreporting.

The developed model provides a valuable tool for COVID-19 screening and testing prioritization, particularly relevant in resource-constrained settings. The use of readily available features makes it easily implementable globally. The high accuracy achieved on a separate test set supports the model's effectiveness. Addressing limitations related to biased symptom reporting highlights the model's robustness despite data imperfections. The study shows that despite limitations, a simple model achieves high accuracy. This model could be useful for prioritizing testing and thus improve efficiency. The significance of this research lies in its ability to assist in managing healthcare resources effectively during future waves of the pandemic or outbreaks of other respiratory illnesses. The study also highlights the importance of robust data collection and sharing for improved model development and understanding of COVID-19 pathogenesis.

This study demonstrates the effectiveness of a simple, machine learning-based model for predicting COVID-19 diagnosis using basic patient information. The model's high accuracy and ease of implementation make it a valuable tool for prioritizing testing in resource-limited settings. Future work could incorporate additional symptoms, improve data quality, and investigate the model's performance in diverse populations. The methodology could be adapted for other respiratory viruses.

The study acknowledges limitations associated with the use of self-reported symptoms, which are susceptible to bias. The availability of features could vary across different settings. Missing data and potentially biased reporting of symptoms amongst those testing negative for COVID-19 could affect the model's generalizability. The model could be improved with more detailed information regarding contact with confirmed cases, such as duration and location. The lack of certain predictive features, like loss of smell and taste, which were not collected in the dataset, may impact overall accuracy.