Predictive modeling of proliferative vitreoretinopathy using automated machine learning by ophthalmologists without coding experience

Proliferative vitreoretinopathy (PVR) is a significant complication following rhegmatogenous retinal detachment (RRD) repair, responsible for a substantial number of surgical failures. While several clinical and biological risk factors have been identified (trauma, aphakia, vitreous hemorrhage, pre-existing PVR), existing predictive formulas based on these factors have shown poor performance, limiting their clinical utility. The field of artificial intelligence (AI), specifically machine learning (ML), offers potential for improved prediction. However, the development of ML models traditionally requires expertise in coding and data science, restricting access for clinicians. Automated machine learning (AutoML) tools offer a solution by simplifying the process and making it accessible to non-experts. This study explores the feasibility of using AutoML to develop ML models for predicting postoperative PVR by ophthalmologists without prior coding experience, using clinical data from electronic health records (EHRs). The low incidence of PVR in clinical cohorts presents a challenge of class imbalance in developing predictive models, highlighting the need for robust data preprocessing techniques.

Previous research has identified various clinical and biological risk factors for PVR, including trauma, aphakia, vitreous hemorrhage, and pre-existing PVR. Multiple formulas have been developed to predict PVR based on these variables; however, their predictive performance has been inadequate for routine clinical use. While AI solutions have been applied to ophthalmic imaging data, their use with clinical data from EHRs is less prevalent. The low incidence of PVR in clinical cohorts poses a challenge for building predictive models due to class imbalance issues, affecting the accuracy and reliability of the models. A recent study demonstrated the feasibility of deep-learning model design by physicians without coding experience, utilizing AutoML programs.

This retrospective cohort study included 506 eyes that underwent pars plana vitrectomy for RRD by a single surgeon between 2012 and 2019. Fifteen clinical features were collected from EHRs (Table 1), encompassing sociodemographic data, past ocular history, retinal detachment characteristics, and other examination findings. Missing data were handled through imputation using the median. To address class imbalance (46 PVR cases, 460 non-PVR cases), random undersampling (RUS) was applied, resulting in a dataset with a 2:1 case-control ratio (46 PVR, 92 non-PVR cases). Two ophthalmologists without coding experience, using an interactive application in MATLAB, developed ML models. Univariate feature selection was performed, creating two feature sets: one including all significant features and another excluding "pre-existing PVR." The ophthalmologists trained five classifiers (Naïve Bayes and Support Vector Machine) and selected the two best-performing models for each feature set using five-fold cross-validation. A data scientist independently developed comparable models as a benchmark using manual coding. Model performance was evaluated using AUC, sensitivity, specificity, PPV, NPV, and F1 score. Statistical analysis included non-parametric tests (Mann-Whitney U and Fisher's exact/Chi-square tests).

Several clinical features were significantly associated with postoperative PVR (Table 2): older age, longer symptom duration, subtotal/total RRD, macula-off detachment, giant tears, vitreous hemorrhage, pre-existing PVR, lower intraocular pressure, and uveitis. Four ML models were developed and evaluated (Table 3, Figure 2): a quadratic SVM and optimized Naïve Bayes model using all significant features (Feature Set 1), and corresponding models using the same features excluding pre-existing PVR (Feature Set 2). The quadratic SVM model (Model 1) using Feature Set 1 demonstrated the best discriminative performance, with an AUC of 0.90, sensitivity of 63.0%, and specificity of 97.8%. The prevalence-adjusted PPV was 74.4%. The optimized Naïve Bayes model (Model 2) using Feature Set 1 had an AUC of 0.86. Models using Feature Set 2 showed lower performance. Benchmarking against manually coded algorithms showed comparable F1 scores (Supplementary Table S2), indicating the reliability of the AutoML approach. Figure 3 presents illustrative case examples, including correctly and incorrectly classified cases, highlighting the model's strengths and limitations.

This study demonstrates the feasibility of ophthalmologists without coding experience developing and utilizing ML models to predict postoperative PVR using AutoML. The high specificity of the best-performing model (Model 1) makes it useful for ruling in PVR; however, its lower sensitivity indicates a need for caution in interpreting negative predictions. The inclusion of pre-existing PVR significantly improved model performance. The study's findings align with previous research indicating the importance of clinical risk factors in PVR development. While this study used a large dataset, the low incidence of PVR necessitated class balancing techniques to address class imbalance. Further external validation is crucial to assess the generalizability of the models for broader clinical use.

This study successfully demonstrates that ophthalmologists without coding expertise can effectively use AutoML tools to build machine learning models for predicting postoperative PVR. While data scientist input was required for initial data preparation, the results suggest the potential for increased clinician involvement in developing AI solutions for ophthalmology. Future research will focus on external validation and assessing the clinical benefits of using these models to guide patient management and potentially reduce the healthcare burden.

The study's primary limitation is the lack of external validation. The absence of variability measures from the graphical interface hindered direct comparison between the AutoML and manually coded models. Although F1 scores were similar, subtle differences in performance might exist due to variations in algorithm optimization thresholds. The use of undersampling for class balancing, while effective, may lead to the loss of information. Further investigation is needed to explore other strategies like oversampling or cost-sensitive learning. Generalizability across different surgical techniques and patient populations requires external validation.