Renal cell carcinoma (RCC), the most common type of kidney cancer, is increasingly diagnosed at localized stages where surgery is the standard treatment. However, a significant proportion of patients (20-50% at 5 years) experience recurrence after surgery. Current prognostic scores, like AUSIS, SSIGN, GRANT, and the leucine risk score, provide only moderate predictive performance, hindering the ability to tailor post-operative surveillance and adjuvant therapy. This limitation necessitates more precise individual risk prediction to improve patient management. The advent of personalized medicine underscores the urgent need for accurate prediction models capable of identifying patients at high risk of recurrence, allowing for intensified surveillance or consideration of adjuvant treatment, while simultaneously identifying low-risk patients suitable for less intensive follow-up. This study aimed to address this need by developing and validating a machine learning model using real-world prospective data from a large, multicenter cohort of surgically treated RCC patients.

Existing prognostic scores for kidney cancer recurrence after surgery, such as the UISS, SSIGN, GRANT, and Leibovich scores, demonstrate only moderate predictive accuracy. Studies using machine learning to predict recurrence have shown promise, but often suffer from limitations including small sample sizes and insufficient methodological detail. The UISS, based on Fuhrman grade, ECOG score, and pT stage, shows a moderate c-index (0.56-0.72). SSIGN, incorporating stage, tumor size, Fuhrman grade, and necrosis, has a varying c-index (0.63-0.78) in external validation. The GRANT score, including Fuhrman grade, age, stage, and lymph node involvement, demonstrates low concordance (0.59). Previous machine learning approaches, while suggesting improved accuracy, lack robust validation and detailed methodology descriptions. This research sought to improve upon existing methods by utilizing a larger, well-characterized dataset and rigorous model validation techniques.

This study leveraged data from the French kidney cancer research network database, UroCCR, encompassing patients who underwent surgery for localized or locally advanced RCC between May 2000 and January 2020. After exclusion criteria were applied (hereditary RCC, non-primary tumors, benign lesions, concomitant malignancies, metastases, insufficient data), a total of 3372 patients were included. Participating centers were randomly assigned to training (n=2241) and testing (n=1131) cohorts to evaluate the model's generalizability. The dataset included clinical, pathological, and biological variables. Missing data were handled using multiple imputation (MICE). Feature selection was performed, removing non-informative variables, resulting in a final set of 24 variables. Multiple time-to-event models were trained on the training dataset: Cox proportional hazards models with LASSO regularization, random survival forests, and gradient-boosted survival trees. Hyperparameter tuning was performed using repeated cross-validation and Bayesian optimization, maximizing the integrated AUC (iAUC). The best performing model, a Cox PH model, was selected and externally validated on the independent testing cohort. Model performance was assessed using integrated AUC, Brier score, and decision curve analysis (DCA). Furthermore, patients were stratified into four risk groups (very low, low, medium, high) based on 5-year recurrence risk, using thresholds determined from the training cohort. Finally, the ML model’s performance was compared to that of several established prognostic scores (UISS, SSIGN, GRANT, Leibovich) using the testing cohort, focusing on integrated AUC and Brier score. Statistical significance was determined using bootstrapping for comparisons between the ML model and the standard risk scores.

The best-performing model, a Cox proportional hazards model incorporating 24 clinical, pathological, and biological variables, achieved an integrated AUC (iAUC) of 0.81 (95% CI 0.77–0.85) on the test dataset. This outperformed the predictive ability of established risk scores (UISS, SSIGN, GRANT, and Leibovich scores), demonstrating its superiority in predicting disease-free survival (DFS) in this patient cohort. Notably, the ML model also displayed better performance in the context of incomplete data, achieving comparable results across different time horizons up to two years, with a slight decrease in accuracy observed at five years, likely attributable to the reduced number of events and patients at risk. The integrated Brier score was 0.11 (0.10–0.13), indicating good calibration. The decision curve analysis (DCA) illustrated the clinical utility of the ML model, highlighting a greater net benefit compared to decisions assuming no recurrence or recurrence in all patients. For a 30% threshold probability, the net benefit of 0.10 translates to detecting 10 additional recurrences per 100 patients without increasing false positives. Importantly, the model's robustness was confirmed through stable predictive metrics across training and external validation. SHAP values were employed to interpret individual predictions, identifying key factors influencing recurrence risk such as tumor size, histological subtype, Fuhrman grade, necrosis, and age. Patient stratification into four risk groups (very low, low, medium, high) achieved an AUC of 0.78 (95% CI 0.74–0.83), showing good discrimination in the test cohort. The percentages of patients within these groups and their corresponding 5-year DFS rates were presented. Comparison with conventional risk scores revealed statistically significant superior performance for the ML model against the UISS, SSIGN, and GRANT scores.

The study's findings highlight the superior performance of the ML-based UroPredict model compared to conventional prognostic scores in predicting kidney cancer recurrence after surgery. This improvement is attributed to the model's ability to integrate a larger number of variables and handle missing data effectively. The model’s strong performance, validated on an independent test set, demonstrates its potential to improve clinical decision-making. The identification of important features, such as tumor size, histological subtype, and Fuhrman grade, aligns with current understanding of RCC prognosis, bolstering the model's clinical relevance. The ability to stratify patients into risk groups further enhances the model’s clinical utility by guiding personalized surveillance strategies and adjuvant therapy decisions, potentially optimizing patient care and resource allocation. The model's limitations, including potential bias due to the inclusion criteria, must be considered.

This study demonstrates the value of machine learning in predicting kidney cancer recurrence after surgery. The UroPredict model offers a significant improvement over existing prognostic scores, enabling more personalized treatment decisions and enhancing patient management. Future research should focus on validating the model in diverse populations and incorporating additional data such as genomic information to potentially further enhance its predictive accuracy.

The study is limited by its reliance on a single national database, potentially limiting generalizability to other populations. The long-term predictive accuracy may decrease due to the limited follow-up time. While missing data was handled by multiple imputation, residual bias might exist. The model's performance might vary across different healthcare settings due to variations in surgical techniques and follow-up practices. The lack of external validation beyond the UroCCR dataset presents an area for future improvement.