Endometrial cancer (EC) is the most common gynecological malignancy in high-income countries, with increasing incidence. While surgery cures most women with localized disease, 10–20% develop distant recurrence, which is typically incurable. Adjuvant chemotherapy can reduce this risk but carries toxicity. Current guidelines recommend adjuvant treatment based on clinicopathological risk factors (histological subtype, grade, lymphovascular space invasion (LVSI), FIGO stage) and molecular classification (POLE mutation, p53 abnormality, mismatch repair deficiency (MMRd), or no specific molecular profile (NSMP)). However, combining these factors is complex, suffers from interobserver variability, and is costly. Histological slides contain much visual information with prognostic potential not fully captured by pathologists' assessments. Deep learning (DL) models using digitized hematoxylin and eosin (H&E)-stained tumor slides have shown promise in predicting molecular alterations, cell composition, and prognosis, often outperforming standard pathologist-based assessments. Multimodal DL models, incorporating morphological information and other clinical data, are particularly promising. Previous studies have used DL models to predict EC recurrence or overall survival, but these often relied on more detailed tumor profiling not readily available in clinical practice. This study addresses the unmet need for a method to predict EC distant recurrence using data from routine clinical diagnostics.

The existing literature highlights the challenges in predicting distant recurrence in endometrial cancer, primarily focusing on the limitations of current gold standard approaches. These approaches, which rely on a combination of clinicopathological and molecular factors, are often cumbersome and expensive to implement in routine clinical practice. The literature review further underscores the potential of deep learning models, particularly multimodal models, to improve prediction accuracy and provide clinically actionable insights. Previous studies have demonstrated the ability of DL models to extract prognostic information from H&E stained images, but many require more extensive genomic or transcriptomic data not always available. This gap in the literature creates the need for a model that can effectively predict distant recurrence using readily available data, thereby improving the personalization of treatment.

This study developed and evaluated HECTOR, a multimodal deep learning model to predict distant recurrence in endometrial cancer. HECTOR is a two-step model. The first step involves self-supervised tumor image representational learning using a vision transformer on a large dataset of H&E-stained whole-slide images (WSIs), enriching the training set with additional cohorts (including TCGA-UCEC). The second step uses a multimodal architecture to predict distant recurrence-free probabilities. This architecture fuses information from the H&E-stained WSI, image-based molecular class (predicted by im4MEC), and surgically assessed anatomical stage (FIGO 2009). The model uses attention-based multiple instance learning and embedding layers to map discrete risk factors to a continuous vector space. Ablation studies compared different model architectures and input combinations, selecting the architecture that yielded the highest C-index. The model was trained and validated on 2,072 patients from eight EC cohorts, including three large randomized trials (PORTEC-1, -2, -3), with two external test sets held out (UMCG and LUMC). The LUMC cohort allowed evaluation with multiple WSIs per patient, simulating a real-world diagnostic scenario. The PORTEC-3 cohort was used separately to evaluate HECTOR's ability to predict adjuvant chemotherapy benefit. Model explainability was assessed using Integrated Gradient (IG) values and linear regression analysis to identify morphological and genomic correlates of HECTOR risk groups. Statistical analyses included Kaplan-Meier curves, log-rank tests, Cox proportional hazards models, and multivariable regression analyses.

HECTOR demonstrated strong prognostic performance across multiple test sets. The mean C-index on fivefold cross-validation was 0.795 (95% CI: 0.768–0.822). C-indices in the internal and external test sets were 0.789, 0.828, and 0.815 respectively. This outperformed the current gold standard of combined pathological and molecular analysis. HECTOR risk stratification identified patients with markedly different outcomes; 10-year distant recurrence-free probabilities were 97.0%, 77.7%, and 58.1% for low, intermediate, and high-risk groups, respectively, in the internal test set. Similar stratification was observed in the external test sets. Multivariable analyses showed that HECTOR retained independent prognostic value even after adjusting for known clinicopathological and molecular risk factors, suggesting that these were already captured by the model. HECTOR also demonstrated superior ability to refine prognosis within the MMRd, NSMP, and p53abn molecular classes. In the LUMC external test set, using multiple WSIs slightly improved performance and risk stratification. Analysis of the PORTEC-3 trial showed a significant interaction between chemotherapy and HECTOR risk score, indicating that adjuvant chemotherapy significantly improved distant recurrence-free probabilities in HECTOR high-risk patients but not in low- or intermediate-risk patients. This predictive accuracy was superior to that provided by current methods for identifying patients likely to benefit from adjuvant chemotherapy. Explainability analyses revealed associations between HECTOR risk scores and established favorable and unfavorable risk factors. Specific morphological features (e.g., smooth luminal borders, inflamed stroma in low-risk; ragged luminal surface, LVSI in high-risk) and genomic alterations (e.g., ARIDIA, CTCF, TP53 mutations) were also identified as correlates of HECTOR risk groups.

HECTOR's superior performance compared to the current gold standard for predicting distant recurrence in endometrial cancer demonstrates the potential of multimodal deep learning to improve personalized treatment decision-making. The model's ability to predict adjuvant chemotherapy benefit offers significant clinical implications for optimizing treatment strategies and reducing unnecessary toxicity. The explainability analyses provide valuable insights into the biological mechanisms underlying EC recurrence, identifying potential therapeutic targets. While the model's performance is impressive, it is important to note that these findings need further validation in diverse, unselected cohorts and prospective trials. The integration of multiple data modalities, including the image-based molecular class and anatomical stage, is a strength of the model, highlighting the benefits of multimodal learning over unimodal approaches.

HECTOR represents a significant advancement in endometrial cancer prognostication, offering a more accurate, cost-effective, and timely prediction of distant recurrence risk compared to current methods. Its capacity to predict the benefit of adjuvant chemotherapy holds significant clinical implications for personalized treatment. Further validation in prospective trials and exploration of its potential for biomarker discovery and treatment optimization are essential next steps.

While the study included a large and diverse dataset, the majority of patients were of European ancestry, potentially limiting the generalizability of the findings to other populations. The model's reliance on multiple instance learning, which does not explicitly consider the spatial relationships between regions within a WSI, might limit its performance. Furthermore, the model did not incorporate preoperative radiology images, which could potentially improve performance. The use of a three-tiered FIGO 2009 stage might also limit the model's sensitivity to subtle differences in stage. Prospective validation in a larger, more diverse population is crucial to confirm these results and assess the clinical utility of HECTOR in real-world settings.