This study introduces TransformEHR, a transformer-based encoder-decoder generative model for predicting disease outcomes using longitudinal electronic health records (EHRs). Pretrained on a large dataset using a novel objective—predicting all diseases and outcomes at a visit from previous visits—TransformEHR achieves state-of-the-art performance on multiple clinical prediction tasks. It shows significant improvements in predicting pancreatic cancer onset and intentional self-harm in patients with PTSD, demonstrating potential for building effective clinical intervention systems. TransformEHR's generalizability is also validated through internal and external dataset evaluations.

Zhichao Yang, Avijit Mitra, Weisong Liu, Dan Berlowitz, Hong Yu