Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction

This paper introduces Med-BERT, a contextualized embedding model pre-trained on a large-scale structured electronic health record (EHR) dataset. Med-BERT adapts the BERT framework to the structured EHR domain, generating contextualized embeddings that improve the accuracy of disease prediction models, particularly those trained on smaller datasets. Experiments demonstrate significant performance boosts in two disease prediction tasks from two clinical databases, showcasing Med-BERT's potential to reduce data collection costs and accelerate AI-aided healthcare.

Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, Degui Zhi