Towards building multilingual language model for medicine

The development of open-source, multilingual medical language models can benefit a wide, linguistically diverse audience. This work contributes: (1) a multilingual medical corpus (MMedC) of ~25.5B tokens across six languages to enable auto-regressive domain adaptation; (2) a multilingual medical multiple-choice QA benchmark with rationales (MMedBench); and (3) comprehensive evaluations of open-source LLMs with and without additional training on MMedC. The final model, MMed-Llama 3 (8B), achieves superior performance among open-source models on MMedBench and strong results on English benchmarks, rivaling GPT-4. The paper releases datasets (with licensing caveats for some books), code, and models to support multilingual medical LLM development.

Pengcheng Qiu, Chaoyi Wu, Xiaoman Zhang, Weixiong Lin, Haicheng Wang, Ya Zhang, Yanfeng Wang, Weidi Xie