Towards building multilingual language model for medicine

The development of open-source, multilingual medical language models can benefit a wide, linguistically diverse audience. This work contributes: (1) a multilingual medical corpus (MMedC) of ~25.5B tokens across six languages to enable auto-regressive domain adaptation; (2) a multilingual medical multiple-choice QA benchmark with rationales (MMedBench); and (3) comprehensive evaluations of open-source LLMs with and without additional training on MMedC. The final model, MMed-Llama 3 (8B), achieves superior performance among open-source models on MMedBench and strong results on English benchmarks, rivaling GPT-4. The paper releases datasets (with licensing caveats for some books), code, and models to support multilingual medical LLM development.