Medicine and Health

Systematic analysis of ChatGPT, Google search and Llama 2 for clinical decision support tasks

S. Sandmann, S. Riepenhausen, et al.

This groundbreaking study by Sarah Sandmann, Sarah Riepenhausen, Lucas Plagwitz, and Julian Varghese explores the capabilities of advanced AI models like GPT-3.5, GPT-4, and Llama 2 in clinical decision support. Discover how GPT-4 outperformed its peers, suggesting effective diagnoses and treatments, while pointing to the crucial need for regulated AI in healthcare!

00:00

Playback language: English

Index

Abstract

This study evaluates the clinical accuracy of GPT-3.5, GPT-4, and Llama 2 LLMs for clinical decision support tasks, benchmarking against Google search. GPT-4 showed superior performance in suggesting initial diagnoses, examination steps, and treatments across various clinical disciplines and disease frequencies. Llama 2 models showed slightly lower performance. While promising, the results highlight the need for robust and regulated AI models in healthcare, with open-source LLMs offering potential advantages in data privacy and transparency.

Publisher

Nature Communications

Published On

Mar 06, 2024

Authors

Sarah Sandmann, Sarah Riepenhausen, Lucas Plagwitz, Julian Varghese

Related Publications

Explore these studies to deepen your understanding of the subject.

Medicine and Health

Population Pharmacokinetic and Exposure–Response Analysis of Finerenone: Insights Based on Phase IIb Data and Simulations to Support Dose Selection for Pivotal Trials in Type 2 Diabetes with Chronic Kidney Disease

N. Snelder, R. Heinig, et al.

Psychology

Probiotics as a Tool for Regulating Molecular Mechanisms in Depression: A Systematic Review and Meta-Analysis of Randomized Clinical Trials

A. Pinhasov, M. Sikorska, et al.

Medicine and Health

Clinical Severity of SARS-CoV-2 Variants during COVID-19 Vaccination: A Systematic Review and Meta-Analysis

Z. Yuan, Z. Shao, et al.

Medicine and Health

Systematic Review and Meta-Analysis: The clinical value of color ultrasound screening for fetal cardiovascular abnormalities during the second trimester

B. Shi, Z. Han, et al.

Listen, Learn & Level Up

Over 10,000 hours of research content in 25+ fields, available in 12+ languages.

No more digging through PDFs, just hit play and absorb the world's latest research in your language, on your time.

listen to research audio papers with researchbunny