ChatClimate: Grounding conversational AI in climate science

Large Language Models have made remarkable progress in question-answering tasks, but hallucination and outdated information remain critical issues, especially in domains like climate change where timely, reliable information is vital. The authors enhance GPT-4 by providing access to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC AR6), creating a conversational AI prototype, ChatClimate (www.chatclimate.ai). They evaluate three setups—GPT-4, ChatClimate (IPCC-only), and a hybrid ChatClimate (IPCC plus in-house GPT-4 knowledge)—on 13 challenging questions. Expert evaluation shows that the hybrid ChatClimate provides more accurate responses and consistent citations, demonstrating the effectiveness of integrating authoritative external sources to improve LLM reliability in climate science.

Saeid Ashraf Vaghefi, Dominik Stammbach, Veruska Muccione, Julia Bingler, Jingwei Ni, Mathias Kraus, Simon Allen, Chiara Colesanti-Senn, Tobias Wekhof, Tobias Schimanski, Glen Gostlow, Tingyu Yu, Qian Wang, Nicolas Webersinke, Christian Huggel, Markus Leippold