Extracting accurate materials data from research papers with conversational language models and prompt engineering

This paper introduces ChatExtract, a method for automating accurate data extraction from research papers using conversational large language models (LLMs) and prompt engineering. ChatExtract employs a series of engineered prompts to identify data-containing sentences, extract the data (in the form of Material, Value, Unit triplets), and verify accuracy through follow-up questions. Tests on materials data show high precision and recall (near 90%) using LLMs like GPT-4, attributed to the conversational model's information retention and the use of redundant, uncertainty-inducing prompts. The method's simplicity, transferability, and accuracy suggest its potential as a powerful tool for data extraction. The authors demonstrate ChatExtract by creating databases for metallic glass critical cooling rates and high-entropy alloy yield strengths.

Maciej P. Polak, Dane Morgan