Exploiting redundancy in large materials datasets for efficient machine learning with less data

This study reveals significant redundancy in large materials datasets, showing that up to 95% of data can be removed from machine learning training with minimal impact on in-distribution prediction performance. Redundant data is linked to over-represented material types and doesn't improve out-of-distribution performance. Uncertainty-based active learning efficiently creates smaller, informative datasets. The research challenges the "bigger is better" approach, emphasizing information richness over data volume for efficient data acquisition and machine learning.

Kangming Li, Daniel Persaud, Kamal Choudhary, Brian DeCost, Michael Greenwood, Jason Hattrick-Simpers