This paper investigates the fundamental limits of learning closed-form mathematical models from finite, noisy datasets. The authors demonstrate a phase transition in model learning, from a low-noise phase where the true model is learnable to a high-noise phase where it is not. Probabilistic model selection proves quasi-optimal for generalization in both phases, unlike standard machine learning approaches which are limited by interpolation in the low-noise phase. The transition region presents a challenge for all methods.

Oscar Fajardo-Fontiveros, Ignasi Reichardt, Harry R. De Los Ríos, Jordi Duch, Marta Sales-Pardo, Roger Guimerà