Quantifying Distribution Shifts and Uncertainties for Enhanced Model Robustness in Machine Learning Applications

Distribution shifts, where training and test datasets differ statistically, challenge real-world machine learning. This study uses synthetic data to investigate model adaptation and generalization across diverse distributions, quantifying associated uncertainties. Synthetic data is generated using the van der Waals equation, with data similarity assessed using Kullback-Leibler divergence, Jensen-Shannon distance, and Mahalanobis distance. Results show that Mahalanobis distance helps identify whether model predictions fall within low-error interpolation or high-error extrapolation regimes, providing insights for enhancing model robustness and generalization.

Vegard Flovik