Bayesian Linear Regression for Accurate and Efficient Atomistic Machine Learning Models

The paper presents a linear Atomic Cluster Expansion (ACE) framework for constructing interatomic potentials that are permutation and isometry invariant, and couples this with Bayesian linear regression to fit to quantum mechanical (DFT) data. The research goal is to develop a computationally efficient, systematically improvable representation of many-body atomic interactions with principled regularization and uncertainty handling. The context is fitting energies, forces, and virial stresses from DFT under locality assumptions, where robust regression and symmetry-respecting basis construction are key to accuracy and efficiency.

The work builds on representation theory of the orthogonal group O(3) to achieve rotational and reflection invariance using generalized Clebsch–Gordan coefficients (citing prior foundational works). Evidence maximization (marginal likelihood) for hyperparameter selection in Bayesian ridge regression is referenced to established literature. The text cites prior references for the sparse linear construction of isometry-invariant bases and for Bayesian model evidence, but detailed comparative review is not provided in the excerpt.

- Basis construction: Define atomic basis functions φ_nlm via radial functions and spherical harmonics, pool over neighbors to form A_nlm features (atomic basis). Take v-th order tensor products to obtain (v+1)-body correlation functions. The A-basis is permutation invariant but not rotation/reflection invariant. Construct an isometry-invariant B-basis by applying a sparse linear transformation B = C A, where C encodes generalized Clebsch–Gordan couplings derived from O(3) representation theory. - Computational scaling: The evaluation cost of a site energy scales linearly with the number of neighboring atoms and with body order (v+1), enabling efficient inference. - Deterministic linear regression: Fit parameters by minimizing a weighted squared loss over energies, forces, and optionally virial stresses: sum of w_E||E−E_DFT||^2 + w_F||F−F_DFT||^2 + w_V||V−V_DFT||^2 with Tikhonov regularization η||c||^2. The problem is recast as argmin ||y − Xc||^2 + η||c||^2 with appropriate scaling of rows for force/virial weights. - Bayesian formulation: Assume independent Gaussian observational noise with precision λ. With a Gaussian prior p(c)=N(c0,Σ0), the posterior p(c|R,y,λ) is Gaussian with closed-form mean and covariance: Σ^{-1}=Σ0^{-1}+λ X^T X and c̄=Σ(Σ0^{-1}c0+λ X^T y). This enables efficient posterior sampling and using the exact posterior mean as the model parameters. - Bayesian Ridge Regression (BRR): Use isotropic Gaussian prior p(c|α)=N(0, α^{-1}I). The log-posterior equals the ridge-regularized objective with penalty η=λ/α, naturally shrinking coefficients. Hyperparameters α and λ are determined by maximizing the marginal likelihood (evidence), whose closed-form involves |αI+λ X^T X| and the quadratic forms in y and c. - Workflow usage: BRR is employed during HAL data generation for fast Bayesian fits; Automatic Relevance Determination (ARD) is used post data generation for a final, sparser model (details of ARD not elaborated here).

- The ACE construction yields a complete permutation-invariant A-basis and, via a sparse linear transformation using generalized Clebsch–Gordan coefficients, an isometry-invariant B-basis. This supports many-body correlations up to (v+1)-body with efficient linear scaling in neighbor count and body order. - The Bayesian linear regression with Gaussian prior and noise provides closed-form posterior mean and covariance, enabling straightforward uncertainty quantification, posterior sampling, and use of the posterior mean for deployment. - BRR connects directly to ridge regularization, ensuring coefficient shrinkage; its hyperparameters (prior and noise precisions) can be selected by evidence maximization in closed form. - In practice, BRR is suitable for rapid model updates during data generation, while ARD is favored for final model refinement to enhance sparsity and relevance; quantitative performance metrics are not provided in the excerpt.

By combining symmetry-aware ACE features with Bayesian linear regression, the approach addresses the need for accurate, efficient, and robust interatomic potential fitting. The symmetry treatment ensures physical invariances, while the linear model maintains computational tractability even at higher body orders. Bayesian inference offers principled regularization and uncertainty handling, mitigating overfitting and providing a mechanism to assess confidence in predictions. The choice of BRR for iterative data-generation phases balances speed and stability, whereas ARD can yield sparser final models focusing on the most informative basis functions. The assumptions of Gaussian, independent noise simplify inference and are motivated by DFT locality and convergence properties; the authors note that alternative noise models could be adopted if warranted.

The paper outlines a linear ACE framework with an isometry-invariant basis constructed via group-theoretic couplings, coupled to Bayesian linear regression for fitting to DFT energies, forces, and virials. Key contributions include efficient scaling with neighbor count and body order, closed-form Bayesian posterior solutions enabling uncertainty quantification, and practical use of BRR during data generation with ARD for final model selection. Future work may include exploring non-Gaussian or correlated noise models, systematic studies of ARD vs BRR trade-offs, and empirical validation across diverse materials systems to assess accuracy, transferability, and computational efficiency.

- Assumes independent, homoscedastic Gaussian noise; real DFT errors may be correlated or heteroscedastic. - Relies on the locality assumption for site energies, which may be imperfect for long-range interactions or insufficient cutoffs. - The A-basis lacks isometry invariance and requires transformation; completeness and truncation choices may impact accuracy. - The excerpt provides no empirical benchmarks, leaving generalizability and performance metrics unspecified. - Regularization strength and hyperparameters depend on evidence maximization and may be sensitive to prior choices and dataset conditioning.