Towards understanding structure-property relations in materials with interpretable deep learning

Materials discovery is slow and often serendipitous; materials informatics (MI) seeks to accelerate it with data-driven methods. A key challenge is learning material representations that not only yield accurate predictions but are interpretable to reveal structure–property relationships. Traditional quantum-mechanics-based simulations (e.g., DFT) are accurate yet computationally expensive across vast chemical spaces. MI transforms raw structural/compositional data into representations amenable to ML, but interpretability remains limited, especially with deep models acting as black boxes. This study asks whether an attention-based deep architecture can both accurately predict properties and provide interpretable signals linking local atomic environments to global material properties. The purpose is to design and validate an interpretable model that quantifies the contribution (attention) of local structures to material-level properties, thereby elucidating structure–property relations in molecules and crystals.

The paper reviews MI pipelines: datasets, representations, and ML algorithms. Prior representation learning includes continuous-filter convolution networks (SchNet), crystal graph networks (CGCNN, MEGNet), and GNNs with physical constraints (Cormorant, SE(3)-Transformer) or higher-order interactions (ALIGNN with line graphs and angular information). Message passing neural networks leverage bonding heuristics but struggle with long-range interactions and global context. Transformer-based models with attention have emerged to model interatomic interactions, yet deriving meaningful, interpretable structure–property relations from attention remains challenging. Pooling strategies (max/average) can obscure interpretability. Incorporating prior knowledge and constraints can improve accuracy but may increase dimensionality and bias, potentially hindering clear interpretation.

The authors propose SCANN, an interpretable attention-based deep learning architecture for atomistic systems. Method highlights: 1) Local structure representation: For each atom (central atom) and its neighbors, the model recursively applies attention mechanisms (LocalAttention layers) to encode the local environment, integrating neighbor identities and geometric arrangement. 2) Global attention (GA): Material-level representation x_S is computed via a key–query attention layer SAttention using queries Q^θ and keys K^θ derived from local structure embeddings C^1. Instead of standard softmax on pairwise attention, SCANN computes an attention matrix A = Q^θ K^θ, sums, for each column j, all incoming attentions from other local structures (excluding self via identity I), s_j = Σ_i A_{ij}(1 − δ_{ij}), then applies ρ(A) = softmax(s_1, …, s_M) to obtain GA scores α^θ = [α_1^θ, …, α_M^θ]. The material representation is x_S = α^θ K^θ, explicitly weighting local structures by their global attentions. 3) Property prediction: A fully connected head F_s maps x_S to the target property ŷ_s, capturing nonlinear relationships. 4) Training and evaluation: Models are trained with MAE losses using train/validation/test splits. For QM9, five properties (E_HOMO, E_LUMO, E_gap, α, C_v) are predicted with 80/10/10 splits repeated five times to report average test MAE. For Materials Project MP 2018.6.1, the standard 60k/5k/4,239 split is used to predict ΔE (formation energy per atom) and E_g (band gap). Comparisons include WaveScatt, SchNet, MEGNet, Cormorant, SE(3)-Trans, and ALIGNN. 5) Interpretability analyses: GA scores are compared with first-principles (DFT) results across four scenarios: correspondence of GA with HOMO/LUMO spatial distributions in small molecules (QM9) and fullerenes; correspondence of GA with structural deformation features (elongated C–C bonds and curvature) in Pt/graphene adsorption; and relation of GA at substitution sites to formation energies in SmFe12-based crystals. 6) Variant SCANN+: To enhance geometric descriptiveness, a Voronoi solid angle embedding and multiple LocalAttention updates are added (detailed in Supplementary), improving accuracy but increasing dimensionality. Implementation/training details (initialization, optimizer) and datasets Fullerene-MD, Pt/Graphene-MD, SmFe12-CD are described in the Supplementary; code is available on GitHub.

- Predictive performance on QM9 (average test MAE over 5 runs): SCANN achieved E_HOMO 41 meV, E_LUMO 37 meV, E_gap 61 meV, α 0.141 Bohr^3, C_v 0.05 cal mol^-1 K^-1; SCANN^1 variant: 32, 31, 52, 0.115, 0.041, respectively. ALIGNN had the lowest MAEs among compared models for four properties. SCANN models were 2–2.5× higher MAE than ALIGNN on some properties but competitive with SchNet, MEGNet, Cormorant, and SE(3)-Trans, especially for E_HOMO, E_LUMO, and E_gap. Chemical accuracy thresholds (QM9) were met by SCANN for E_HOMO (41 meV), E_LUMO (34 meV), and C_v (0.05 cal mol^-1 K^-1). - Predictive performance on MP 2018.6.1: SCANN obtained ΔE = 29 meV atom^-1 and E_g = 260 meV; relative to ALIGNN (22 meV atom^-1, best reported), SCANN’s ΔE MAE is 24% higher and E_g MAE 19% higher. SCANN outperformed CGCNN and SchNet for ΔE, and outperformed CGCNN and MEGNet for E_g. SCANN+ further reduced errors (ΔE 28 meV atom^-1; E_g 225 meV), second only to ALIGNN. - Interpretability via GA scores: • Small molecules (QM9): GA maps correspond closely to DFT HOMO/LUMO orbital localizations for representative molecules (dimethyl butadiene, thymine, methyl acrylate, dimethyl fumarate), supporting chemically meaningful attention. Statistical analysis over C/N/O sites shows higher relative GA for E_HOMO on O > N > C, with sp/sp^2 carbons > sp^3, consistent with electronegativity and π-electron density; E_LUMO GA showed less differentiation among elements. • Fullerenes: For symmetric C60, GA scores are uniform across atoms; for C70 (D5h) and C72 (D6h), GA patterns reflect molecular symmetry and degenerate HOMO/LUMO shapes, capturing ring-localized LUMO in C72 and planar node features in C70. • Pt/graphene deformation: High GA highlights elongated C–C bonds and convexed (high-curvature) carbon sites, aligning with DFT-derived contributions to deformation energy ΔU. • SmFe12-based crystals: In SmFe12, Fe sites share equal GA (symmetric Fe cage). Upon substitution, Mo yields GA similar to Fe (minor cage perturbation), whereas Co/Al substitution significantly increases GA at substituted sites and slightly reduces GA at Fe, suggesting altered stability. The ratio of GA at substituted sites to the minimum Fe GA correlates with DFT formation energies, separating elements into two groups (Cu/Zn/Mo vs Al/Ti/Co/Ga) that track stabilization trends.

The results demonstrate that SCANN can jointly achieve competitive predictive accuracy and provide interpretable attributions from local atomic environments to material properties. The GA mechanism aggregates pairwise attentions to quantify the global importance of each local structure in forming the overall material representation, yielding attention maps that align with first-principles observables: orbital localization in molecules and fullerenes, deformation features in Pt/graphene, and stability trends in SmFe12 substitutions. This supports the central hypothesis that attention-weighted local structures encode physically meaningful structure–property relationships. While ALIGNN attains the best raw accuracy by incorporating explicit angular information and higher-order interactions, SCANN balances accuracy with interpretability without heavy reliance on extensive engineered priors. The chemical-accuracy-level results for several QM9 properties indicate practical utility when DFT labels are the reference. The interpretability analyses suggest SCANN can guide hypothesis generation in materials design by highlighting atoms or sites most relevant to target properties.

The paper introduces SCANN, an interpretable attention-based deep architecture for molecules and crystals that learns local structure embeddings and assigns global attention weights to build material-level representations for property prediction. Across QM9, MP, and in-house datasets, SCANN delivers competitive accuracy (meeting chemical accuracy for several QM9 targets) and, crucially, provides GA-derived insights that correspond to DFT orbital distributions, structural deformations, and substitution-driven stability trends. A geometry-enhanced variant (SCANN+) improves accuracy further while increasing model complexity. The approach shows promise for accelerating materials discovery by pinpointing structure features most responsible for properties. Potential future directions include integrating richer geometric descriptors while controlling complexity and bias, extending to larger and more diverse materials datasets, and further validating attention-based interpretability against causal or perturbation-based analyses.

SCANN underperforms ALIGNN on several benchmarks, indicating room for improvement in capturing higher-order geometric effects. Incorporating additional priors (as in SCANN+) increases dimensionality and may introduce bias, potentially hindering interpretability. Attention scores, while informative, do not guarantee causal explanations; care is needed when interpreting attention as explanation. The model and analyses rely on DFT-derived labels, which carry their own approximations, and publication does not report performance on experimental datasets. Some affiliations and dataset details are in the Supplementary; generalizability to out-of-distribution chemistries remains to be fully established.