Graph neural networks (GNNs) are powerful tools for modeling complex systems by representing data as graphs where nodes represent features and edges represent their relationships. GNNs have diverse applications, including social network analysis, brain structure modeling, gene regulatory network analysis, and material property prediction. In quantum chemistry, GNNs offer a faster alternative to computationally expensive methods like solving the Schrödinger equation for predicting material properties. Existing GNN architectures for material property prediction, such as SchNet, CGCNN, MEGNet, and ICGCNN, represent materials as graphs with atoms as nodes and bonds as edges, using elemental properties as node features and interatomic distances as edge features. While these models can implicitly represent many-body interactions, they often lack the explicit inclusion of bond angles, which are crucial for many material properties, particularly electronic properties like band gaps. The explicit inclusion of angle-based information has been shown to improve model performance in other methods. This work aims to develop a GNN architecture that efficiently incorporates bond angle information to improve prediction accuracy.

The paper reviews existing GNN architectures for material property prediction, highlighting their limitations in explicitly incorporating bond angle information. Models like SchNet, CGCNN, MEGNet, and ICGCNN are mentioned, emphasizing their reliance on atomic distances and elemental properties as features. The authors note that many important material properties are sensitive to bond angles and local geometric distortions, suggesting that explicitly including this information could significantly improve prediction accuracy. They cite previous work showing that adding angle-based information improves models with handcrafted features. The growing interest in explicitly incorporating bond angles and other many-body features in GNNs is also highlighted, setting the stage for the ALIGNN model's introduction.

ALIGNN uses a line graph neural network approach to incorporate bond angle information. The line graph L(g) is derived from a graph g; in this case, the nodes of g represent atoms, the edges represent bonds, the nodes of L(g) represent bonds, and the edges of L(g) represent bond angles. The model alternates between message passing on g and L(g), propagating information between bond angles and atom representations. Initial node features in the atomistic graph are nine elemental properties (electronegativity, group number, covalent radius, valence electrons, first ionization energy, electron affinity, block, and atomic volume). Initial edge features in the atomistic graph are interatomic bond distances represented by a radial basis function (RBF) expansion. Initial edge features in the line graph are RBF expansions of bond angle cosines. ALIGNN utilizes edge-gated graph convolution for updating both node and edge features in both graphs. One ALIGNN layer consists of an edge-gated graph convolution on g followed by one on L(g). The overall architecture uses N layers of ALIGNN updates followed by M layers of edge-gated graph convolution updates on g. Sigmoid Linear Unit (SiLU) activations are used. Global average pooling is performed over nodes, and a fully connected layer predicts the target properties. The model is implemented using PyTorch and DGL. The hyperparameters (ALIGNN layers, GCN layers, input features, embedding features, hidden features, normalization, batch size, and learning rate) are detailed in Table 1. The models were trained on datasets from the Materials Project, JARVIS-DFT, and QM9, using different train-validation-test splits for each dataset. For regression tasks, mean squared error (MSE) loss was minimized, and for classification tasks, negative log likelihood loss was minimized. The AdamW optimizer with a one-cycle learning rate policy was used.

ALIGNN demonstrates improved performance compared to existing GNN models and classical force-field inspired descriptors (CFID) across multiple datasets and properties. On the Materials Project dataset, ALIGNN achieves lower mean absolute errors (MAEs) for formation energy and band gap compared to CGCNN, MEGNet, and SchNet (Table 2). Similarly, on the JARVIS-DFT dataset (Table 3), ALIGNN significantly outperforms CFID and CGCNN for various properties, including formation energy, band gap, bulk and shear modulus, magnetic moment, and Seebeck coefficient. The superior performance is further highlighted by the MAD:MAE ratio, where higher ratios indicate better predictive performance. ALIGNN also shows competitive performance on the QM9 molecular dataset (Table 5), outperforming competing models for some properties. In classification tasks (Table 4), ALIGNN achieves high ROC AUC scores for various material properties, indicating its effectiveness for material classification. Ablation studies (Table 6 and Figure 3) reveal the importance of both ALIGNN and GCN layers, with a balance between the two resulting in optimal performance. Increasing hidden features, edge input features, and embedding features beyond certain thresholds provides diminishing returns. The learning curve analysis (Figure 4) shows no sign of diminishing returns with increasing dataset size. While ALIGNN exhibits slightly higher computational cost per epoch compared to some competing models, its faster convergence allows for lower overall training cost and comparable or better accuracy.

The results demonstrate that ALIGNN's explicit inclusion of bond angle information significantly improves the accuracy of material property predictions compared to GNN models that rely solely on atomic distance information. The model's superior performance across diverse datasets and a wide range of material properties highlights its robustness and generalizability. The ablation study confirms the importance of the line graph component in capturing crucial angular information and achieving better generalization. The findings have implications for materials science, where understanding and predicting material properties based on atomic structure is essential for designing and discovering new materials. The faster convergence of ALIGNN compared to other models, despite its increased computational cost per epoch, results in lower overall training time, making it a practical and efficient model for large-scale material property prediction. The superior performance suggests that future GNN architectures for material property prediction should incorporate angle information explicitly.

The ALIGNN model presented in this paper offers a significant advancement in GNN-based material property prediction. By explicitly incorporating bond angle information through a novel line graph architecture, ALIGNN achieves superior accuracy compared to existing models across diverse datasets and properties. The model's efficiency and generalizability make it a valuable tool for materials discovery and design. Future research could focus on exploring even more complex graph representations, incorporating additional many-body interactions, and applying ALIGNN to other materials and properties.

While ALIGNN demonstrates significant improvements, certain limitations exist. The model's computational cost per epoch is slightly higher than some competing models, although its faster convergence mitigates this to some degree. The performance on some electronic properties in the JARVIS-DFT dataset shows room for improvement. The study primarily focuses on DFT-calculated properties; further evaluation with experimental data is needed to fully assess the model's predictive power. Dataset bias could influence the model's performance; careful consideration of dataset composition and potential biases in future studies is warranted.