Transferring chemical and energetic knowledge between molecular systems with machine learning

Molecular simulations are fundamental tools in various applied sciences, offering accurate predictions of system properties. The integration of machine learning (ML) techniques holds significant promise for enhancing these simulations, particularly in addressing the challenge of limited sampling capabilities in large, complex systems like proteins and DNA. While ML has been applied to predict atomistic properties and identify free-energy states, the sheer volume of data and the computational cost of sampling the system's phase space remain significant hurdles. This problem is especially pronounced in biologically relevant macromolecules where the long timescales involved hinder the ability of even advanced simulation techniques to study systems under realistic conditions. For example, accurately predicting drug efficacy, which depends on ligand-target binding kinetics (drug residence time), remains a significant challenge. The free-energy landscapes of such interactions are often complex, characterized by high barriers separating metastable states, trapping simulations in limited regions of the energy landscape for extended periods. Enhanced sampling techniques and coarse-grained representations have improved sampling, but are still insufficient for many real-world scenarios. To overcome these limitations, this research explores transfer learning, a strategy that leverages knowledge gained from simpler systems to predict properties of more complex systems. This paper proposes a novel methodology based on transfer learning to predict the free energy of a complex molecular system by using data from a simpler, computationally affordable system. The methodology involves a new hypergraph representation of molecules that accurately encodes multi-atom interactions, and a novel neural network model that processes these hypergraphs to predict free energy. This addresses the limitations of existing neural network methods for processing hypergraphs, which often assume scalar features for hyperedges and lack pooling mechanisms for variable-size inputs.

Machine learning has seen increasing application in molecular simulations, with several studies focusing on predicting atomistic properties and identifying free-energy states. Various ML techniques have been employed, often leveraging first-principles calculations or quantum mechanics. However, many of these studies are limited by the availability of data, particularly for large, complex systems. The use of transfer learning in molecular simulations remains relatively unexplored, with limited applications mainly in approximating quantum mechanical calculations or inferring material and molecular properties. Existing methods based on neural networks for processing hypergraphs and simplicial complexes often have restrictions, such as assuming scalar features for hyperedges and lacking pooling mechanisms for variable-size inputs. These limitations motivated the development of the novel approach presented in this paper, which addresses these issues and enables the transfer of knowledge between molecular systems of vastly different sizes and complexities.

This study employs a novel hypergraph representation of molecules, addressing the limitations of traditional graph-based representations that only model pairwise interactions. The hypergraph representation encodes higher-order interactions (two, three, and four atoms), accurately capturing the information needed to describe the potential energy of a molecular system. The potential energy (Ep) is classically modeled as the sum of bond energy (Ebond), non-bond energy (Enon-bond), angle energy (Eangle), and dihedral energy (Edihedral). The hypergraph representation uses vertices to represent atoms and hyperedges to represent these various types of interactions. A hyperedge feature set of size five encodes the type of interaction and its value, while a vertex feature set of size two includes the mass and radius of the corresponding atom. This representation allows the model to capture the complex interplay of multi-atom interactions within a molecule. The free-energy calculation is based on metadynamics simulations, which utilize collective variables to explore the conformational space of the molecule. The free-energy surface F(s) is then computed using Equation (2), where s(x) represents a set of collective variables. A novel hypergraph neural network (HNN) model was developed to process the hypergraph representations and predict free-energy values. The HNN model comprises message-passing layers and a pooling layer designed to handle variable-size hypergraphs. The message-passing layers iteratively update the representation of nodes and hyperedges based on their interactions, while the pooling layer aggregates information from variable-size conformations into a fixed-size feature vector. Transfer learning is performed by training the HNN on a simpler molecular system and then applying it to a more complex system without additional training data from the target system. The performance of the HNN model is evaluated using the Area Under the Curve (AUC) of the receiver operating characteristic (ROC) curve, assessing the model's ability to distinguish between high and low free-energy conformations. In addition to supervised classification, an unsupervised approach is employed to cluster conformations based on the HNN's predicted free-energy values, allowing for the recognition of secondary structures. Statistical tests (Wilcoxon signed rank test) are used to assess the similarity of free-energy predictions between different clusters.

The research demonstrated the effectiveness of the proposed HNN model in transferring knowledge between molecular systems. In the first experiment, transfer learning from alanine dipeptide to tri-alanine yielded an AUC of 0.89, indicating good performance in classifying low and high free-energy conformations. More significantly, transfer learning from tri-alanine to deca-alanine achieved a remarkable AUC of 0.92. This is a challenging task because deca-alanine can form secondary structures not present in tri-alanine (or alanine dipeptide), and the significantly higher structural complexity of deca-alanine makes free-energy prediction more difficult. This result clearly shows that the model can successfully transfer knowledge from a smaller, simpler system to a larger, more complex system to predict the free-energy of the latter. Furthermore, the unsupervised application of the HNN model to deca-alanine successfully grouped conformations into clusters based on similar free-energy values. The model correctly identified clusters representing different secondary structures (helix, hairpin, extended), even distinguishing subtle structural differences that are difficult to capture with standard methods like RMSD. While the model shows a remarkable ability in classifying high and low free energy conformations, its performance in predicting exact free-energy values in a regression setting was not equally good. A detailed analysis of outliers, such as the discrepancies observed in the predictions for clusters 2, 4, and 9 of the deca-alanine system, highlights the sensitivity of the model to subtle structural differences. The model's ability to discriminate these subtle variations, despite the overall excellent performance, suggests potential avenues for future improvements.

The findings of this study address the significant challenge of predicting free-energy landscapes for complex molecular systems, a problem that hampers progress in various fields, including drug discovery. The proposed methodology significantly reduces the computational cost and time required for such predictions by leveraging transfer learning. The ability of the HNN model to accurately classify low and high free-energy conformations, even in systems exhibiting different secondary structures, indicates a significant advancement in applying machine learning to molecular simulations. The success of the unsupervised approach in identifying secondary structures based on free-energy predictions demonstrates the model's capacity to extract meaningful information from complex systems with limited prior knowledge. The results have implications for accelerating molecular simulations and enabling investigations of larger, more complex systems that were previously intractable. Future work should focus on improving the model's performance in free-energy regression and extending its applicability to even more complex systems such as protein-ligand complexes.

This research provides a proof of concept demonstrating the effectiveness of hypergraph-based neural networks for predicting energetic properties of molecular systems. The successful transfer learning from simpler to more complex systems, achieving high accuracy in classification, opens up new avenues for studying complex biological systems. Future work will concentrate on improving the model for regression tasks, expanding its application to diverse systems, and integrating it into molecular dynamics simulation workflows as a runtime plugin or post-processing tool.

While the HNN model demonstrated remarkable performance in classifying low and high free-energy conformations, its predictive power in terms of exact free-energy values remains to be improved. The model's current architecture and training methodology may not be universally applicable to all types of molecular systems. Further investigation is needed to determine the model's performance on systems with greater diversity in amino acid composition and more complex tertiary structures. The accuracy of the free-energy predictions depends on the quality of the data used for training. Bias in the training dataset could affect the model's performance and generalizability.