Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equations

Spintronics, leveraging both spin and charge degrees of freedom, offers functionalities exploited in sensing, memory storage, and emerging applications in communication and information processing. However, predicting the behavior of these devices, often involving intricate magnetic textures and complex dynamics across numerous hidden variables (local magnetizations), presents a significant challenge. Micromagnetic simulations, the dominant approach, are computationally expensive, requiring extensive simulation times and struggle to quantitatively match experimental results due to inherent imperfections and noise in real-world experiments. This necessitates a faster and more adaptable modeling technique. The increasing use of artificial neural networks in physics provides an alternative. Machine learning has been successfully employed for material discovery and learning dynamics from time-series data. While deep neural networks have been used in nanomagnetism to extract features and explore materials, they have not been used to model, fit and forecast the long-term experimental behavior of solid-state nanocomponents. Neural Ordinary Differential Equations (NODEs), which excel at predicting trajectories of dynamical systems, offer promise, but their original form needs modification to overcome two main limitations for application to spintronics: (1) requiring the measurement of all system variables, and (2) the lack of design for dealing with time-varying external inputs. This research proposes a solution to these challenges. The authors modify the NODE formalism to accurately predict the behavior of a non-ideal nanodevice, including noise, using minimal training data, and demonstrate the approach's capabilities on two distinct applications.

The existing literature extensively covers micromagnetic simulations as the primary method for predicting spintronic device behavior. However, the computational cost and inability to accurately fit experimental data are significant drawbacks. While machine learning techniques, such as deep neural networks, have shown promise in related areas like material discovery and feature extraction from magnetic thin films, their application to long-term prediction of experimental solid-state nanocomponent behavior has been limited. Prior work using machine learning techniques in nanomagnetism and micromagnetics has focused on short-term predictions, often neglecting the impact of noise and imperfections which are pervasive in real experiments. Existing NODE methods require measuring the evolution of all system variables and are not designed for time-varying external inputs, presenting significant obstacles to their direct application in spintronic modeling. Therefore, this paper bridges this gap by modifying the NODE framework to overcome these limitations.

The authors address the limitations of applying standard NODEs to spintronic systems in two key ways: 1. **Handling Incomplete System Dynamics:** Standard NODE training requires measurement of all system variables, which is often impractical in experiments where only a single physical quantity (e.g., average magnetization) is typically measured. The authors address this by leveraging the embedding theorem, which reconstructs the state space from a time series of a single measured variable. Specifically, they use time-delayed versions of the measured variable as inputs to the NODE, rather than higher-order derivatives which are more susceptible to noise. This approach allows the NODE to learn the underlying dynamics from the limited available data. They justify their approach mathematically using theorems by Takens and Sauer et al. This approach avoids the noise amplification associated with using higher-order numerical derivatives. 2. **Incorporating Time-Varying Inputs:** NODEs in their standard form lack a mechanism for directly handling external time-varying inputs which commonly influence spintronic devices (e.g., magnetic fields, currents, voltages). The authors extend the NODE framework to include such inputs by augmenting the input vector with time-delayed versions of the external input signals, allowing the NODE to capture the influence of these inputs on system dynamics. The modified NODE framework is trained using a combination of micromagnetic simulations or experimental data. The authors employ a stochastic gradient descent algorithm to optimize the NODE's parameters (synaptic weights and neuron thresholds), minimizing the mean squared error between the predicted and observed trajectories. This training process involves randomly selecting mini-batches of data and updating the NODE's parameters iteratively until convergence. The trained NODE, now capable of predicting the behavior of the system with time-varying inputs and from incomplete knowledge of system dynamics, is then used to predict the system's behavior under novel conditions, including inputs and parameters not seen during training. This methodology is applied to both simulated and experimental datasets.

The authors demonstrate the effectiveness of their modified NODE approach through two key applications: 1. **Skyrmion-based Reservoir Computer:** They modeled a device containing a magnetic skyrmion, a chiral spin texture with potential for applications in memory and computing. The inputs are the perpendicular magnetic anisotropy (PMA) and the Dzyaloshinskii-Moriya interaction (DMI) constants. The output is the average magnetization perpendicular to the thin film. Training data was generated from micromagnetic simulations showing outstanding agreement between the predictions and micromagnetic simulations. The trained NODE accurately predicted the system's breathing frequency for varying PMA and DMI values, outperforming micromagnetic simulations by a factor of over 200 in terms of simulation speed (20 minutes vs 3 days). Further experiments demonstrated the robustness of the model by testing it with different inputs and number of delayed states (k). The higher the k value, the higher the accuracy in noisy situations. The authors concluded that the physical system can be described by two variables in absence of noise. 2. **Experimental Spintronic Nano-oscillator:** They applied their method to experimental data from spin-torque nano-oscillators used as reservoir computers for spoken digit recognition. Remarkably, a training dataset of just 5 milliseconds of experimental data was sufficient to train a NODE capable of accurately predicting the oscillator's response to different inputs, achieving a high accuracy in classification tasks. Furthermore, it was found that the inclusion of noise in the NODE model was crucial for matching the experimental results and for preventing overfitting. The experiment would have taken weeks experimentally and hundreds of years using micromagnetic simulations. In contrast, the trained NODE model required merely two hours. In both applications, the proposed NODE approach significantly outperformed micromagnetic simulations in terms of simulation speed and ability to accurately model noisy experimental data. The ability to train a NODE using a small amount of data, and its ability to generalize to unseen inputs and parameters, highlights the power and efficiency of this method for modeling spintronic devices. This ability was particularly evident in the spoken digit recognition experiment where noise impacted the performance critically.

This research successfully addresses the limitations of using standard NODEs for modeling complex spintronic devices. By incorporating the embedding theorem and a method for handling time-varying inputs, the authors developed a robust and efficient methodology for predicting system behavior from limited data. The results show that the trained NODE models generalize effectively to unseen inputs and parameters, indicating a capacity to learn the underlying physical dynamics. The superior prediction accuracy and significant speed improvement compared to micromagnetic simulations demonstrate the practical applicability and potential of the method for accelerating research and development in spintronics. The ability of the NODE to accurately incorporate and model noise highlights the realistic representation of experimental conditions, a crucial factor often overlooked by traditional micromagnetic simulations. The findings of this study provide valuable insights into the capabilities of NODE-based modeling of physical systems and contribute a novel approach to bridging the gap between theoretical modeling and experimental validation in spintronics.

This work introduces a powerful new technique for modeling complex dynamical systems in spintronics, using NODEs modified to handle limited data and time-varying inputs. The results from both simulated and experimental datasets demonstrate significant speed improvements and increased accuracy compared to existing methods. The successful prediction of complex reservoir computing tasks, such as spoken digit recognition, highlights the practical potential of this approach. Future research could explore the application of stochastic NODE theory to model systems exhibiting intrinsically stochastic behavior, thereby further broadening the applicability of this promising technique.

While the proposed method shows significant advantages, some limitations should be noted. The accuracy of the NODE model relies heavily on the quality and quantity of training data. The choice of the time delay parameter (Δt) might affect the training process and prediction accuracy, although the results were relatively insensitive in the experiments conducted. Additionally, the approach is best suited for systems governed by deterministic equations; further research is required to extend its applicability to systems with strong stochastic components. Finally, the generalization ability of the model might be challenged when significant changes in the physical system architecture or material properties are introduced, requiring further data acquisition to ensure accurate prediction.