The manipulation and control of nanoscale magnetic spin textures are crucial for next-generation computing. Topologically protected spin textures, like skyrmions, are promising information carriers due to their mobility and stability, potentially leading to applications in neuromorphic and reservoir computing. Materials under investigation include ultra-thin multilayers (e.g., Pt/Co/X) and van der Waals ferromagnets (e.g., Cr2Ge2Te6, Fe3GeTe2, CrI3). Real-space imaging is essential to observe individual spin textures and their responses to external stimuli. Lorentz transmission electron microscopy (LTEM) is a powerful technique for imaging magnetic spin textures, allowing simultaneous observation of magnetic domains and microstructure, and enabling in situ experiments under various stimuli (temperature, magnetic field, electric current). LTEM provides quantitative information about the sample's magnetic induction via the phase of the electron wave (Aharonov-Bohm equation). Phase retrieval is typically done using the transport of intensity equation (TIE), which requires a through-focal series (TFS) of images. However, TIE suffers from reduced resolution and other techniques like off-axis holography or 4D-STEM have limitations for in situ experiments. Existing machine learning approaches also need a TFS. The single image TIE (SITIE) is susceptible to noise and non-magnetic contrast. This work introduces SIPRAD, a method for reconstructing the magnetic phase shift from a single defocused LTEM image, addressing the limitations of existing techniques for in situ experiments.

The literature extensively covers the importance of nanoscale magnetic spin textures for fundamental understanding and applications in novel computing paradigms. Skyrmions, in particular, are highlighted for their topological protection, mobility, and stability, making them attractive candidates for information carriers. Various materials, including ultra-thin multilayers and van der Waals ferromagnets, are being explored. Lorentz transmission electron microscopy (LTEM) is established as a powerful technique for imaging these textures. The existing phase retrieval methods, such as TIE, off-axis holography, and 4D-STEM, and their limitations for in-situ experiments are discussed. The challenges of using SITIE due to its sensitivity to noise and non-magnetic contrast are also highlighted. The paper then introduces the concept of using automatic differentiation (AD) and deep image priors (DIPs) for improved phase reconstruction in LTEM.

SIPRAD uses automatic differentiation (AD) to reconstruct the magnetic phase shift (φm) from a single defocused LTEM image. Unlike previous AD approaches that directly learn the phase shift from a TFS, SIPRAD employs one or two deep image priors (DIPs) for regularization. A DIP is a generative convolutional neural network (CNN) trained to create a desired image, offering noise robustness. In the simplest case (uniform amplitude and φ), a single DIP is used, initialized with an approximate phase shift (obtained via SITIE). The DIP iteratively learns to output an improved φm, minimizing the mean-squared error (MSE) between the simulated and experimental LTEM images. The image formation forward model includes the electron exit wavefunction (ψ(r) = a(r)e^(iφ(r))), where a(r) is the amplitude, and the microscope transfer function (aperture, phase transfer function, damping envelope). For samples with non-uniform amplitude (due to surface contaminants etc.), an optional amplitude reconstruction branch is added, using a second DIP to output an amplitude map, which is then scaled to generate the electrostatic phase shift (φe). Both DIPs are simultaneously optimized during backpropagation. The dual-valued constraint is implemented to ensure a physically realistic amplitude map. The methodology includes details about the image formation forward model, incorporating the Aharonov-Bohm equation and the microscope transfer function, emphasizing the separation of electrostatic and magnetic phase shifts. The algorithms were implemented in PyTorch using the Adam optimizer. The input for the phase DIP is a SITIE phase reconstruction of the input image, and the input for the amplitude DIP is a thresholded version of the input image. Each DIP is pre-trained for 200 iterations before the full algorithm is run. The transport of intensity equation (TIE) and SITIE methods are also implemented for comparison, using the open-source PyLorentz software. Micromagnetic simulations of CGT are performed with Mumax3 to generate simulated datasets for testing and evaluation.

SIPRAD demonstrates significantly improved accuracy and robustness to noise compared to SITIE and TIE, especially at larger defocus values and higher noise levels. Figure 3 shows the accuracy of SIPRAD and SITIE for varying defocus and noise levels. SIPRAD maintains high accuracy across a wider range of parameters. SIPRAD accurately reconstructs phase shifts and integrated magnetic induction maps from simulated data with and without added noise (Figure 2). The SIPRAD method accurately determines the location and width of domain walls, which can be important when trying to use domain information to calculate magnetic material parameters. In contrast, SITIE produces inaccurate domain wall locations and widths. Critically, SIPRAD can handle samples with amplitude and electrostatic phase variations caused by heterogeneities or surface contamination (Figure 4). This contrasts with SITIE, which incorrectly interprets non-magnetic contrast as magnetic domains. Even with surface contamination, SIPRAD, using a fixed non-uniform amplitude map, accurately reconstructs magnetic phase shift and integrated magnetic induction from experimental data of exfoliated CGT flake (Figure 5), outperforming SITIE and TIE. In an in situ field-cooling experiment on CGT (Figure 6), SIPRAD successfully tracks magnetic bubble rearrangement and disappearance, highlighting its effectiveness for dynamic imaging. SIPRAD reconstruction time for a 512x512 image is 35 seconds on an NVIDIA A100 GPU.

The superior performance of SIPRAD over SITIE and TIE across various conditions (high defocus, high noise, sample heterogeneity) demonstrates its suitability for real-world LTEM experiments, particularly in situ studies where obtaining a TFS is not feasible. The ability to handle amplitude and electrostatic phase variations is a significant advancement. Although simultaneous reconstruction of amplitude and magnetic phase is computationally more challenging, the method of using a fixed, non-uniform amplitude map provides an effective workaround. SIPRAD's efficiency in processing single images allows for higher time resolution in in situ studies, crucial for observing dynamic processes. The integration of a DIP provides robust regularization without requiring manual parameter optimization. The findings suggest broader applications of SIPRAD in studying a variety of magnetic spin textures in various materials.

SIPRAD, an AI-enabled phase-retrieval method for LTEM, significantly improves the accuracy and robustness of magnetic phase reconstruction from single, noisy, defocused images. Its ability to handle sample heterogeneities makes it particularly useful for in situ experiments. Future work could explore real-time applications by optimizing computational efficiency further and investigating the application to other types of spin textures and materials.

The current implementation of SIPRAD is not real-time, although the processing time is acceptable for post-processing of data. The simultaneous reconstruction of both amplitude and magnetic phase can be challenging at high noise levels, necessitating a fixed amplitude map in some cases. The accuracy of the reconstruction can be affected by the accuracy of the initial SITIE estimate and user input material parameters, such as absorption coefficient. The use of periodic boundary conditions in the forward model can introduce artifacts near image edges.