The field of quantum many-body physics is experiencing a surge in the application of machine learning (ML) techniques, particularly for solving complex inverse problems. One such application involves analyzing image-like data from experimental techniques such as STM. Van der Waals moiré superlattices are particularly well-suited for this ML-assisted analysis due to their unique characteristics: (i) they exhibit a wide range of correlated quantum many-body phenomena, including interaction-induced insulating phases, magnetism, superconductivity, and electronic nematic order; (ii) they are highly tunable, allowing for the generation of large datasets from a single sample; and (iii) their large moiré unit cells enhance the spatial resolution of STM, providing access to intra-unit-cell physics. This study focuses on electronic nematic order in TDBG, a moiré system exhibiting spontaneous rotational symmetry breaking at certain electron concentrations, as observed in previous STM experiments. While some simple limiting cases have been compared to data, a systematic analysis of the microscopic form of nematicity remains lacking. This work aims to address this gap by considering a more general case including leading terms describing nematic order and strain in a continuum-model description of TDBG. The goal is to reconstruct the parameters of this Hamiltonian from STM data using CNNs in a supervised learning procedure, thereby extracting underlying microscopic physics.

Recent research has highlighted the potential of machine learning in quantum many-body physics, particularly in addressing inverse problems. Several studies have explored ML-assisted analysis of experimental data from imaging techniques like STM, photoemission, and others. The application of ML algorithms to data from imaging techniques like STM has shown promise in various fields, particularly with Van der Waals moiré superlattices. These systems display a variety of correlated quantum many-body phenomena, such as interaction-induced insulating phases, magnetism, superconductivity, and electronic nematic order. Despite significant research on these phenomena, understanding their origin and relationships remains challenging. Moiré superlattices offer advantages due to their tunability and large unit cells, making them ideal for data-driven approaches. Previous works have focused on detecting the presence or absence of nematic order or performed phenomenological data analysis of STM measurements using ML. In contrast, this study focuses on extracting the underlying microscopic physics of nematicity in TDBG.

The study employs CNNs in a supervised learning framework to reconstruct the parameters of a Hamiltonian describing nematic order and strain in TDBG from STM data. The non-interacting band structure of TDBG features valence and conduction flat bands near charge neutrality where correlation-driven phenomena emerge. STM experiments probe the band structure and wave functions through the LDOS, typically studied as a function of either position or energy. Two forms of nematicity are considered: graphene nematicity (GN), a local order parameter, and moiré nematicity (MN), associated with the moiré scale. The Hamiltonian includes terms for both GN and MN, along with strain. A dataset of 12,000 synthetic LDOS images is generated, incorporating Gaussian noise to avoid overfitting and simulate experimental conditions. The images are labeled by their corresponding nematicity parameters. The CNN architecture consists of convolutional layers, batch normalization, max pooling layers, dense layers, and a dropout layer to prevent overfitting. The ADAM optimizer minimizes the mean squared error (MSE) between true and predicted parameters. Initially, the CNN is trained to predict the orientation of the nematic director. Subsequently, the CNN architecture is modified to include multiple input channels representing LDOS maps at different energies and point spectra, to improve the prediction accuracy of the full set of nematicity parameters. Finally, the model's ability to distinguish nematicity from strain is tested by incorporating strain parameters into the dataset and training the CNN to predict both nematic and strain parameters.

The CNN successfully learns to predict the nematic director orientation from LDOS images at a single energy, even when the precise nature of nematicity is unknown. However, predicting the finer details of nematicity parameters (GN, MN, and a phase angle α) requires incorporating LDOS data at multiple energies and point spectra. Including multiple energy channels significantly improves prediction accuracy for all nematic parameters. The CNN demonstrates the ability to distinguish nematicity from heterostrain, accurately predicting nematic parameters even when strain is present. Analysis reveals that outliers in strain angle predictions are associated with small strain intensities, a characteristic feature rather than a methodological limitation. The trained CNN is then applied to experimental STM data, predicting non-zero values for nematicity and strain across different electron fillings. For higher fillings where nematic order is visually apparent, the CNN predicts MN to dominate. The predicted parameters accurately reproduce key features in the experimental data. At lower fillings, where stripe-like features are less pronounced, the agreement between the CNN's prediction and the experimental data is less precise. This suggests a potential crossover from predominantly MN to GN nematicity at lower fillings. Strain remains relatively constant across different fillings, though predicted strain is sometimes higher at low fillings than expected from experimental topography, possibly due to differences between the continuum model and experimental spectroscopy.

This study demonstrates a successful machine learning approach to extract the microscopic form of the nematic order parameter in TDBG from STM data, overcoming the challenge of disentangling nematicity from strain. The inclusion of multiple energy channels in the CNN architecture is crucial for accurate prediction of the full set of nematicity parameters, highlighting the importance of considering energy-dependent correlations in the LDOS. The results demonstrate the ability of the method to accurately predict nematic parameters even in the presence of significant heterostrain, resolving a longstanding debate in the field. The robustness of the method against inhomogeneous disorder further emphasizes its generality and ability to extract microscopic physics. The successful application to experimental data provides valuable insights into the interplay between nematicity and strain in TDBG across different electron fillings. However, some discrepancies between the predicted and experimental results, particularly at low fillings, warrant further investigation.

This work presents a novel machine learning method for extracting the microscopic form of nematic order in TDBG from STM data. The use of multi-channel CNNs incorporating energy-dependent correlations in the LDOS is key to distinguishing nematicity from strain and accurately predicting the nematicity parameters. This methodology is broadly applicable to other systems and types of instabilities, offering a powerful tool for investigating the microscopic physics of quantum materials. Future research could focus on refining theoretical models, exploring other moiré systems, and investigating the effects of different types of disorder.

The study relies on synthetic data generated from a continuum model, which might not perfectly capture all aspects of the real system. The accuracy of the predictions depends on the quality and quantity of the input data, and the presence of noise or artifacts in experimental data could affect the results. While the method shows robustness against some forms of disorder, other types of imperfections might require further investigation. The interpretation of results at lower electron fillings requires further experimental and theoretical work to fully understand the observed discrepancies between the CNN's predictions and the experimental data.