Artificial-intelligence-driven scanning probe microscopy

Scanning probe microscopy (SPM), encompassing techniques like scanning tunneling microscopy (STM) and atomic force microscopy (AFM), has revolutionized materials science, nanoscience, chemistry, and biology. Its ability to map surface properties and manipulate surfaces with atomic precision is unparalleled. However, SPM's success hinges on constant human intervention. The process involves selecting suitable scanning regions, assessing image quality, and manually conditioning the probe—a trial-and-error process based on the operator's experience. This human dependency severely limits the throughput and applicability of SPM, particularly for large-scale data acquisition and complex nanoscale manipulations. The low yield of usable data—due to probe morphology artifacts and the state of the sample region—further exacerbates this issue. Previous attempts to automate SPM have focused on scripted operations or automated region selection but have not addressed the critical issue of dynamic probe quality management under varying experimental conditions. Machine learning (ML), particularly deep learning techniques, offers a promising approach to overcome these limitations. While ML has been previously applied to assist human operators in specific SPM tasks, fully autonomous operation in diverse scenarios with varying probe defects and ill-defined conditioning protocols remains a challenge. This research aims to address this gap by developing DeepSPM, a fully autonomous AI-driven SPM system capable of continuous and reliable data acquisition.

Existing literature reveals several strategies aimed at improving SPM efficiency. Analytical simulations have been used to link probe morphology and image quality, while inverse imaging techniques attempt to characterize the probe using sample features. Probe manipulation techniques like field ion microscopy have also been explored. However, these methods are often difficult to implement broadly, especially for large datasets. Efforts toward SPM automation include scripted operation and automatic region selection, but these often fall short in handling dynamic probe quality issues. Recent studies have incorporated ML to improve specific aspects of SPM, such as detecting and repairing specific probe defects or assessing image quality from a limited number of scan lines. However, achieving fully autonomous operation across varied probe defects and without pre-defined conditioning protocols has remained elusive. This study builds upon these existing efforts, combining several ML techniques to create a robust, general-purpose autonomous SPM system.

DeepSPM, the autonomous SPM system developed in this research, consists of three core components: (1) algorithmic sample region selection and measurement; (2) a convolutional neural network (CNN) classifier trained via supervised learning to assess probe state; and (3) a deep reinforcement learning (RL) agent to repair the probe through appropriate conditioning actions. The system functions as a closed-loop control system. Initially, an algorithmic approach selects a suitable scanning region. Subsequently, an image is acquired, and the CNN classifier evaluates its quality. If deemed good, the image is processed and stored, and the system proceeds to the next iteration. However, if the image is classified as bad, DeepSPM identifies the cause (e.g., lost contact, probe crash, bad sample region, or bad probe) and takes corrective action. Lost contact or probe crashes trigger re-establishment of contact in a new region. Bad sample regions are identified algorithmically based on height variations. If the probe is identified as faulty, the RL agent initiates probe conditioning. The RL agent uses a second CNN (action CNN) to select actions from a pre-defined list of 12 common conditioning actions (voltage pulses or probe dips). The outcome of each action is evaluated by the classifier CNN, providing feedback to the RL agent through a reward system. The RL agent learns to maximize cumulative reward, thereby minimizing conditioning steps. The RL agent's performance was benchmarked against random action selection, demonstrating significant improvements in probe conditioning efficiency. The entire DeepSPM system was tested over an 86-hour period, where it autonomously managed probe quality, identified and avoided bad sample regions, and acquired over 16,000 images. The sample used for training and testing was magnesium phthalocyanine (MgPc) molecules adsorbed on a silver surface, imaged using a low-temperature STM with a metallic Pt/Ir probe. The data set used for training the classifier CNN consisted of 7589 images of MgPc molecules on Ag(100), with 25% labelled as "good probe" and 75% labelled as "bad probe". Data augmentation was used to improve the performance of the CNN classifier.

DeepSPM's autonomous operation demonstrated significant improvements in SPM data acquisition efficiency. Over an 86-hour period, the system successfully scanned a 1.2 µm² area, acquiring more than 16,000 images, handling two lost contacts, avoiding 1075 rough regions, and repairing the probe 117 times. A manual inspection of the acquired images revealed that 87% of the images labelled as "good" by DeepSPM were indeed defect-free. Approximately 86% of the conditioning episodes initiated by DeepSPM were determined to be necessary. The RL agent consistently outperformed random action selection in probe conditioning, reducing the average number of conditioning steps by ~28% during testing and ~34% during autonomous operation. This superior performance underscores the ability of the RL agent to intelligently select actions that improve probe quality, even without direct control over the atomic-scale probe structure. The fact that DeepSPM continued to improve its probe-conditioning strategy during the 86-hour autonomous operation highlights the advantage of continuous training within the operational workflow. The classifier CNN demonstrated high accuracy in distinguishing between "good" and "bad" probe images in a test data set, with a threshold of 0.9 for classification. This accuracy was maintained during the long-term autonomous testing phase. The method of finding the next imaging region employed in DeepSPM effectively minimized probe travel distance, thereby minimizing the effect of piezo creep and ensuring efficient use of the scan area. The pre-processing of STM images using RANSAC effectively removed background gradients, improving image quality for analysis.

DeepSPM successfully addresses the long-standing challenge of fully autonomous SPM operation. The system's ability to continuously acquire high-quality data without human intervention significantly enhances the efficiency and scalability of SPM experiments. The RL agent's superior performance compared to random action selection demonstrates the feasibility of using ML to optimize complex experimental protocols. The findings highlight the potential of ML to address the complexities of SPM, where precise analytical models are often lacking. The success of DeepSPM in handling diverse probe defects and the absence of pre-defined conditioning procedures expands the applicability of the method to a wide range of SPM techniques and sample types. The system is not only efficient but also robust, handling various issues such as lost contact and sample roughness. The 86-hour autonomous operation exemplifies the practicality of the system for real-world applications. This work demonstrates that advanced AI can be leveraged to overcome current limitations in SPM operation. This has a profound impact on the future direction of nanoscale research and the related fields such as nanofabrication.

This research presents DeepSPM, a fully autonomous AI-driven system for scanning probe microscopy. DeepSPM successfully demonstrates continuous, high-quality data acquisition over extended periods, significantly improving the efficiency and scalability of SPM experiments. The integration of a deep reinforcement learning agent for intelligent probe conditioning represents a significant advance in SPM automation. Future research could focus on integrating DeepSPM with other ML approaches for identifying adverse conditions and regions of interest, enhancing its capabilities further. The public availability of the source code makes DeepSPM readily adaptable and generalizable to various SPM techniques and sample systems, paving the way for high-throughput atomically precise nanofabrication.

While DeepSPM demonstrates significant improvements in autonomous SPM operation, some limitations exist. The RL agent's effectiveness depends on the availability of a sufficiently diverse training dataset to capture the spectrum of probe states and conditioning actions. The accuracy of the classifier CNN and the efficiency of the RL agent could be further improved with larger and more diverse training datasets. The current implementation focuses on constant-current STM imaging. Adapting the system to other imaging modes might require retraining the CNN classifiers and the RL agent. The performance of DeepSPM could be sensitive to the specific characteristics of the STM setup, requiring potential adjustments for different microscope configurations.