Rapid and flexible segmentation of electron microscopy data using few-shot machine learning

Material microstructures control functionality in technologies such as catalysts, energy storage devices, and quantum computing architectures. Scanning transmission electron microscopy (STEM) enables atomic-scale characterization of structure, chemistry, and defects across materials classes and has been central to understanding features from dislocation networks to secondary phases and point defects. Conventional STEM image analysis relies on manual or semi-automatic expert-driven workflows that are time-intensive, potentially biased, error-prone, and difficult to scale to large, multimodal data volumes or high-speed in situ/correlative studies, contributing to reproducibility challenges. These issues are acute in complex oxides, where properties are sensitive to even trace defects, motivating new methods that provide accurate, rapid, and statistically robust microstructural characterization. A key challenge is the diversity of microstructural features and data modalities; extracting quantitative, semantically meaningful descriptors (e.g., area fractions, feature abundances via segmentation) is central to linking to physical models. Traditional segmentation methods (e.g., Otsu, watershed, k-means) often lack generalizability across materials and image types and may require extensive, tailored preprocessing. While machine learning, particularly CNN-based approaches, has shown promise for microstructural classification and pixel-wise segmentation, practical deployment is hindered by large labeled training data requirements and limited generalization across diverse data.

Existing image segmentation techniques used in microscopy include global and adaptive thresholding (e.g., Otsu, multi-Otsu, Yen), watershed segmentation, and clustering methods such as k-means. These approaches typically partition images based on pixel intensity distributions and often incorporate blurring or morphological operations to clean segmentations, but they generally separate foreground/background and can miss distinctions between microstructural classes. Their outputs are inherently intensity-based rather than leveraging size or shape, limiting their ability to distinguish different micrograph structures. Recent applications of machine learning, including CNNs, have advanced both image-level and pixel-level classification in materials science; however, these methods commonly require large labeled datasets and may not generalize well across varied materials systems and imaging modalities, posing challenges for practical, rapid STEM data analysis.

The study presents a flexible, semi-supervised few-shot machine learning approach for segmenting STEM images across three oxide systems: SrTiO3/Ge epitaxial heterostructures, La0.8Sr0.2FeO3 thin films, and MoO3 nanoparticles. The few-shot framework leverages limited labeled examples (prototypes) to classify pixels, enabling rapid reconfiguration to new feature classes with minimal annotation. The authors benchmark the approach against common analysis categories: (1) thresholding methods (e.g., Otsu, adaptive mean, adaptive Gaussian, multi-Otsu, Yen), which primarily separate background from foreground; (2) segmentation methods capable of multiple intensity-based classes, often combined with blurring and morphological operations; and (3) clustering approaches based on image properties, including k-nearest neighbors (KNN), structural similarity index measure (SSIM)-based clustering, and the prototype-driven few-shot method. Figure-based comparisons illustrate that conventional methods tend to produce intensity-based partitions, whereas the few-shot method uses prototypes to better capture semantically distinct microstructural classes. The semi-supervised nature reduces labeled data requirements and improves robustness to noise and variability across datasets.

• The proposed semi-supervised few-shot segmentation method is more robust to noise, more easily reconfigurable, and requires less labeled data than conventional image analysis and segmentation methods.
• Traditional thresholding and segmentation techniques primarily separate images into foreground/background or intensity-based classes, often missing distinctions between different micrograph structures; they inherently classify based on intensity rather than size or shape.
• Clustering baselines such as KNN and SSIM group pixels by centroid or structural similarity thresholds, whereas the few-shot approach classifies using learned prototypes, facilitating more semantically meaningful segmentation of STEM images across multiple oxide systems.
• The approach enables rapid image classification and microstructural feature mapping, supporting high-throughput characterization and integration with autonomous microscopy platforms.

By reducing labeled data requirements and improving robustness to noise and variability, the few-shot semi-supervised segmentation approach addresses key barriers in STEM image analysis: scalability, generalization across diverse microstructures, and speed. Unlike intensity-focused traditional methods, the prototype-driven classification yields segmentations that better align with meaningful microstructural classes, enhancing the extraction of quantitative descriptors (e.g., phase fractions, feature abundances). These advances directly support high-throughput and autonomous workflows, enabling more consistent, reproducible, and statistically rigorous characterization across different oxide materials and imaging conditions.

The work introduces a rapid, flexible semi-supervised few-shot machine learning framework for segmenting STEM images across multiple oxide systems. It demonstrates superior noise robustness, ease of reconfiguration, and reduced data requirements compared to conventional segmentation and clustering techniques, enabling faster, more generalizable microstructural feature mapping. The approach is positioned to accelerate high-throughput characterization and integration with autonomous microscopy platforms.