Uncovering material deformations via machine learning combined with four-dimensional scanning transmission electron microscopy

The study addresses the challenge of detecting and mapping unexpected lattice deformations across large sample areas in nanomaterials, where such deformations (strain, dislocations, bending, ripples) critically influence electronic, optical, and catalytic properties. Traditional high-resolution TEM/ADF-STEM provide atomic detail but limited field of view; nanobeam electron diffraction and, more recently, 4D-STEM enabled by fast direct electron detectors (e.g., EMPAD) allow large-area, momentum-resolved mapping. However, conventional 4D-STEM strain and phase mapping often require substantial a priori knowledge (e.g., mask placement), making generalization to diverse samples with unforeseen features difficult. The paper’s purpose is to develop a rapid, semi-automated, unsupervised, data-driven approach—divisive hierarchical clustering of 4D-STEM diffraction data—to automatically reveal multi-scale material deformations without prior structural assumptions, thereby enabling comprehensive structural mapping over entire flakes and samples.

The introduction situates the work within advances in nanomaterial synthesis and characterization: epitaxial heterostructures in III–V semiconductors and 2D materials can generate local strain/dislocations affecting band structure; even single-crystal metallic nanoprisms exhibit deformations that modulate optical/catalytic behavior. 4D-STEM, empowered by direct electron detectors (notably EMPAD), has enabled high-precision strain mapping over micrometer scales, but existing workflows generally require predefined masks and prior structural knowledge. Prior machine learning in microscopy, including unsupervised methods, has been applied to tasks such as identifying stacking order and twin boundaries. The authors aim to extend unsupervised learning to discover deformations and fine structures in 4D-STEM data at multiple scales, overcoming limitations of prior mask-based or supervised approaches.

Overview: The authors implement a divisive (top-down) hierarchical clustering pipeline for 4D-STEM datasets acquired with EMPAD. The workflow comprises preprocessing of diffraction patterns, recursive clustering with automatic selection of the number of clusters, and visualization in real space and low-dimensional manifold space. They also present an alternative view that clusters real-space images derived from individual momentum-space pixels for virtual imaging. Data acquisition: 4D-STEM data were acquired on an aberration-corrected Thermo Fisher Titan Themis. 2D lateral heterojunctions were measured at 80 kV with 1.6 mrad convergence (~3 nm probe). Silver nanoprisms and cross-sectional semiconductors were measured at 300 kV. EMPAD exposure: 1.86 ms per frame (1.0 ms acquisition + 0.86 ms readout). Dose rate ~1e5 e− Å−2 s−1. Scan sizes typically 256×256 (range 64×64 to 512×512). EMPAD calibration: 151 ADU/e− at 80 kV; 579 ADU/e− at 300 kV. Preprocessing of diffraction patterns: - Alignment: Correct drift due to beam tilt by aligning the center of mass (CoM) of the central beam. Apply a circular mask around the center beam; compute CoM with sub-pixel precision; translate the pattern to center. Iterate until the standard deviation of CoM over all scan positions and the error to detector center are both <0.01 pixel. - Masking and feature selection: 1) Ring mask to remove low-angle scattering and zero-padding from alignment. Determine inner/outer radii from radial and rotational standard deviation (STD) profiles to exclude central bright beam/background and noise. 2) Feature selection via STD: In the ring-masked patterns, select pixels with high intensity STD (top ~30%) per pattern; flatten these selected pixels to a feature vector representing each scan position. The 30% threshold was empirically chosen and shown to work for both thin 2D materials and thicker nanoprisms. Clustering architecture: - Divisive hierarchical approach: Start with the full dataset; perform clustering; then recursively sub-cluster each resulting cluster to reveal finer-scale features (composition, orientation, strain, ripples, bending, etc.). - Base clustering method: K-means clustering on feature vectors using squared Euclidean distance. Number of clusters (K) selected automatically via the elbow method by computing within-cluster sum of squares (WSS). For speed in determining K, Mini-Batch K-means is used, as precise WSS is unnecessary. - Mapping and visualization: Cluster labels are mapped back to real space to form color-coded structural maps. Uniform Manifold Approximation and Projection (UMAP) is used solely for visualization by projecting high-dimensional diffraction data into a 3D manifold, aiding interpretation of cluster distributions and variances. Alternative modality—clustering real-space images (virtual imaging): - Re-index 4D data in momentum-major order (kx, ky, x, y). Each momentum-space pixel defines a real-space image over the scan. Hierarchical clustering can be applied to these images to segment diffraction space by similarity of their generated real-space contrasts (equivalent to placing virtual objective apertures). - Preprocessing for real-space images: Normalize image intensities to mitigate dominance by large intensity variations across diffraction space; then perform hierarchical clustering to discover groups corresponding to virtual bright-field (BF), dark-field (DF) for specific lattice orientations, low-angle ADF (LAADF), and thickness-sensitive regions. Method comparisons and parameter choices: - Dimensionality reduction: PCA and UMAP were tested for feature extraction. PCA captures dominant variance but misses minor features unless re-applied per sub-cluster (computationally costly). UMAP as a feature extractor introduced striping artifacts masking real deformations. Therefore, STD-based feature selection was preferred; UMAP retained only for visualization. - Clustering algorithms: Compared Agglomerative, BIRCH, Spectral, Mini-Batch K-means, and K-means using ground truth labels from a manually measured WS2 rotation map. Mini-Batch K-means is fastest but less stable due to random initialization; Spectral clustering best accuracy but slow. K-means chosen for balance of speed and performance in each hierarchical round. Outputs: Real-space cluster maps and 3D UMAP manifolds at each hierarchy level; optional virtual BF/DF/LAADF/thickness images via real-space image clustering.

- The divisive hierarchical unsupervised approach rapidly (<10 minutes typical) identifies multi-scale structural features across entire 4D-STEM datasets without prior sample knowledge. - WS2–WSe2 lateral heterojunction: - Round 1–2: Automatically separated crystalline sample from amorphous substrate and distinguished WS2 from WSe2 due to discrete lattice constant differences; interface mapping revealed defects not apparent in ADF-STEM. - Accuracy for discrete feature (WS2 vs WSe2 based on lattice constant): 99.9%. - Round 3: Within WS2, identified continuous lattice rotations (graded interfaces); histogram of rotation angles confirms continuous distribution. Reported clustering accuracy for this continuous feature: 84%. - Within WSe2, identified four clusters corresponding to ripple-induced lattice tilts; three corner regions showed directional ripples (anisotropic second-order spot intensities), central region flat. UMAP manifold indicated continuous tilt variations, consistent with strain relaxation from lattice mismatch. - WS2–WSe2 superlattices: - Hierarchical clustering resolved flakes (round 1), composition (round 2), directional uniaxial strain, and minor ripples within narrow WSe2 stripes, with ripple aspect ratio ~1/30, demonstrating high sensitivity to subtle deformations. - Silver nanoprisms: - Identified bending contours and separated their two sides, evidenced by intensity variations of conjugate diffraction spots (Bragg condition fulfilled on opposite sides). Virtual dark-field reconstructions reproduced high-contrast contours consistent with DF-TEM literature. Simulations supported thickness uniformity (∼10 nm) and explained intensity differences. - UMAP manifolds showed the nanoprism separated from amorphous background and bending contour forming a thin, asymmetric manifold consistent with continuous deformation. - Virtual imaging via real-space image clustering: - For superlattices, segmented diffraction space into clusters yielding virtual BF and DF images that separated flakes by lattice orientation; also identified a region indicative of thickness variations. - For cross-sectional InGaP/GaAs, segmented reciprocal space into six parts: virtual BF (center beam), LAADF highlighting amorphous carbon capping layer, a higher-angle elastically scattered region reflecting thickness, and three DF images indicating crystalline orientations—two with small tilts (strain in InGaP) and one twin domain. - Performance and robustness: - High accuracy for discrete features (>99%) and reasonable accuracy for continuous features (~84%). - Precision test over 100 K-means runs on a continuous-feature dataset: 88% of runs achieved accuracy >75%, indicating reliability despite random initialization.

The proposed unsupervised, divisive hierarchical clustering directly addresses the need for rapid, large-area discovery of unexpected deformations in crystalline materials. By eschewing a priori masks and model assumptions, the method uncovers both discrete (composition, lattice constant changes) and continuous (rotations, ripples, bending contours) features. Mapping cluster labels back to real space and projecting to low-dimensional manifolds enables intuitive interpretation of spatial distributions and deformation continua. Comparative analyses showed that STD-based feature selection retains minor yet physically meaningful variations that PCA can miss without per-cluster re-computation; UMAP is best used for visualization to avoid artifacts. The method complements precise strain-mapping techniques by providing a quick, semi-automated initial analysis that flags regions of interest (e.g., ripples with aspect ratio ~1/30, bending contours in nanoprisms), guiding subsequent quantitative analysis. Its applicability across 2D heterostructures, 3D nanoprisms, and cross-sectional semiconductors highlights its generality. The demonstrated virtual imaging pathway further broadens utility by reconstructing BF/DF/ADF-like contrasts and disentangling thickness, amorphous scattering, and crystalline orientation contributions.

The study introduces a fast, semi-automated, divisive hierarchical clustering framework for 4D-STEM that discovers multi-scale lattice deformations without prior knowledge. Validated on WS2–WSe2 junctions and superlattices, silver nanoprisms, and InGaP/GaAs cross-sections, the method accurately identifies discrete features (>99% accuracy) and reliably differentiates continuous deformations (~84% accuracy), revealing strain, lattice rotations, ripples, and bending contours. The approach provides immediate, data-driven structural insights, enabling precise follow-up analyses with specialized mapping tools and moving toward autonomous characterization workflows. Future directions include integrating this rapid discovery step with high-accuracy quantitative strain/phase retrieval, improving preprocessing (e.g., beam tilt alignment) to reduce artifacts, refining clustering stability, and extending to broader materials and other large, multidimensional microscopy datasets.

- The clustering-based approach provides rapid discovery rather than highly accurate quantitative deformation maps; continuous features are distinguished with moderate accuracy (~84%), necessitating further processing for precise mapping. - Mini-Batch K-means used for elbow estimation can be unstable due to random initialization; overall clustering outcomes on continuous features show variability (though 88% of runs exceed 75% accuracy). - The 30% STD threshold for feature selection is empirically chosen; while effective across tested systems, it may require tuning for other materials or conditions. - UMAP feature extraction introduced striping artifacts; thus, manifold learning is restricted to visualization. - Preprocessing alignment (center beam CoM) can induce striping artifacts in virtual BF images if beam tilt alignment is imperfect, underscoring sensitivity to acquisition/alignment quality. - The method focuses on diffraction-based features; absolute quantification of strain/tilt still depends on subsequent specialized analyses.