Modern scanning transmission electron microscopes (STEM) can focus sub-angstrom electron beams to quantify structure and chemistry from elastic and inelastic scattering processes. Chemical composition is revealed by spectroscopic techniques like energy-dispersive X-rays (EDX) and electron energy loss (EELS) from inelastic interactions. However, high-resolution chemical imaging requires high doses, often exceeding specimen limits, resulting in noisy or incomplete chemical maps. Significant efforts to improve detector hardware have approached the limits set by inelastic processes. Elastically scattered electrons in a high-angle annular dark-field (HAADF) detector provide high signal-to-noise ratio (SNR) for direct interpretation of atomic structure but lack comprehensive chemical information. Achieving the lowest doses at highest SNR demands fusing both elastic and inelastic scattering modalities. Current analysis of detector signals like HAADF and EDX/EELS is performed separately. Correlative imaging ignores shared information, missing opportunities for information recovery. Data fusion, common in satellite imaging, goes beyond correlation by linking signals to reconstruct new information and enhance accuracy. Successful data fusion uses an analytical model representing the relationship between modalities, providing a meaningful combination without artificial connections. Fused multi-modal electron microscopy is introduced to recover high SNR nanomaterial chemistry by linking correlated information within HAADF and EDX/EELS. Chemical maps are recovered by reformulating the inverse problem as a nonlinear optimization, finding solutions accurately matching the actual chemical distribution. This approach significantly improves SNRs for chemical maps (often 300–500%), reducing doses by over an order of magnitude while maintaining consistency with original measurements. The technique is demonstrated on EDX/EELS datasets at sub-nanometer and atomic resolution, allowing for measurement of local stoichiometry with less than 15% error without knowledge of inelastic cross-sections. Convergence and uncertainty are assessed through simulations, providing ground-truth validation.

The introduction thoroughly reviews existing methods in electron microscopy for chemical imaging and their limitations. It highlights the limitations of high-dose requirements for high-resolution chemical imaging using EDX and EELS, the incomplete chemical information provided by HAADF imaging, and the potential benefits of data fusion techniques for improving the signal-to-noise ratio (SNR) and reducing the electron dose. The authors cite relevant research on atomic-resolution chemical mapping using EDX and EELS, direct detection electron energy-loss spectroscopy, and the challenges of quantitative analysis of atomic-resolution X-ray spectral imaging. The review also explains the advantages of data fusion and its applications in other fields, establishing the context and motivation for their proposed method.

Fused multi-modal electron microscopy recovers chemical maps by solving an optimization problem. This problem seeks a solution that strongly correlates with: (1) the high SNR HAADF modality, (2) the chemically sensitive spectroscopic modality (EELS and/or EDX), and (3) encourages sparsity in the gradient domain, reducing spatial variation. The optimization function is: argmin_x≥0 || Σ_i - Σ_i x_i||₂² + λ₁ Σ_i C_i^T (x_i - b^T log(x_i + ε)) + λ₂ Σ_i||∇x_i||_TV (1) The three terms define the multi-modal approach: 1. A forward model assumes the simultaneous HAADF is a linear combination of elemental distributions (x), where the incoherent linear imaging approximation for elastic scattering scales with atomic number as Z^γ (γ typically around 1.7). γ is bounded between 2 for Rutherford scattering and 4/3 for Lenz-Wentzel expressions for electrons in a screened Coulombic potential. 2. Data fidelity is maintained using maximum negative log-likelihood for spectroscopic measurements dominated by low-count Poisson statistics. In a higher count regime, this can be replaced by least-squares error. 3. Channel-wise total variation (TV) regularization enforces a sparse gradient magnitude, reducing noise by promoting image smoothness while preserving sharp features. This sparsity constraint is powerful in recovering structured data. The weights balancing the contributions of these terms are selected to achieve accurate recovery. The optimization problem is solved using gradient descent with total variation regularization, initializing the first iterate with the measured data. The step size is determined using Lipschitz continuity and heuristic methods. Regularization parameters are manually selected; the authors suggest L-curve methods or cross-validation for future automated optimization. Uncertainty in the recovered chemical image is approximated using estimation theory and the model's Hessian, leading to standard error maps. The average standard error provides a metric for reconstruction accuracy. Inelastic scattering simulations for atomic imaging use the abTEM simulation code, which involves propagating the initial STEM probe into the sample, calculating inelastic transition potentials, and propagating the inelastically scattered electrons to the EELS entrance aperture. Elastic signals are calculated with the conventional PRISM method. The computational demands of these simulations are high.

The fused multi-modal electron microscopy technique demonstrates significant improvements in SNR and dose reduction for chemical mapping. The method is validated using both experimental and simulated data at sub-nanometer and atomic resolution. **High-SNR Recovery:** The technique significantly improves the SNR of EDX and EELS signals, leading to clearer and more accurate chemical maps of heterogeneous nanomaterials. The study shows a demonstration on cobalt sulfide (CoS) nanocatalysts, Co3xMnO4 supercapacitor nanoparticles, and copper-sulfur heterostructured nanocrystals. The results demonstrate a significant reduction in noise and an improvement in the quality of chemical maps. **Atomic-Scale Resolution:** The fused multi-modal approach successfully maps chemical structures at atomic length scales, revealing detailed information about the elemental distribution and interfaces within the nanoparticles. The method accurately recovers the core-shell structure of Co3xMnO4 nanoparticles and the atomically sharp interface between ZnS and Cu0.64S0.36 in a heterostructure. **Quantitative Chemical Concentration:** The technique provides stoichiometrically meaningful chemical maps without specific knowledge of inelastic cross-sections. The ratio of pixel values in the reconstructed maps quantifies elemental concentration. The results agree with the expected stoichiometry of materials such as NiO, ZrO2, and a synthetic gallium oxide crystal. The study reports a stoichiometric error of <15%. **Dose Requirements:** The study investigates the effect of electron dose on the accuracy of the reconstruction. The results show that accurate chemical maps can be recovered even at low doses (HAADF SNR ≥ 4 and chemical SNR ≥ 2). A correlation between RMSE and average standard error is observed, confirming the accuracy of the estimations. **Simulations:** Inelastic multislice simulations of an FePt nanoparticle are used to provide ground truth solutions and validate recovery at atomic resolution under multiple scattering conditions. The recovered chemical distributions from the simulations match the original images, showing the efficacy of the technique. The simulations also highlight the computational demands of this approach, emphasizing the efficiency of the PRISM STEM-EELS approximation used for speeding up the computations.

The fused multi-modal electron microscopy technique addresses the limitations of traditional electron microscopy methods for chemical imaging by effectively combining elastic and inelastic scattering signals. The significant improvement in SNR and reduction in dose requirements offer advantages for imaging radiation-sensitive materials. The quantitative accuracy in determining chemical concentrations, even without knowledge of inelastic cross-sections, is a significant advancement. The close correlation between RMSE and average standard error indicates the reliability of the reconstruction accuracy estimates. However, the technique's success is dependent on careful selection of parameters and preprocessing of spectroscopic data. Limitations exist in recovering chemical maps for elements with insignificant contrast in HAADF, particularly low-Z elements in the presence of heavy elements. Future directions include incorporating additional elastic imaging modes (e.g., ABF) to improve visualization of light elements and automated optimization of hyperparameters. The results suggest a potential pathway for future atomic characterization of matter by fusing all scattered and emitted signals in an electron microscope.

The study presents a model-driven data fusion algorithm significantly improving electron microscopy spectroscopic maps at nanometer to atomic resolutions. By fusing elastic and inelastic signals, this method enhances the SNR and enables low-dose chemical imaging of radiation-sensitive materials. Quantitative accuracy in determining chemical concentration is demonstrated across various materials. While the method shows robustness, careful parameter selection and preprocessing are crucial. Future work could focus on incorporating additional modalities and automating hyperparameter optimization to further enhance the technique's capabilities.

The accuracy of fused multi-modal electron microscopy depends on the careful selection of hyperparameters. Incorrect selection can lead to variations in stoichiometry estimates. The technique may struggle with elements showing insignificant contrast in the HAADF modality, especially low-Z elements in the presence of heavy elements. Preprocessing of spectroscopic data, including proper background subtraction and peak separation, is critical for accurate results. Spurious atom artifacts can appear in atomic-resolution reconstructions but are manageable through downsampling or using lower-resolution reconstructions as initial guesses.