Topaz-Denoise: general deep denoising models for cryoEM and cryoET

Cryo-electron microscopy (cryoEM) is a crucial technique for resolving protein structures. However, the low SNR in cryoEM images significantly impacts the accuracy and efficiency of structure determination. Low SNR hinders several steps, including particle orientation identification. Improving cryoEM image denoising not only enhances downstream analysis but also speeds up data collection by allowing for lower electron doses. Existing methods like downsampling, bandpass filtering, and Wiener filtering have limitations, particularly when dealing with small or non-globular proteins. The field is also pushing towards high-throughput cryoEM, requiring increased throughput. Collecting shorter micrograph exposures could increase speed, but this reduces SNR. Therefore, advanced denoising is crucial for maintaining image quality at lower doses and higher throughput. While deep neural networks have shown promise in image denoising, they typically need ground truth images, which are unavailable in cryoEM. The Noise2Noise framework addresses this by using paired noisy images for training. Although some neural network-based denoising tools have emerged for cryoEM, systematic evaluations and pre-trained general models have been lacking. This paper aims to address these gaps by developing Topaz-Denoise, a large-scale, publicly available denoising method for cryoEM and cryoET.

Image denoising has been a focus of the computer vision and signal processing communities. Advances in deep learning have significantly improved image restoration and inpainting. However, these methods generally require ground truth images for training. The Noise2Noise framework provided a significant advancement by enabling denoising model training from paired noisy images, bypassing the need for ground truth. Several subsequent methods have built upon this approach. In cryoEM, neural network-based denoising software has recently emerged, but lacked systematic evaluations and pre-trained models that generalize across different datasets. Topaz-Denoise aims to address these shortcomings by providing such a general model trained on a large and diverse dataset.

Topaz-Denoise utilizes the Noise2Noise framework. The key insight is that individual movie frames collected by modern direct detector devices (DDDs) are independent observations of the same underlying signal. These paired frames serve as the training data. The model was trained on thousands of micrographs from DDDs (K2, Falcon II, Falcon III) collected under diverse imaging conditions. This allows the model to learn the complex image formation process in cryoEM without making specific assumptions about the noise characteristics. The authors trained several model architectures, including affine, FCNN, and U-net models, with both L1 and L2 loss functions. The U-net architecture, with its skip connections, proved particularly effective. For 3D cryoET denoising, the Noise2Noise framework was adapted to 3D, training a model on dozens of aligned cryoET tilt-series. The performance of these models was evaluated using various metrics, including SNR calculations, visual inspection, and downstream analyses such as 3D reconstruction.

Topaz-Denoise significantly improves micrograph interpretability and SNR. The U-net model trained on a large dataset consistently outperforms conventional methods such as low-pass filtering. SNR improvements of over 2 dB on average were observed compared to low-pass filtering, and roughly 20 dB compared to raw micrographs. The model generalizes well across various imaging conditions and camera types, including non-DDD cameras. The application of Topaz-Denoise to the clustered protocadherin dataset revealed more complete particle picking, especially for difficult-to-identify projections (top views), leading to a 2.15x increase in the number of particles picked and the identification of a putative new conformation. Furthermore, Topaz-Denoise enables shorter exposure times (10–25% of typical exposure times) without sacrificing downstream analysis. This corresponds to a substantial reduction in electron dose, allowing for increased throughput and reduced data collection time. The 3D denoising model for cryoET showed similar improvements in SNR and visual quality compared to models trained on individual datasets. The model effectively improved contrast and detail in tomograms while reducing background noise. Quantitative comparisons are provided in tables comparing SNR across different datasets and denoising methods. The improvement in SNR translates to a significant increase in collection efficiency (up to 65% increase on K2 systems), as shown by the experimental measurements on a Titan Krios with Gatan K2 and K3 systems. The authors also demonstrated that the denoised images are unsuitable for ab initio reconstruction, but excellent for assisting particle picking.

Topaz-Denoise addresses several critical bottlenecks in cryoEM and cryoET workflows caused by low SNR. The improved SNR enhances particle identification and picking, leading to more complete datasets for 3D reconstruction. The ability to reduce electron dose while maintaining image quality significantly speeds up data collection. The results with clustered protocadherin illustrate the ability of Topaz-Denoise to reveal new structural information that was previously inaccessible. The generalizability of the models across different datasets and camera types makes them broadly applicable to the cryoEM community. The 3D denoising model for cryoET offers similar benefits, potentially facilitating tomogram segmentation and analysis. The connection between the Noise2Noise objective function and SNR maximization is discussed, providing a theoretical basis for the method's success. The authors acknowledge that neural network denoising models can sometimes produce hallucinations (artifacts that are not present in the original data), highlighting the importance of using denoised images primarily for visualization and particle identification, rather than directly for reconstruction.

Topaz-Denoise offers a powerful and versatile approach for improving cryoEM and cryoET data analysis. The pre-trained general models provide immediate benefits to researchers without the need for additional training. The software's modularity and integration into popular cryoEM packages makes it readily accessible. Future work could explore the application of Topaz-Denoise to other types of microscopy or develop more advanced denoising models to further enhance performance and address potential limitations.

The authors acknowledge that denoised images may exhibit hallucinations or artifacts. They recommend using denoised images for particle identification and visualization, but not directly for reconstruction. The generalizability of the models is empirically shown, however, specific performance might vary across different datasets and instruments, especially those not well represented in the training data. The SNR quantification is based on estimates rather than true ground truth values.