Volumetric fluorescence microscopy is crucial for studying cells and organs, but most modalities suffer from anisotropic point spread functions (PSFs), leading to inferior axial resolution compared to lateral resolution. This anisotropy poses significant challenges, particularly in reconstructing complex structures like neuron morphologies in whole-brain imaging, hindering accurate visualization and analysis. While advanced hardware can achieve isotropic resolution, these methods are often complex and limited in applicability. Deep learning offers a potential computational solution, but existing supervised learning approaches require precisely-matched training data which are difficult to obtain. Unsupervised methods, like CycleGAN, offer an alternative but can suffer from artifacts and distortions due to weak constraints. This paper introduces Self-Net, a two-stage deep self-learning approach designed to address these limitations by learning from unpaired lateral and axial images within the same dataset. This self-learning strategy leverages the inherent anisotropy to improve axial resolution while avoiding the need for registered training data or complex physical models. Self-Net uses unsupervised learning for realistic anisotropic degradation and supervised learning for high-fidelity isotropic recovery, thus improving accuracy and suppressing artifacts. The study aims to demonstrate Self-Net's effectiveness and generalizability across different microscopy platforms and its potential to facilitate various biological and neuroscience applications.

Several advanced hardware modalities achieve isotropic 3D imaging, but their complexity limits wide applicability. Deep learning offers a data-driven alternative, with successful applications in overcoming limitations of fluorescence microscopy. Supervised learning approaches are common, but obtaining paired training data for isotropic recovery is challenging. Semi-synthetic methods, like CARE, require accurate PSF estimation, while CycleGAN-based approaches use unpaired data but can suffer from artifacts due to weak constraints. The recently developed OT-CycleGAN addressed this, but training is resource-intensive, and the weak cycle consistency constraint can cause noise and distortion.

Self-Net employs a two-stage self-learning strategy. First, unsupervised learning using a CycleGAN model maps high-resolution (HR) lateral images (downsampled to match axial resolution) to blurred axial images from the same volumetric dataset. This stage learns the anisotropic degradation process. The second stage uses a supervised learning approach (DeblurNet), trained on the synthetically blurred axial images (from stage one) and the corresponding HR lateral images. This stage focuses on high-fidelity isotropic recovery. The two stages are alternately optimized, with DeblurNet's loss guiding the generation of more realistic blurred images in the first stage. During testing, DeblurNet processes the axial slices of the raw image stack to achieve isotropic restoration. The performance of Self-Net was evaluated using simulated fluorescence beads and synthetic tubular structures, comparing it against OT-CycleGAN and a supervised learning approach (CARE). Real biological data acquired from various microscopy platforms (wide-field, two-photon, confocal, light-sheet, STED, and iSIM) were also used to assess Self-Net's generalizability and effectiveness in improving resolution isotropy. Quantitative metrics like RMSE and SSIM, along with Fourier spectrum and FWHM analysis were utilized to evaluate the image quality and isotropy improvement.

Self-Net demonstrated superior isotropic recovery compared to OT-CycleGAN in both simulated and real datasets. Self-Net achieved high-fidelity reconstructions with significantly faster training and inference speeds than OT-CycleGAN (10x fewer parameters, 20x faster training, 5x faster inference). The performance of Self-Net was comparable to the supervised CARE method, even without paired data, showcasing its robustness. Self-Net successfully enhanced axial resolution in various biological samples (mouse liver, kidney, brain vessels, neurons) imaged using different microscopy techniques. Moreover, Self-Net improved the resolution isotropy in super-resolution microscopy data (STED and iSIM), achieving near isotropic resolution in STED data with limited depletion power. The DeepIsoBrain pipeline, utilizing Self-Net, enabled isotropic whole-brain imaging at 0.2 x 0.2 x 0.2 µm³ resolution, significantly improving the efficiency and accuracy of single-neuron morphology reconstruction.

Self-Net addresses the long-standing challenge of resolution anisotropy in fluorescence microscopy. Its self-learning approach eliminates the need for paired training data or complex physical models, making it widely applicable and efficient. The superior performance compared to existing methods, particularly in terms of speed and accuracy, highlights its potential impact on various biological imaging applications. The successful application in super-resolution microscopy demonstrates Self-Net's capacity to push the limits of current technologies. The development of DeepIsoBrain for isotropic whole-brain imaging significantly advances neuroscience research by improving the accuracy and efficiency of single-neuron reconstruction. These findings suggest that Self-Net is a valuable tool for enhancing the quality and interpretability of volumetric fluorescence microscopy data.

Self-Net provides a fast, efficient, and high-fidelity solution for isotropic resolution restoration in volumetric fluorescence microscopy. Its self-learning approach and superior performance make it a valuable tool for various biological imaging applications, especially in neuroscience where high-resolution whole-brain imaging is critical. Future research could explore the application of Self-Net to other imaging modalities and investigate ways to further improve its robustness and efficiency.

While Self-Net demonstrates significant improvement in resolution isotropy, its performance is still dependent on the extent of anisotropy in the input data. For very high levels of anisotropy (where the difference between lateral and axial resolution exceeds four times), performance may degrade. Further investigation is needed to optimize Self-Net for such extreme cases. The training data used were limited in types of samples and microscopy techniques, potentially restricting the generalizability of Self-Net to other biological samples or microscopy platforms. Further testing on diverse datasets is necessary to fully validate its applicability.