Deep learning enables reference-free isotropic super-resolution for volumetric fluorescence microscopy

Three-dimensional fluorescence microscopy often suffers from anisotropic resolution where axial resolution is worse than lateral, due to diffraction, axial undersampling, and imperfect aberration correction. Even advanced super-resolution modalities (3D-SIM, STED) and light-sheet fluorescence microscopy (LSFM) typically retain poorer axial resolution. Deep learning has been used for image restoration and isotropy but prior methods generally require paired high-resolution targets or fixed degradation models (e.g., PSF-based blurring), which may not match real experimental variability and can limit performance and practicality, especially for large-scale volumetric data. Unsupervised approaches such as optimal transport-driven cycleGANs alleviate paired data needs but still typically rely on separate high-resolution reference volumes under similar conditions, which are difficult to obtain at isotropic resolution. The study aims to develop an unsupervised, reference-free, single-volume training framework that enhances axial resolution to achieve near-isotropic volumetric reconstructions by learning from high-resolution lateral information within the same dataset, thereby reducing data acquisition burden and improving robustness to real-world variability and artifacts.

The paper reviews advancements in tissue clearing and LSFM enabling large-scale 3D visualization, while highlighting persistent anisotropy (axial 2–3× worse than lateral). Prior deep learning methods for isotropic reconstruction include supervised strategies pairing high-resolution lateral images with synthetically blurred axial images (Weigert et al.) and GAN-based super-resolution using a microscope degradation model (Zhang et al.); these approaches depend on accurate priors and fixed degradation models which may not capture fluctuating conditions and diverse samples. Unsupervised cycle-consistent GANs framed via optimal transport have been applied to deconvolution microscopy, estimating blur and reducing artifact generation, with potential simplifications when PSFs are known. However, obtaining unmatched high-resolution 3D references under similar noise/illumination for optimal transport remains challenging, especially for isotropic volumes. Classical deconvolution (e.g., Richardson–Lucy) is used as a baseline but struggles with unmodeled artifacts and space-varying degradations.

Framework: An optimal transport-driven cycle-consistent GAN (OT-cycleGAN) with two 3D generators and 2D discriminators enables reference-free isotropic super-resolution from a single 3D anisotropic volume. - Generators: G (forward path) maps anisotropic input to isotropic-like output (3D deblurring/upsampling). F (backward path) maps isotropic-like output back to anisotropic (blurring/downsampling), emulating the degradation process. - Discriminators: Two groups of 2D discriminators, D_x (forward) and D_y (backward). In the forward path, D_x compares axial maximum intensity projections (MIPs) from G(y) to lateral (XY) 2D images from the real volume y, with randomized projection depths to emulate lateral information from adjacent slices. In the backward path, D_y compares 2D images from F(x) to 2D images from the input y on corresponding orthogonal planes (XY, XZ, YZ). For systems with unequal XZ/YZ quality (e.g., OT-LSM), separate discriminators per axial plane are used and slice sampling replaces projection sampling. - Losses: Least-squares adversarial losses for each plane, summed over XY, YZ, XZ in the forward path to enforce isotropy consistent with lateral resolution, plus backward adversarial losses to match degraded outputs to the input distribution. Cycle-consistency L1 loss between F(G(y)) and y with λ=10 stabilizes training and encourages mutual invertibility. The formulation constrains the solution space via optimal transport between distributions without requiring explicit high-resolution target volumes. - Architectures: G is a 3D U-Net. F is selected empirically per dataset: a deep linear 3D generator (no downsampling) for simulations, CFM brain, and OT-LSM beads; a 3D U-Net for OT-LSM brain images. Discriminators are 2D CNNs (four or six in total depending on whether XZ/YZ share characteristics). - Data and preprocessing: Single 3D image stack per experiment used for both training and testing. OT-LSM volumes were diced into overlapping sub-volumes (e.g., 200^3–250^3 voxels with 20–50 voxel overlap; ~3000 sub-regions for brain, ~580 for beads). CFM volumes (1–2 GB) were loaded whole. Augmentations included random crops and axis flips; for CFM, random rotations about Z. OT-LSM images were median filtered (radius=2) to remove salt-and-pepper noise, normalized to [0,1] with percentile saturation (synthetic 0.25%, CFM 0.03%, OT-LSM 3%), and sheared in YZ to correct 45° imaging geometry. Inference on large volumes used overlapping tiles reassembled into the full volume. - Training details: Kaiming initialization, Adam optimizer with learning rate 1e-4. Example: simulation baseline selected at iteration 11,000, ~19 hours training with 16-bit 148^3 voxels per iteration, ~24 GB GPU memory (RTX 3090). Inference on 700^3 voxels took 3–5 min. Metrics: PSNR, SSIM, MS-SSIM, BRISQUE, SNR. - Datasets/experiments: • Simulation: 900^3 volume with 10,000 tubular structures deformed by elastic fields; axial Gaussian blur σ=4; training/inference on 120^3–144^3 subregions. • Confocal fluorescence microscopy (CFM): Thy1-eYFP mouse cortical region; lateral resolution ~1.24 µm; z-step 3 µm; volume ~1270×930×800 µm^3 resampled to 1 µm isotropic. A 90° rotated acquisition provided a high-resolution lateral proxy for axial reference; registered via BigWarp. • OT-LSM beads: 0.5-µm fluorescent beads; stack ~360×360×160 µm^3 resampled to 0.5 µm isotropic. • OT-LSM brain: Thy1-eYFP mouse cortex ~930×930×8600 µm^3 resampled to 0.5 µm; uncalibrated system exhibiting spherical aberration, doubling artifacts (illumination–detection asynchrony), and motion blur stripes (stage drift). Variation with separate plane discriminators was used. Additional controlled test with calibrated near-isotropic “ground-truth” and synthetic axial blurring (depth-wise Gaussian σ=10) for quantitative evaluation. RL deconvolution (DeconvolutionLab2, 10 iterations) with measured PSF was used as a comparison.

Simulation: - Large-volume metrics (random 700^3 region): PSNR improved from 16.94 dB (input) to 19.03 dB (output), MS-SSIM from 0.70 to 0.86 (~+0.16). Improvements persisted under varying axial blurring levels and z-undersampling rates. - Axial MIP evaluation (47 regions of 700×700 pixels, 15-slice depth): Output metrics approached those of lateral MIPs from a perpendicularly rotated high-resolution reference. - FWHM analysis (n=317 tubular objects): Mean FWHM mismatch versus ground truth reduced from 2.95 px (SD 2.41) to 0.61 px (SD 0.60), ~5× improvement. - Segmentation-based assessments (Otsu, ISODATA, mean-thresholding) showed consistently higher PSNR/SSIM after reconstruction. Fourier spectrum analysis indicated restoration of high-frequency content toward ground-truth/lateral references. CFM (Thy1-eYFP mouse cortex): - Achieved near-isotropic resolution: example cylindrical dendrite FWHM XY 1.71 µm; XZ improved from 6.04 µm (input) to 1.76 µm (output), nearly matching lateral; cross-sectional profiles showed converging FWHM values. - Anatomical detail recovery: suppressed apical dendrites and micro-circuitry became visible; texture matched lateral 90°-rotated imaging without introducing lateral-plane distortions. - Neuron tracing (NeuroGPS-tree + manual correction): reconstructions from network output enabled accurate tracing of neurites that were discontinuous in input due to z-undersampling; verification against lateral imaging yielded precision 98.31% and 98.26% for two neurons. - Quantitative ROI comparison (n=31 MIPs, 140×140 µm^2, 150 µm depth): mean PSNR improvement of 2.42 dB versus reference-lateral ROIs, acknowledging emission differences between sessions. OT-LSM PSF deconvolution (0.5-µm beads): - Network output produced near-spherical bead images; axial elongation corrected: mean axial FWHM reduced from ~3.91 ± 0.28 µm to ~1.95 ± 0.12 µm, closely matching lateral input ~1.98 ± 0.13 µm. Lateral deviation minimal (~0.13 ± 0.06 µm mismatch). OT-LSM brain with artifacts: - Resolution enhanced uniformly in XZ and YZ; contrast improved. Compared with RL deconvolution (10 iterations using measured PSF), the network recovered fine details and also corrected unmodeled artifacts: reduced image doubling in XZ and horizontal ripple stripes in YZ due to stage drift. Improvements were consistent across 8600 slices and replicated over six independent volumes. - Controlled quantitative test (calibrated near-isotropic ground truth + synthetic axial blur σ=10): network recovered axial details better than RL in ROI visuals and corrected stripe artifacts; PSNR and MS-SSIM improved over input; no-reference BRISQUE indicated perceptual naturalness close to ground truth; overall 3D PSNR gain ~1.98 dB.

The study demonstrates that reference-free, unsupervised deep learning can convert anisotropic volumetric fluorescence microscopy data to near-isotropic resolution using only a single 3D stack. By leveraging high-resolution lateral information as a statistical proxy and constraining the mapping via optimal transport with cycle consistency, the approach localizes learning to data-specific characteristics and mitigates hallucination risks commonly associated with GAN-based restorations. Results across simulations, CFM, and OT-LSM show substantial axial resolution gains, recovery of biologically accurate fine structures validated by orthogonal imaging and quantitative metrics, and robustness to diverse, space-varying degradations including artifacts not captured by standard PSF models. Compared with classical RL deconvolution, the method better restores high-frequency details and corrects system-induced artifacts. The framework reduces dependency on prior knowledge of the imaging process and on additional reference datasets, facilitating deployment for large-scale volumetric microscopy where imaging conditions vary.

The authors present an OT-cycleGAN-based, reference-free super-resolution framework that learns from a single anisotropic 3D volume to produce isotropic reconstructions by exploiting lateral-plane information and cycle-consistent distribution matching. The method delivers consistent axial resolution enhancement, anatomically accurate detail recovery, and correction of real-world artifacts across simulation, confocal, and light-sheet modalities, often surpassing PSF-based deconvolution in challenging conditions. The approach is practical to deploy—no registration, PSF modeling, or separate target acquisition is required—and is expected to be applicable across diverse volumetric fluorescence imaging scenarios. The authors note it is not a one-size-fits-all solution and can be complemented by simple post-processing (e.g., histogram matching) for optimal visualization.

- Not a one-size-fits-all solution for all anisotropy problems; performance and visualization may depend on dataset characteristics. - Potential visualization artifacts may appear more discernible in 2D MIP renderings (due to discriminator training on 2D MIPs) or under highly saturated intensity ranges; these are mitigated in full 3D visualization and can be reduced via local histogram matching. - Metric comparisons to orthogonal reference volumes can be affected by differences in fluorescence emission across imaging sessions and registration imperfections. - Training on large 3D volumes can be resource- and time-intensive (e.g., high GPU memory and long training times).