Alternation of inverse problem approach and deep learning for lens-free microscopy image reconstruction

Lens-free (in-line) holographic microscopy records only intensity at the sensor, requiring computational reconstruction to retrieve phase and amplitude of the sample. Simple back-propagation suffers from twin-image artifacts due to missing phase. Model-based inverse methods improve results but remain trapped by local minima when sample optical path differences exceed λ/2, leading to phase wrapping. This is prevalent for thick biological samples such as cells in suspension or undergoing mitosis, hindering quantitative live-cell imaging, especially at high densities where simple constraints (e.g., positive phase) fail. Prior deep learning methods have advanced phase recovery and holographic reconstruction, and CNN-based phase unwrapping has shown robustness to noise and aliasing in digital holography. However, end-to-end CNNs may hallucinate, fail to generalize, or be adversarially fragile, reducing confidence. The study investigates whether alternating a physics-based inverse reconstruction with a CNN unwrapping step can overcome wrapping while preserving data fidelity, enabling robust reconstruction of dense cell suspensions.

- Inverse problem approaches for in-line holography use forward models with parameter fitting and regularization via gradient descent to mitigate twin-image artifacts and partial coherence effects. - Phase unwrapping has a rich history (e.g., minimum Lp-norm, graph cuts) and recent CNN approaches can robustly unwrap noisy, aliased phase maps and even reconstruct phase directly from holograms. - Deep learning has improved lens-free holographic reconstruction (autofocusing, phase recovery), and cross-modality image transformation has been demonstrated. Yet, DL can hallucinate, overfit (poor generalization), and exhibit adversarial fragility. - The proposed method differs from: (1) end-to-end DL predicting images from raw holograms; (2) single-pass physics-informed DL applied once post-inversion; and (3) DL replacing iterative regularization. Instead, it alternates: an initial physics-based reconstruction, a CNN unwrapping, then a final physics-based reconstruction to enforce measurement consistency.

Overall three-step alternation algorithm: (1) First holographic reconstruction from intensity-only measurements using a simplified forward model; (2) CNN unwrapping to correct phase wrapping errors in the reconstructed optical path difference (OPD) and absorption maps; (3) Second holographic reconstruction initialized by the CNN output, with a more complete model and stronger regularization to enforce data fidelity and correct CNN errors. Physical and forward model: Inline lens-free setup with partially coherent illumination at wavelength λ and sample-to-sensor distance Z. The sample transmission is modeled by OPD L(x,y) and (negative) absorption A(x,y). The complex field after the sample is E(L,A)=exp(2iπL/λ + A). Propagation to the detector uses the Fresnel propagator, with partial coherence accounted for by a convolution kernel K and uniform background B. The measured intensity I(L,A)=B·|E(L,A) convolved with h_{Z/2}|^2 convolved with K (expressed as convolutions). Due to partial coherence and intensity-only detection, phase information is incomplete and L is only known modulo λ, causing wrapping. Inverse problem formulation: Reconstruct L and A by minimizing e(L,A)=∫ |I(L,A)−I_meas|^2/I_meas dr + α·ζ(L,A), using gradient-based conjugate-gradient optimization with analytically derived gradients. - First reconstruction: fast, 20 iterations, K approximated by a Dirac (ignores partial coherence), no explicit unwrapping. Regularization combines: total variation on the complex field E to favor edges without propagating wraps; sparsity on L; positivity for absorbance A. - Second reconstruction: 70 iterations, includes partial coherence via kernel K, and regularization on L and A that enforces smoothness on L and A (including Laplacian on A), positivity of A, and non-negativity of unwrapped L. This step assumes the CNN has unwrapped L and serves to improve data fidelity and correct CNN artifacts. CNN for unwrapping: A 20-layer fully convolutional network; each layer has 5×5 convolution (32 features), batch normalization, and ReLU. The output layer is a 2-feature convolution mapping to L and A. No spatial down/up-sampling (same-size input/output). Trained to map first-reconstruction outputs (L_recons, A_recons) to ground-truth (L, A) on synthetic data. Synthetic training data: Images of dense cell suspensions simulated as homogeneous spheres with random radii (5–20 µm), numbers (500–5000; 179–1792 cells/mm²), and refractive indices: real part Δn_r∈[0.01,0.05], imaginary part Δn_i∈[0,0.005]. Intensity holograms simulated using the forward model with a specified Z and partial coherence (kernel K derived by deconvolving partial from total coherence responses). Poisson noise added; background B varied (105±30%). First reconstructions computed with simplified model generate CNN inputs exhibiting realistic wrapping and modeling errors. Training used 1000 image pairs (1000×1000 px), random 121×121 vignettes (12,800 per epoch), Adam optimizer, learning rate 1e-4, 10 epochs (~10 h on a GTX Titan GPU). Quantitative evaluation on simulations: Compared reconstructed Δn_r and Δn_i per cell to ground truth. For each known cell region S^(k) and volume V^(k), Δn_r^(k)_recons ∝ volume-normalized integral of L_recons over S^(k), and Δn_i^(k)_recons ∝ volume-normalized integral of A_recons (constants given). Reported peak signal-to-noise ratio (PSNR) vs ground truth and linear regressions (slope, R^2). Experimental setup: Cytonote lens-free system (Iprasense) with LED illumination at λ=457 nm (20 nm bandwidth), source 50 mm from sample, CMOS detector 6.4×4.6 mm² (3840×2748 px, 1.67 µm pitch). Sample-to-sensor distance Z between 1–4 mm (e.g., Z≈1830 µm for PC3 suspensions; Z≈3500 µm for PC12). Fluorescence microscopy used for positional/morphological validation. Biological samples: Non-adherent PC3 cells expressing GFP; suspensions in 20 µm chambers at low and high densities. Additional suspensions: Molt4, Jurkat, CHO. Adherent PC12 cells differentiated with NGF used to assess generalization to adherent morphologies; adherent fibroblasts provided a challenging failure case.

Simulations: - Visual quality: First reconstructions show severe phase wrapping and artifacts, especially at high densities (e.g., 1627 cells/mm²). CNN outputs closely match ground truth appearance; final reconstructions resemble CNN outputs while improving measurement consistency. - Data fidelity and regularization: CNN introduces a noticeable mismatch to measurements (as per convergence plots), which is corrected by the final reconstruction, yielding lower data misfit with appropriate regularization. - PSNR: CNN increases PSNR by almost a factor of two versus the first reconstruction; final reconstruction PSNR slightly below CNN but with improved data fidelity. - Quantitative Δn correlations: For Δn_r, strong linear correlation with ground truth up to ~0.05 and densities up to ~900 cells/mm², with regression slopes ~0.84–0.96 and R^2>0.75; at highest density (~1618 cells/mm²), correlation holds up to ~0.03. For Δn_i, meaningful correlation only at low density (~360 cells/mm²) with slopes ~0.6 and R^2~0.7. In both cases, the second reconstruction slightly outperforms the CNN. Experiments (PC3 suspensions): - Low density (~290 cells/mm²; ~8450 cells): First reconstructions are unintelligible; CNN and final reconstructions show clear rounded cells. OPD profiles reach ~1000 nm (~5π at λ=405–457 nm), indicating successful unwrapping. Final reconstruction corrects CNN errors visible in line profiles. Fluorescence confirms cell positions and shapes. - High density (~1140 cells/mm²; ~34,000 cells): Alternation approach recovers cell morphology; discrepancies arise for largest cells at very high density, consistent with simulation limits. Effective operation up to ~1000 cells/mm² is concluded. Generalization: - Other suspension lines (Molt4, Jurkat, CHO): Similar behavior—CNN unwraps; final step improves data fit. - Adherent PC12 neurons: CNN unwraps cell bodies but hallucinates rounded spots and loses neurite details; final reconstruction restores fine structures and corrects hallucinations, demonstrating complementarity. - Adherent fibroblasts: Failure case—CNN-induced errors (e.g., spurious additional cells) not corrected by the final step. Throughput: Enables observation of ~35,000 cells simultaneously over ~30 mm² field of view while mitigating phase wrapping, with quantitative Δn_r estimation up to ~1000 cells/mm² in simulations.

The alternating reconstruction addresses the core challenge of phase wrapping in lens-free holography by leveraging complementary strengths of DL and physics-based inversion. The CNN effectively jumps over local minima associated with 2π ambiguities, producing unwrapped OPD and absorption maps, while the final inverse step enforces consistency with the forward model and corrects CNN hallucinations or generalization errors. On simulated dense suspensions, the approach substantially improves PSNR and yields quantitative agreement with ground truth refractive index (real part) within specified density and index ranges. Experiments on PC3 suspensions validate applicability: phase shifts up to ~5π are correctly unwrapped, and fluorescence corroborates reconstructions. The method generalizes to multiple suspension cell lines and, to some extent, to adherent neurons where the second reconstruction recovers fine features lost by the CNN. However, failures on fibroblasts highlight domain limitations. Overall, alternating DL with model-based optimization enhances robustness and confidence over DL-only pipelines and extends lens-free microscopy to denser and thicker biological samples.

This work introduces an alternation framework combining an initial physics-based holographic reconstruction, a CNN trained for phase unwrapping, and a final physics-based reconstruction. The method resolves phase wrapping artifacts that hinder conventional inverse approaches, enabling accurate reconstruction of dense cell suspensions and quantitative estimation of refractive index (real part) up to ~1000 cells/mm² in simulations. Experimental validations on PC3 and other suspension cell lines show clear improvements, with OPD unwrapped up to ~5π and consistency with fluorescence. The final reconstruction step mitigates CNN hallucinations and improves data fidelity, and in some adherent-cell cases (PC12) restores fine structures not captured by the CNN. Limitations remain for some adherent morphologies (e.g., fibroblasts) and at very high densities or index values. Future work could broaden training data to encompass diverse adherent morphologies, incorporate stronger physics-informed architectures, improve robustness to adversarial perturbations and noise, and adapt models across varying distances and imaging conditions to enhance generalization.

- Generalization: The CNN, trained on synthetic suspensions, can hallucinate structures and lose fine details on unseen adherent morphologies; failures observed on adherent fibroblasts. - Domain specificity: Performance degrades at very high densities and larger refractive index contrasts; effective up to ~1000 cells/mm² and Δn_r ~0.05 (simulations) with reduced correlation at higher densities. - Measurement-specific training: Networks are trained for specific sample-to-sensor distances (e.g., Z=1830 µm, 3500 µm), limiting transferability without retraining. - Quantitative validation: Experimental datasets lacked quantitative phase imaging ground truth; evaluation was largely qualitative with positional/morphological validation by fluorescence. - CNN-data mismatch: CNN outputs may deviate from measurements, requiring the final reconstruction to correct; residual errors can persist in difficult cases.