Alternation of inverse problem approach and deep learning for lens-free microscopy image reconstruction

Lens-free microscopy offers a minimalist approach to inline holography, recording only intensity measurements in the sensor plane. Image reconstruction requires computational methods to retrieve the sample image by solving the inverse problem, often using optimization algorithms relying on gradient computation. However, these algorithms are susceptible to local minima, leading to unsatisfactory convergence when phase wrapping errors occur. These errors, arising when the optical thickness of the sample exceeds λ/2, are particularly problematic in samples with large optical thickness, such as dense cell suspensions or cells undergoing mitosis. This significantly limits the application of lens-free microscopy in live cell imaging. While simple positive phase constraints can correct phase wrapping errors at low cell concentrations, this approach fails at higher densities due to the accumulation of errors. Recent studies have explored the use of Convolutional Neural Networks (CNNs) for phase unwrapping in digital holography microscopy, demonstrating robustness to noise and aliasing. However, existing deep learning methods for lens-free holographic reconstruction often don't address phase wrapping directly, and suffer from drawbacks like hallucination, generalization issues, and adversarial fragility. This research aims to address these limitations by proposing a hybrid approach that combines the strengths of inverse problem approaches and deep learning.

Several inverse problem approaches have been employed to improve lens-free microscopy image reconstruction, including forward model-based algorithms using parameter fitting and regularization with gradient descent. These methods, while effective in some cases, struggle with the phase wrapping issue present in thicker samples. Methods focusing on phase unwrapping in digital holography have emerged, using techniques like graph cuts and minimum lp-norm methods, but these can also be computationally demanding and susceptible to errors. The use of CNNs for phase unwrapping has shown promise, offering robustness to noise and aliasing in digital holography microscopy. Deep learning methods have improved lens-free holographic reconstruction, but often neglect the crucial problem of phase wrapping. While some have utilized CNNs to transform images between different imaging techniques or to remove phase wrapping, the inherent limitations of CNNs, such as hallucination, generalization issues, and vulnerability to adversarial examples, remain a concern. Therefore, there's a need for a novel approach that combines the strengths of both inverse problem methods and deep learning to achieve robust and accurate reconstruction in lens-free microscopy, particularly for high-density cell samples.

The proposed approach is a three-step algorithm. First, a holographic reconstruction is performed using a null sample as initialization. This provides a first guess of the sample image, which is then fed into a CNN trained specifically for phase unwrapping. The CNN, trained on synthetic datasets of cells in suspension, corrects the phase wrapping errors in the initial reconstruction. Finally, the CNN's output is used as initialization for a second and final holographic reconstruction step. This step refines the reconstruction, further correcting any errors introduced by the CNN and ensuring a better fit to the measured data. The holographic reconstruction itself uses a regularized optimization problem to minimize a criterion combining data fidelity and regularization terms. The forward model incorporates the partial coherence of the light source, using a convolution kernel to account for this effect. Two different regularization terms are used – a simpler one for the initial reconstruction (favoring sharp edges and positive absorbance) and a more complex one for the final reconstruction (incorporating smoothness constraints on the unwrapped OPD). The CNN architecture consists of 20 layers, with each layer having a convolutional layer, batch normalization, and a ReLU activation function. The training data for the CNN was generated using simulations, creating synthetic images of cells in suspension with varying parameters such as cell density, radius, and refractive index. The simulation incorporates the Fresnel propagator and adds Poisson noise to mimic real-world acquisition noise. The evaluation of reconstruction quality was performed quantitatively by comparing the reconstructed refractive indices to the ground truth values for the simulated datasets. Experimental validation was conducted using a Cytonote lens-free setup with PC3, PC12, Molt4, Jurkat, and CHO cells. Fluorescence microscopy was used for comparison to assess the accuracy of the cell positions and shapes in the reconstructed images.

The study demonstrated the effectiveness of the proposed alternation approach on both simulated and experimental datasets. In simulated data, the CNN significantly improved the peak signal-to-noise ratio (PSNR) compared to the initial reconstruction. The final reconstruction, while slightly less accurate than the CNN output, further improved the data fit and showed a strong linear correlation between reconstructed and ground truth refractive index values at lower cell densities (up to 900 cells/mm²). At higher densities (1618 cells/mm²), correlation remained, though less strong. In experimental data from PC3 cells in suspension, the method effectively removed phase wrapping errors, producing clear images of individual cells even at high densities (up to ~1000 cells/mm²). Qualitative comparison with fluorescence microscopy validated the reconstruction results. The method was also applied to different cell lines (Molt4, Jurkat, and CHO cells), producing similar results. Further testing on adherent PC12 cells demonstrated the ability of the algorithm to reconstruct complex cellular structures, with the final reconstruction step correcting errors introduced by the CNN. However, the method did not generalize well to all adherent cell types; specifically, adherent fibroblast cells were not successfully reconstructed, illustrating the limitations of the trained CNN on this cell morphology. The maximum OPD values achieved reached approximately 1 µm, demonstrating successful unwrapping of phases with shifts of up to approximately 5π.

The results demonstrate the effectiveness of the proposed method in addressing the phase wrapping problem in lens-free microscopy image reconstruction of cells in suspension. The combination of inverse problem optimization and deep learning provides a powerful approach, leveraging the strengths of each technique. The inverse problem approach provides a physically grounded model and the deep learning approach enhances the reconstruction by correcting phase wrapping errors and improving the handling of local minima. The method's success in reconstructing high-density cell samples significantly extends the applicability of lens-free microscopy for live-cell imaging. The ability of the final reconstruction step to correct CNN errors underscores the complementarity of the two approaches. While the method generalizes well to different cell lines in suspension, its performance on adherent cells is less consistent, highlighting the importance of training data diversity and the need for further refinement to handle the complexities of adherent cell morphologies.

This study presents a novel three-step reconstruction approach for lens-free microscopy that successfully mitigates phase wrapping errors. The alternation between inverse problem optimization and deep learning offers a robust solution for high-density cell suspensions, enabling the imaging of thousands of cells simultaneously. The results highlight the potential of this hybrid approach for advancing live cell imaging capabilities in lens-free microscopy. Future work could focus on improving the generalization of the CNN to a wider variety of cell types, potentially through the use of more diverse training data or more advanced network architectures.

The CNN's performance is dependent on the quality and diversity of the training data. The current model was trained primarily on simulated data of cells in suspension, limiting its generalizability to other cell types or morphologies, as shown by the less successful reconstruction of adherent fibroblasts. Further refinement of the CNN architecture and training data might improve its performance on adherent cells and other complex scenarios. The study primarily focuses on quantitative analysis of the real part of the refractive index, while the analysis of the imaginary part is less conclusive at higher densities. Future research should further explore the quantitative aspects of the method for a wider range of parameters.