Fluorescence lifetime imaging microscopy (FLIM) is a valuable technique for analyzing endogenous fluorescence in biological samples, providing insights into metabolic state, pathological conditions, and tissue constitution. FLIM measures the fluorescence lifetime of molecules, which is independent of intensity but sensitive to the bio-environment, offering contrast based on lifetime differences. This has applications in cancer detection, cellular phenotyping, and drug delivery monitoring. Conventional FLIM analysis often uses statistical methods like histogram analysis or phasor characterization, which may struggle to provide cellular-level information without reference data, such as histology images. Co-registering FLIM and histology images is challenging due to potential structural changes during tissue preparation and the unavailability of histology images alongside FLIM images. Deep learning (DL) offers a promising alternative for automated, high-throughput image analysis. Several studies have successfully used DL for virtual histological staining from various imaging modalities, such as translating label-free autofluorescence intensity images to multiple histology stains or generating virtual H&E stains from multi-channel fluorescence images. This study explores DL for generating virtual H&E staining from label-free FLIM images to enable rapid and precise cellular-level interpretation and identify label-free autofluorescence lifetime signatures for various tumor cells. The study compares the effectiveness of different input data formats (intensity and weighted lifetime images) to highlight the advantages of using FLIM for virtual H&E staining and evaluates the method across multiple cancer types.

The existing literature demonstrates the successful application of deep learning in virtual histological staining. Studies like Rivenson et al. (2019) showed the effectiveness of translating label-free autofluorescence intensity images into multiple histology stainings using a supervised GAN with total variation constraints. Li et al. (2020) synthesized bright-field images to H&E, Picrosirius Red, and Orcein stains. Borhani et al. (2019) used a custom DL model to generate virtual H&E stains from multi-modal microscopy images. Cao et al. (2023) applied CycleGAN to transform ultraviolet photoacoustic microscopy into virtual H&E staining. Other examples include transferring H&E to virtual IHC and IHC to virtual multiplex immunofluorescence. However, these studies did not utilize FLIM data, which contains valuable lifetime information alongside intensity. While some studies incorporated lifetime information, they often utilized multi-modal microscopy techniques which were more complex than the method used in this current study.

The study utilized a supervised deep learning model, pix2pix GAN, along with the Deep Image Structure and Texture Similarity (DISTS) metric to synthesize H&E-stained histology images from FLIM images. **Sample Preparation and Data Collection:** Surgically resected, early-stage non-small cell lung carcinomas were used to create tissue microarrays (TMAs). FLIM images were collected using a Leica STELLARIS 8 FALCON FLIM Microscope, with excitation and emission wavelengths determined by a λ-to-λ scan of the tissue. The same samples were then H&E stained and imaged using a bright-field slide scanner. In total, 84 lung cancer tissue samples, 4 colorectal cancer slides, and 4 endometrial cancer sections were used. **Data Post-processing:** FLIM images were reconstructed using exponential fitting to estimate lifetimes, and intensity and lifetime images were exported separately. Intensity images were stitched using MIST, and affine transformation in MATLAB was employed for co-registration of FLIM and H&E images. The co-registered images were resampled into 256x256 pixel patches. Three input formats were tested: greyscale intensity, false-colour lifetime images with normalized intensity as the alpha channel (α-FLIM), and intensity-weighted false-colour lifetime images (IW-FLIM). **GAN Architecture:** The pix2pix GAN, a conditional model with a U-Net-like generator and a multi-layer discriminator, was used. The loss function included the original GAN loss, L1 distance between true and synthetic H&E images, and the DISTS loss to improve image quality. **DISTS Loss:** The DISTS metric combines structural and texture information, improving the synthesis of realistic histological images. **Blinded Evaluation:** Twelve lung TMA cores were used for blinded evaluation by three experienced pathologists, assessing nuclei detail, cytoplasm detail, overall staining quality, and diagnostic confidence. **Implementation Details:** The model was implemented using PyTorch. 72 lung cancer samples were used for training, and 15 independent samples for testing. Colorectal and endometrial cancer data were trained via transfer learning. Data augmentation (horizontal flipping and rotation) was used during training.

The study successfully generated clinical-grade virtual H&E staining from label-free FLIM images. **Virtual H&E Staining from FLIM:** The method effectively reconstructed various cellular components in lung tissue samples, faithfully reproducing morphological and textural attributes of different cell types (tumor cells, stromal components, inflammatory cells, and red blood cells). **Blind Evaluation of Virtual Staining Quality:** Blind evaluation by three experienced pathologists showed high agreement between true and virtual H&E-stained images in terms of nuclei detail, cytoplasm detail, overall staining quality, and diagnostic confidence. **Virtual H&E Staining on Various Tissue Samples:** The method successfully generated virtual H&E stains for colorectal and endometrial cancers, demonstrating consistency with true H&E images. While performance was slightly less remarkable on FFPE lung biopsies, the virtual images did not compromise clinical decision-making. **Lifetime Signatures of Various Cell Types in Lung Tissue:** The study identified distinct lifetime signatures for seven different cell types (tumor cells, fibroblasts, lymphocytes, plasma cells, neutrophils, macrophages, and RBCs) in the tumor microenvironment. Macrophages showed the longest lifetime, while RBCs showed the shortest. **Comparison of Different Image Formats:** The study compared virtual H&E staining using three different input formats: intensity, α-FLIM, and IW-FLIM. While all formats produced acceptable results, IW-FLIM yielded the best results, both visually and quantitatively, indicating that incorporating both intensity and lifetime information improves the accuracy of virtual staining.

This study demonstrates the feasibility of generating high-quality virtual H&E stained images from label-free FLIM images using a deep learning approach. The incorporation of lifetime information from FLIM significantly enhanced the accuracy and detail of the virtual staining compared to using intensity information alone. The ability to generate virtual H&E images directly from FLIM data eliminates the need for traditional H&E staining, thereby reducing the time and resources required for tissue analysis. The identification of distinct lifetime signatures for various cell types opens new avenues for label-free characterization of cellular morphology and phenotype. While the FLIM acquisition time is currently longer than traditional H&E staining, this can be significantly reduced with optimized scanning parameters. The method's applicability across different cancer types (lung, colorectal, endometrial) highlights its potential for broad use in pathology. Although the pix2pix and DISTS loss function were leveraged in this study, other advanced deep learning models may yield superior results with further optimization.

This study presents a novel deep learning-based method for generating virtual H&E staining from label-free FLIM images. The approach successfully generates clinical-grade virtual histology images, providing rapid and precise cellular-level interpretation and enabling the identification of distinct lifetime signatures for various cell types. The method demonstrates potential for biomarker-free tissue histology and has significant implications for improving efficiency and expanding the capabilities of FLIM in cancer diagnostics and research. Future work could explore other advanced DL models and loss functions for further improvements in image quality and explore the clinical applications of FLIM-based virtual H&E staining.

The main limitation is the longer acquisition time of FLIM images compared to traditional H&E staining, although this can be improved with optimization. The study used a specific FLIM system and staining protocol; the generalizability to other systems needs further investigation. The performance on FFPE biopsies was slightly less impressive compared to TMAs, indicating the need for further refinement of the method for this specific tissue type. The study focused on a limited number of cancer types; further research is required to validate the approach across a wider range of tissues and pathologies.