Fluorescence lifetime imaging microscopy (FLIM) provides unique insights into cellular metabolism, dynamics, and protein activity by measuring the temporal decay of fluorophore emission. Unlike conventional fluorescence intensity imaging, FLIM is sensitive to environmental factors influencing fluorophore lifetimes, such as oxygen concentration, pH, and protein-protein interactions. This makes FLIM a powerful tool across various biological applications, including cancer research (cell detection, drug delivery monitoring, and efficacy studies) and clinical diagnostics. However, widespread adoption of FLIM is hindered by limitations in imaging speed and field of view (FOV). The short fluorescence lifetimes (0.1-7 ns) of fluorophores necessitate detectors with sub-nanosecond temporal resolution, typically achieved using point-scanning confocal microscopes with photomultiplier tubes and time-correlated single-photon counting (TCSPC) electronics. Point-scanning systems, however, suffer from photobleaching and lack the instantaneous full FOV information crucial for imaging dynamic processes or in vivo applications. Wide-field FLIM often employs TCSPC with gated optical intensifiers and CCD cameras, but emerging technologies based on Single Photon Avalanche Diode (SPAD) arrays offer advantages such as picosecond temporal resolution and single-photon sensitivity. SPAD arrays with CMOS technology offer a promising alternative to intensifier-based systems. While previously limited in pixel count, recent advancements have enabled the development of SPAD sensors with larger formats. This study demonstrates a 0.5 megapixel (MP) wide-field FLIM system utilizing a SPAD array and a novel artificial neural network (ANN) for rapid lifetime estimation, addressing the speed and FOV challenges in current FLIM technologies. The use of a large-area detector is particularly advantageous for imaging dim biological samples, as the speed advantage of wide-field imaging over raster-scanning increases with the number of pixels.

Existing FLIM techniques often utilize point-scanning confocal microscopes with photomultiplier tubes and TCSPC, which although offering high temporal resolution, are limited by speed and photobleaching. Wide-field FLIM has been attempted using TCSPC and gated optical intensifiers with CCD cameras, but these setups also present challenges in terms of speed and sensitivity. The development of SPAD arrays offers a promising alternative, promising improved speed and sensitivity; however, previous SPAD arrays were limited in pixel count. Recent advances in SPAD technology have enabled the development of larger arrays, opening up possibilities for high-speed, wide-field FLIM. Existing methods for lifetime extraction from FLIM data include least-squares (LSQ) deconvolution, which is computationally intensive, and phasor analysis, which provides fast visualization but may have limitations in accuracy. The emergence of machine learning, particularly deep learning, has provided opportunities for faster and more efficient lifetime estimation. Several studies have demonstrated the use of artificial neural networks (ANNs) for rapid and fit-free analysis of FLIM data, significantly improving processing speed compared to traditional methods.

This research employed a 0.5 MP time-gated SPAD camera (described in detail in Ref. 38 and the “Methods” section) operating in gated mode. The camera's gate (approximately 3.8 ns with a super-Gaussian profile) was scanned in steps as small as 36 ps, generating a temporally resolved spatial image by stacking the frames. Samples were imaged with 0.47 µm/pixel spatial sampling (except for Fig. 2g-i, which used 0.33 µm/pixel). Lifetime extraction was performed using two methods: least-squares (LSQ) deconvolution and a custom-designed artificial neural network (ANN). The LSQ method involved convolving a fluorescence decay model with the instrument response function (IRF), modeled as a super-Gaussian function. The ANN, shown in Fig. 1b, comprised an input layer (IL), an output layer (OL), and three hidden layers (HLi). The IL received a 1D vector representing the time-resolved photon-count signal for each pixel. The ANN was trained on a large dataset of 2 million simulated fluorescence decay curves with added Poisson and Gaussian noise, reflecting the characteristics of the photon detection process and electronic noise. The performance of both methods was evaluated by comparing their results on both simulated and experimental data (Convallaria and HT1080 cells). The pile-up correction was performed using equation (2) to correct for the pile-up effect. The background subtraction was performed by taking the average of the first few frames before the signal is observed and subtracting this from all the frames. A sliding threshold was applied to the data (from Ntot = 1300 to Ntot = 50) to account for the non-uniform response of the SPAD array. For mosaic imaging, the sample was moved in tiles with approximately 10% overlap. Image stitching was performed using BigStitcher (https://imagej.net/BigStitcher).

The 0.5 MP SPAD array enabled wide-field FLIM at 1 Hz acquisition speed with high-photon-count (HPC) and low-photon-count (LPC) data acquisitions. The ANN-based lifetime retrieval provided significantly faster processing (1000-fold speed improvement) compared to LSQ deconvolution. Both HPC and LPC data yielded similar mean lifetimes using both LSQ and ANN methods (Table 1), indicating reliable lifetime retrieval even at low signal-to-noise ratios. Imaging of HT1080 cells showed comparable results between the two methods, with lifetimes matching previously reported values. A 3.6 MP fluorescence lifetime image of Convallaria was achieved using mosaic acquisition with the 0.5 MP array, demonstrating the scalability of the system. The ANN processed this 3.6 MP dataset in only 36 seconds. The study provided a quantitative comparison of the LSQ and ANN methods for lifetime retrieval, which is shown in Figure 4. The results indicated that both methods provide similar lifetime estimations. The ANN provided accurate results, with a root mean square error of 0.0725 on a test set of approximately 1 million synthetic data points.

This study successfully demonstrated a high-speed, wide-field FLIM system using a large-format SPAD array and ANN-based lifetime estimation. The significant speed improvement achieved by the ANN makes real-time FLIM feasible for dynamic biological processes. The ability to acquire and process large FOV images opens new avenues for studying complex cellular structures and interactions. The results obtained on both high and low signal samples demonstrated the robustness of the method. The excellent agreement between the ANN and LSQ deconvolution validates the accuracy of the neural network approach. The successful acquisition and processing of a 3.6 MP image highlight the potential for scaling up the system to achieve even larger FOVs and temporal resolution. This methodology can be applied to various biological and medical imaging applications where speed and high spatial resolution are crucial.

This work successfully demonstrated a high-speed, wide-field FLIM system utilizing a large-format SPAD array and a novel ANN-based lifetime estimation method. The 1000-fold speed improvement over LSQ deconvolution enables real-time FLIM, while the scalability to multi-megapixel resolution opens possibilities for more comprehensive biological imaging. Future research could explore extending this method to multi-exponential decay analysis and further optimizing the ANN for improved accuracy and speed. Advancements in SPAD technology (increased fill factor and quantum efficiency) will further enhance the capabilities of this FLIM platform.

The current system retrieves mono-exponential lifetimes, limiting its application in scenarios with complex multi-exponential decays. The mosaic acquisition approach necessitates careful sample positioning to minimize stitching artifacts. Further optimization of the ANN might be needed to improve accuracy and robustness in extremely low-photon-count conditions. The current design of the SPAD array camera has a 7% fill factor and a 10% photon detection probability at 510 nm, which limits the overall detection sensitivity and could affect the accuracy of the measurements.