Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation

The study addresses the challenge of achieving fast, wide-field fluorescence lifetime imaging microscopy (FLIM) suitable for biologically relevant and potentially clinical applications. Conventional point-scanning TCSPC systems provide high temporal resolution but are slow, can cause photo-bleaching, and lack instantaneous full field-of-view capture, limiting studies of dynamic samples. Wide-field approaches using intensifiers and CCD/CMOS sensors or SPAD arrays offer alternatives, but time-gated cameras require gate scanning and TCSPC SPAD arrays often have low fill-factor. The authors aim to demonstrate scan-less wide-field FLIM using a large-format (0.5 MP) time-gated SPAD camera and to overcome computational bottlenecks in lifetime extraction by employing a pre-trained artificial neural network (ANN), enabling near real-time lifetime imaging at up to 1 Hz acquisition and extending to multi-megapixel fields via mosaic imaging.

Prior wide-field FLIM implementations commonly used microchannel plate-based gated intensifiers coupled to position-sensitive sensors (CCD/CMOS). SPAD arrays fabricated in CMOS have emerged as alternatives, operating either in TCSPC or time-gated modes, offering picosecond timing and single-photon sensitivity. Earlier SPAD arrays were limited in pixel count; recent advances produced large arrays such as SwissSPAD (512×512) and a megapixel time-gated SPAD sensor. Analysis methods include least-squares (LSQ) deconvolution, center-of-mass (CMM) with background correction, and phasor analysis for rapid visualization. Deep learning has been applied to FLIM for fast, fit-free decay parameter estimation. The paper builds on these developments by combining a large-format time-gated SPAD imager with an ANN to accelerate lifetime retrieval while maintaining accuracy.

Imaging hardware and acquisition: A 1 MP SPAD image sensor was operated in time-gated mode with only half the array read out, yielding 0.5 MP (500×1024 pixels). The gate has an average width of ~3.8 ns with a super-Gaussian profile and can be shifted in steps as small as 36 ps. For each gate position, 255 binary frames are summed into an 8-bit image (or 9-bit by summing two registers). The gate starts before excitation and is shifted through the decay to sample the fluorescence. The camera has 7% fill factor and ~10% photon detection probability at 510 nm. Samples were imaged at 0.47 µm/pixel (most datasets) or 0.33 µm/pixel (HT1080 and mosaic). Illumination used a 470 nm picosecond diode laser (47 ps nominal pulse width) at 25 MHz repetition rate with FITC excitation/emission filtering and 20×/0.4 NA or 40×/0.75 NA objectives. Pre-processing: Background subtraction was performed by averaging pre-signal frames and subtracting from all frames. Pile-up correction used I_corr = −I_max ln(1 − I_rec/I_max), where I_max is the maximum count per pixel given bit depth (e.g., 255 for 8-bit), and I_rec is the recorded value. To handle non-uniform response across the array, a sliding total-count threshold N_tot was applied from 1300 (right edge) to 50 (left edge) to remove low-signal pixels. LSQ deconvolution: The system impulse response function (IRF) is dominated by the SPAD gate (laser pulse width <47 ps negligible). The IRF at each pixel is modeled as a super-Gaussian: g_N(t) = exp{−2[(t − t′)/w_N]^{2N}}, with w_N related to FWHM by w_N = 2(0.5 ln 2)^{1/(2N)} FWHM. Order N=6 matches measured rise times (~0.61 ns vs measured 0.55±0.08 ns). The fluorescence decay model is d(t) = A_0 exp(−t/τ) + b for t ≥ t_0, and b for t < t_0. The measured signal is f(t) = g_N(t) ⊗ d(t). LSQ optimization (MATLAB lsqcurvefit) estimates [w_N, t_0, τ, b, A_0, t′] per pixel. Artificial neural network (ANN): A fully connected network takes a 1D input vector of the time-resolved signal normalized to [0,1]; for 200 time bins, the input layer has n0=200 nodes. Three hidden layers with sizes 100, 50, and 25 each use ReLU activations; the output layer predicts τ. Separate ANNs were trained for 200-bin (108 ps shift) and 30-bin (504 ps shift) data. Training used 2 million synthetic decay curves (mono-exponential τ in 0.5–5 ns, start t0 in 5–10 ns, amplitude A in 2–32, gate width w in 3.6–6 ns), generated by convolving with the super-Gaussian gate per the model, with added noise. Noise model combined Poisson (photon counting) with Gaussian electronic noise; Gaussian noise mean and standard deviation were randomly drawn from [−2, 2] and [0, 5] counts, respectively. Training used mini-batch size 128, Adam optimizer, MSE loss; validation set of 1 million curves controlled overfitting (training loss <10% smaller than validation). Test set (1 million) yielded MSE 0.0053 ns^2 and RMSE ~0.072 ns for 200-bin data; for 30 bins, MSE 0.0634 ns^2 with ~0.25 ns SD. The pre-trained model inference on a 1024×1024 dataset with 200 bins on an Intel Core i7-10510U CPU took ~8.5±0.5 s. Samples: Convallaria (lily of the valley) slides for benchmarking. HT1080 human fibrosarcoma cells expressing Clover (pcDNA3-Clover) were prepared: cultured in DMEM with supplements, transfected via Amaxa nucleofector (Kit T, program L-005) with 5 µg DNA, replated, laminin-coated dishes, fixed in 4% paraformaldehyde, mounted with Fluoromount-G. Acquisition regimes: High photon count (HPC) Convallaria used 504 ps gate shift, ~330 ms exposure per frame; total acquisition 10 s (and for mosaic ~16 min). Low photon count (LPC) Convallaria used ~33 ms exposure per frame for 1 Hz acquisition. HT1080 used 108 ps gate shift with total ~400 s acquisition. Mosaic imaging covered 8 tiles with ~10% overlap to produce a 3.64 MP lifetime image; stitching used BigStitcher.

- Demonstrated scan-less wide-field FLIM with a 0.5 MP (500×1024) time-gated SPAD camera, achieving up to 1 Hz acquisition in low-photon-count mode. - ANN-based lifetime retrieval matched LSQ deconvolution accuracy with vastly reduced processing time: ANN processed full images in 2.7–3.6 s versus LSQ taking 23.2–58 min (approximately 388–1288× speedup; up to ~1000× reported for megapixel scale). Mean absolute difference between methods across datasets was 0.14 ± 0.12 ns. - Convallaria results: HPC (10 s acquisition, 504 ps gate shift, ~330 ms exposures) and LPC (1 s total, ~33 ms exposures) produced consistent mean lifetimes between methods. Reported means (Table 1): HPC LSQ 1.29±0.49 ns vs ANN 1.22±0.27 ns; LPC LSQ 1.20±0.53 ns vs ANN 1.28±0.34 ns. Photon counts: LPC <2700 photons/pixel; HPC >8500 photons/pixel. - HT1080 cells expressing Clover: mean lifetimes LSQ 2.41±0.29 ns vs ANN 2.31±0.34 ns (absolute difference 0.10 ± 0.05 ns), consistent with literature (~2.6 ns). HT1080 were significantly dimmer than Convallaria (∼100 photons/s in bright regions vs ∼2500 for Convallaria). - Large field-of-view mosaic: 3.64 MP (1875×1942) Convallaria lifetime image over ~618×650 µm (0.33 µm/pixel). Acquisition ~16 min in HPC mode (could be reduced to 10–20 s in LPC mode), with ANN processing time ~36 s. - ANN robustness: Test-set RMSE ~72 ps for 200-bin data; maintained unbiased estimates across lifetimes. Benchmarks on synthetic data indicated ~200 ps resolution (2σ) for lifetimes in 0.5–2 ns with 500 ps gate shift sampling. - Operational insights: Wide-field acquisition can be up to N^2 faster than raster scanning for dim/sparse samples (N = pixels per row), and the large SPAD array enables imaging of multiple cells across a large FOV.

The results demonstrate that a large-format time-gated SPAD camera enables rapid, wide-field FLIM while mitigating limitations of point-scanning systems such as photo-bleaching risk and slow acquisition. By coupling the hardware with an ANN trained on physics-based synthetic data including realistic noise, lifetime retrieval is accelerated by three orders of magnitude compared to LSQ without sacrificing accuracy, enabling near real-time visualization at megapixel scales. Consistent lifetimes between high and low photon count conditions show reliable performance even with reduced SNR, a key requirement for biological samples that are dim or sensitive to bleaching. The ability to tile and stitch multiple fields extends FLIM to multi-megapixel images with modest processing time, providing large FOV imaging suitable for heterogeneous samples or studies requiring context. Trade-offs remain between temporal sampling (gate shift) and acquisition time that can be tuned per application. Overall, the approach advances FLIM towards practical, high-throughput biological and potentially clinical workflows by addressing both acquisition and analysis bottlenecks.

This work showcases 0.5 MP scan-less wide-field FLIM using a time-gated SPAD camera and a pre-trained ANN for rapid mono-exponential lifetime estimation. The system achieved up to 1 Hz acquisition, produced accurate lifetimes on Convallaria and HT1080 samples, and scaled to a 3.6 MP mosaic with fast processing. The methodology paves the way for real-time or near real-time FLIM over large fields of view. Future work includes extending ANN and analysis to multi-exponential decays, further improving detector performance (fill factor, quantum efficiency), optimizing gate strategies to reduce acquisition time, and deploying faster parallel inference to reach true real-time megapixel FLIM.

- Lifetime retrieval assumes mono-exponential decays; multi-exponential behaviors were not addressed and typically require higher SNR and more complex fitting. - Time-gated cameras require scanning the gate to achieve temporal resolution, increasing acquisition time compared to single-shot methods; reducing time bins can impact accuracy. - The sensor readout in gated mode uses only half of the 1 MP array (effective 0.5 MP), and the fill factor is 7%, limiting photon collection efficiency. - Non-uniform sensor response necessitated sliding thresholds and pre-processing; performance varies across the array. - Low photon count samples present challenges; although ANN performed well, extreme low SNR may limit precision. - One-photon excitation power is constrained by photobleaching, limiting photon budgets and acquisition flexibility.