Mid-infrared single-pixel imaging at the single-photon level

Mid-infrared (MIR) imaging is crucial for applications such as biomedical diagnosis, defect inspection, molecular spectroscopy, and remote sensing, where sensitivity at low photon flux can enable improved detection sensitivity, longer working distances, and noninvasive capabilities. Conventional MIR focal plane arrays (FPAs) face limitations including high dark noise, low pixel counts, thermal susceptibility, cryogenic operation, and high cost. Emerging room-temperature MIR materials (e.g., colloidal quantum dots, black phosphorus, graphene, tellurium nanosheets) show promise but still suffer from dark-current suppression and scalable pixelation challenges. Single-pixel imaging, which uses a spatial light modulator (SLM) to project patterns and a single-element detector to measure correlated intensities, offers advantages of cost, speed, and compatibility with compressed sensing and machine learning. However, extending single-pixel photon-sparse imaging to the MIR has been impeded by the lack of single-photon MIR detectors and high-fidelity spatial modulators at long wavelengths (e.g., DMD diffraction effects at MIR, early-stage MIR modulators). The research question addressed here is whether single-photon-level MIR single-pixel imaging can be realized at room temperature by bypassing MIR detector and modulator limitations. The authors devise a nonlinear frequency-upconversion approach—nonlinear structured detection—where high-resolution patterns are applied to a near-infrared pump inside a nonlinear crystal to modulate the MIR signal via sum-frequency generation (SFG), simultaneously translating MIR photons to the visible for sensitive silicon single-photon detection. They show real-time video (10 Hz, 16×16 pixels), single-photon imaging down to 0.5 photons/pulse, and compressive imaging at 25% sampling aided by deep learning denoising, enabling MIR single-photon computational imaging for low-photon-flux applications.

The paper situates the work within several strands: (1) Limitations of MIR FPAs (high dark noise, low pixel counts, thermal sensitivity, cryogenic operation, high cost) and the challenges facing emerging MIR materials (dark current suppression, large-area deposition). (2) Single-pixel imaging provides economic and practical benefits and can leverage compressed sensing and machine learning for sub-Nyquist sampling and low-light imaging; prior photon-sparse single-pixel demonstrations largely remained in the visible/NIR due to detector and modulator availability. (3) Prior MIR compressive imaging efforts used few-pixel infrared sensors, remaining far from single-photon sensitivity. (4) Conventional SLMs (liquid crystal, micromirror/DMD) face parasitic diffraction and beam-steering limits at longer wavelengths, impairing MIR intensity modulation; graphene metasurface MIR modulators are emerging but currently low-resolution (e.g., 6×6). (5) Optically controlled modulators have been applied for THz single-pixel imaging, showing sub-wavelength resolution, but typically without spectral transduction; thus detection remains limited by conventional detectors. Frequency upconversion via coherent processes has enabled MIR imaging using EMCCD FPAs or quantum-correlated approaches, but at higher cost/complexity. These gaps motivate a single-pixel, nonlinear upconversion architecture enabling high-resolution patterning and room-temperature single-photon detection of MIR signals using silicon detectors.

Concept and theory: The approach uses nonlinear structured detection via sum-frequency generation (SFG) in a nonlinear crystal to imprint high-resolution pump patterns onto a MIR object field and simultaneously upconvert MIR photons to the visible. Under paraxial and slowly varying envelope approximations, the SFG field E_up ∝ E_s(x,y)·E_p(x,y). At phase-matching, the SFG intensity obeys I_up(x,y) = I_s(x,y) × I_p(x,y). The pump intensity pattern thus acts as a spatial transmission mask for the MIR image. A sequence of orthonormal patterns (e.g., Hadamard) is applied; for each pattern and its inverse, differential single-pixel measurements P_up^(m) are recorded to mitigate offsets and low-frequency noise. The image O(x,y) is reconstructed from the weighted sum over M patterns. Experimental setup: A dual-color, passively synchronized ultrafast fiber laser provides a MIR signal at 3070 nm (bandwidth 0.9 nm) via difference-frequency generation (EDFL at 1550 nm mixed with YDFL) and a pump at 1030 nm (bandwidth 0.3 nm). Both operate at 14.6 MHz repetition rate; one YDFL branch is amplified to 280 mW for pumping. The MIR beam illuminates a transmission mask (four letters), forms an object image that is relayed into a periodically poled lithium niobate (PPLN) crystal (channel 10×1×1 mm³). The pump is spatially patterned by a DMD (Texas Instruments DLP650LNIR; 1280×800 micromirrors; 10.8 μm pitch; ±12° tilt; up to 10.752 kHz binary rate) and combined with the MIR via a dichroic mirror into the PPLN. The crystal operates near phase matching at 48.6 °C with a 20.9 μm poling period. The pump beam size is adjusted to cover the object image while fitting within the crystal aperture. Coincidence pulsed pumping with optimized temporal delay leverages high peak power and ultrashort gating to enhance conversion efficiency and suppress background. Upconverted SFG light at 771 nm is spectrally filtered (to reject pump-induced parametric fluorescence) and detected by a single-element silicon detector. Detectors and acquisition: Two detectors are used: (i) analog Si photodiode (Thorlabs PDA100A2), NEP ~3 pW/Hz^1/2, 11 MHz bandwidth, 75.4 mm² area; (ii) Si avalanche SPCM (Excelitas SPCM-AQRH-54), 63% detection efficiency at 771 nm, dark counts <100 Hz, 180 μm active diameter. Timing control and data acquisition are performed by an FPGA module (Altera Cyclone II) with up to 250 MHz sampling clock and a 12-bit ADC (Analog Devices AD7091R8; 1 μs sampling time). The system synchronizes laser triggers, DMD updates, and data capture. Patterning and mapping verification: Representative Hadamard patterns programmed on the DMD are imaged to the PPLN center (pump patterns) and observed in the SFG output to confirm faithful nonlinear spatial mapping. Full sets of patterns are recorded for validation (Supplementary Movie 1). The optical patterns are corrected for pump beam inhomogeneity measured at the DMD all-on state. Reconstruction algorithms: For Hadamard encoding, complementary pattern differencing is used; orthonormality allows reconstruction via transpose of the measurement matrix. For compressive imaging, random masks are employed to satisfy incoherence conditions. Image recovery uses a primal–dual algorithm minimizing an l1-type objective (details in Supplementary Note 5). Noise suppression is further improved with a plug-and-play deep convolutional neural network Gaussian denoiser. Operating conditions and optimization: The pump pulse duration is set longer than the MIR signal, temporally confining MIR photons within the pump’s central envelope to maximize conversion. The system employs picosecond-scale optical gating to reduce ambient scattering background. Spatial resolution is limited by the upconversion system’s numerical aperture and the PPLN phase-matching angular bandwidth. In the reported configuration, DMD pixel pitch is 10.8 μm; achieved imaging resolution is ~91 μm. Field of view is bounded by the pump diameter and/or crystal aperture.

- Demonstrated MIR single-pixel computational imaging at the single-photon level using nonlinear structured detection and frequency upconversion, enabling the use of a room-temperature silicon detector. - Real-time operation: Achieved 10 Hz video frame rate for 16×16 reconstructed pixels (Hadamard encoding; Supplementary Movie 2). - Spatial resolution and mapping: DMD mask pixel size 10.8 μm; measured imaging resolution ~91 μm. Verified faithful mapping of DMD patterns to MIR via SFG (Fig. 2), with some blur due to phase-matching angular bandwidth and limited spatial frequency response. - Photon-sparse imaging (photon counting): Using an SPCM at 771 nm with strict spectral filtering and ultrashort optical gating, reconstructed high-contrast images at MIR fluxes from 100 down to 0.5 photons/pulse. Examples: • 100 photons/pulse with 5 ms integration per pattern (16×16 and 64×64). • 0.5 photons/pulse with 1 s integration per pattern still yields identifiable images (greater noise for higher pixel counts due to fewer photons per pixel and added dark counts). - Compressive imaging: With random masks and a primal–dual recovery algorithm, obtained recognizable images at 25% undersampling (64×64) for 100 and 5 photons/pulse; residual noise reduced substantially using a deep CNN denoiser, enabling high-contrast reconstruction at 1 photon/pulse with 25% measurements (Fig. 5). - System specifications supporting sensitivity: Upconversion SFG at 771 nm; SPCM detection efficiency 63% at 771 nm; dark noise <100 Hz. Analog PD NEP ~3 pW/Hz^1/2 with 11 MHz bandwidth. FPGA-driven synchronized acquisition up to 250 MHz. - Demonstrated applicability by imaging features engraved on a silicon substrate under single-photon illumination, pertinent to noninvasive chip inspection. - Differential measurement (pattern/inverse) suppressed offsets and low-frequency illumination fluctuations, improving SNR and reconstruction fidelity.

The work directly addresses the long-standing challenge of achieving MIR single-photon-level single-pixel imaging at room temperature by combining two key functions within a single nonlinear device: (1) high-fidelity spatial modulation of MIR images via optically patterned near-infrared pumping, and (2) spectral upconversion of MIR photons into the visible to leverage efficient, low-noise silicon single-photon detectors. This removes reliance on MIR FPAs or MIR single-photon detectors, both of which are costly or impractical for room-temperature operation. The approach exploits the superior timing performance of single-element detectors and electronics compared to FPAs, enabling real-time operation and compatibility with time-of-flight and depth imaging. The results show that under ultralow photon flux (down to 0.5 photons/pulse) the system can reconstruct images by combining coincidence pulsed pumping, tight spectral filtering, and differential measurements to suppress temporal and spectral background noise. Compressive sensing with random masks further reduces measurement numbers, and plug-and-play deep denoising enhances reconstructions at extreme photon scarcity (1 photon/pulse with 25% sampling). The observed limitations in resolution and field of view stem from the phase-matching angular bandwidth and crystal aperture, but can be mitigated by optical design changes (shorter/wider crystals, chirped poling) and increased pump power. Overall, the findings validate nonlinear structured detection as a viable route for MIR single-photon single-pixel imaging and open pathways toward 3D/time-resolved and spectro-volumetric MIR imaging.

This study introduces and validates a nonlinear structured detection paradigm for MIR single-pixel imaging at the single-photon level. By imprinting high-resolution SLM-generated pump patterns onto MIR scenes via SFG in PPLN and upconverting MIR photons to the visible, the system enables room-temperature single-photon detection with silicon devices. The platform delivers: (i) real-time imaging at 10 Hz for 16×16 pixels, (ii) photon-sparse reconstructions down to 0.5 photons/pulse (with longer integration), and (iii) compressive imaging at 25% sampling, enhanced via a deep-learning denoiser to operate at 1 photon/pulse. These advances offer a low-cost, high-sensitivity alternative to MIR FPAs, with superior timing performance suitable for time-of-flight and depth imaging. Future directions include improving spatial resolution and field of view through crystal engineering (shorter/wider or chirped-poling crystals), increasing pump power to maintain conversion efficiency, integrating advanced reconstruction/denoising for lower sampling ratios and faster processing, extending to non-binary/non-sparse scenes, and leveraging tunable MIR sources for compressive 4D spectro-volumetric imaging.

- Spatial resolution is limited by the upconversion imaging system’s numerical aperture and the PPLN phase-matching angular bandwidth; this also constrains faithful mapping of high spatial frequencies, contributing to image blur. - Field of view is bounded by pump beam diameter and/or crystal aperture. Enhancing FOV may require a larger-aperture or shorter crystal and higher pump power. - Image quality degrades with increasing reconstructed pixel counts at fixed photon budgets, due to fewer photons per pixel and higher sensitivity to photon-counting (Poisson) fluctuations. - Longer integration times needed at very low photon flux increase contributions from dark counts/background, reducing SNR. - Demonstrations focus on binary, sparse transmission objects; while methods are general, performance on complex natural scenes may require more measurements and advanced reconstruction. - Phase-matching and narrowband operation constrain spectral flexibility; maintaining efficient conversion with larger FOV/resolution requires careful pump/temperature tuning and higher pump power. - Compressive reconstructions with stronger undersampling or lower photon counts demand heavier computation and longer processing times.