Non-line-of-sight (NLOS) imaging, the technique of imaging objects hidden from direct view, has gained significant traction in recent years. Applications span diverse fields, including robotic vision, autonomous driving, rescue operations, remote sensing, and medical imaging. A typical NLOS scenario involves a relay surface where light reflects off the hidden object and then reaches a sensor. Current NLOS imaging techniques, however, often rely on dense measurements taken at regular intervals across a large relay surface. This requirement severely limits their practicality in variable and unpredictable environments. This paper addresses this limitation by proposing a novel Bayesian framework for NLOS imaging that operates effectively with arbitrary illumination and detection patterns. The framework's robustness and versatility are achieved through the incorporation of virtual confocal signals and a sophisticated regularization algorithm, opening up new possibilities for NLOS applications where the relay surface is small, irregular, or only partially visible.

Existing NLOS imaging algorithms generally fall into three categories based on their representation of the hidden surface: point-cloud-based, mesh-based, and voxel-based. Voxel-based methods, offering a balance of efficiency and accuracy, are predominantly used. Early voxel-based methods, such as back-projection algorithms, model the measured photon intensity as a linear operator acting on the albedo. Subsequent improvements incorporated rendering techniques for speed and filtering techniques for noise reduction. The light-cone transform (LCT) and its directional extension (D-LCT) formulate the problem as a deblurring task, while the frequency-wavenumber migration (F-K) method utilizes the wave equation. However, LCT, D-LCT, and F-K methods are primarily suitable for confocal setups. The phasor field (PF) method directly addresses non-confocal settings, modeling the NLOS detection as diffractive wave propagation. The signal-object collaborative regularization (SOCR) method employs priors on both the target and the measured signal to enhance reconstruction quality. While advancements have been made, these methods typically require large relay surfaces and dense measurements, limiting their applicability to real-world scenarios. Some recent works have explored sparse measurements, demonstrating that it is possible to reconstruct hidden scenes with confocal or circular scans, or by incorporating compressed sensing techniques. However, these approaches still face challenges when dealing with irregular or very small relay surfaces.

This paper introduces a Bayesian framework for NLOS reconstruction applicable to any spatial illumination and detection point pattern. The core innovation is the introduction of virtual confocal signals at regular grid points. These signals, along with the measured signals and an estimated signal, are jointly optimized within a Bayesian framework using a novel algorithm called Confocal Complemented Signal-Object Collaborative Regularization (CC-SOCR). The CC-SOCR framework treats the reconstructed target (u), measured signal (b), estimated signal (b), and virtual confocal signal (d) as random vectors. The reconstruction is achieved by maximizing the joint posterior probability P(u, b, d|b). This maximization is framed as an optimization problem involving three key assumptions. First, the measured signal's conditional distribution depends only on the target and the estimated signal. Second, the joint prior distribution of the target and estimated signal incorporates regularization terms. Third, the conditional distribution of the virtual confocal signal depends on the target, estimated signal, and a regularization term enforcing consistency between confocal and non-confocal signals. The resulting optimization problem is solved using an alternating iteration method, iteratively updating the target, signals, and regularization parameters. The regularization terms in the optimization problem incorporate sparsity and non-local self-similarity priors for the target, Wiener filtering for the measured signal, and a data-driven orthogonal dictionary for the virtual confocal signal, which is used to enhance the reconstruction. These regularization terms are carefully designed to handle noise and incomplete data, allowing for high-quality reconstructions even with very limited measurements.

The CC-SOCR method demonstrates superior performance compared to existing methods, including Laplacian of Gaussian filtered back-projection (LOG-BP), F-K, LCT, D-LCT, PF-BP, PF-RSD, and SOCR. The authors conducted extensive experiments using both synthetic and real-world data. In experiments using synthetic data simulating a pyramidal target and a square relay surface with sparse measurements, CC-SOCR successfully located the target and reconstructed its albedo and surface normal with high accuracy, while the LOG-BP method failed. Experiments using real-world confocal data from the Stanford dataset demonstrated CC-SOCR's capability to reconstruct a statue with high fidelity, even when using only a small fraction of the available measurements or with measurements taken from an irregular relay surface (such as a horizontal shutter, a set of window edges, or a fence). Results on non-confocal data from the phasor field method dataset showed that CC-SOCR provided faithful reconstructions even under highly incomplete measurement scenarios, while the other methods struggled with artifacts or noise. Experiments with non-planar relay surfaces, using data from the Stanford dataset, further showcased the method's ability to reconstruct hidden objects, demonstrating its superior robustness to variations in relay geometry. In all scenarios, CC-SOCR consistently exhibited higher reconstruction quality and robustness compared to other methods in terms of classification error (percentage of excessive and missing voxels), noise, and detail preservation.

The CC-SOCR method significantly advances the field of NLOS imaging by addressing the long-standing limitations of requiring dense, regularly spaced measurements on large relay surfaces. The introduction of virtual confocal signals and the sophisticated Bayesian framework allow for high-quality reconstructions even with coarse or irregularly distributed measurements, significantly extending the applicability of NLOS imaging to diverse scenarios. The results consistently outperform existing methods in terms of accuracy, robustness, and efficiency. The flexibility and adaptability of the proposed framework suggest a paradigm shift, freeing NLOS imaging from its reliance on specific relay surface characteristics, thus expanding the potential applications of the technique.

This paper presents a novel Bayesian framework for NLOS imaging, the CC-SOCR method, which dramatically improves robustness and efficiency by eliminating the need for dense, regularly spaced measurements. The use of virtual confocal signals and effective regularization techniques enables accurate reconstructions of hidden objects even under challenging conditions. Future research directions could explore incorporating more sophisticated priors, extending the framework to handle more complex scenarios (e.g., multiple scattering), and optimizing the computational complexity further.

While the CC-SOCR method demonstrates remarkable performance, there are some limitations. The computational complexity, although manageable for many scenarios, can still be high for extremely large datasets. Global convergence of the optimization algorithm is not guaranteed, although convergence of the subproblems is ensured. The accuracy of the reconstructions depends on the fidelity of the physical model used; deviations from the assumed model may impact the results.