Lensless light-field imaging through diffuser encoding

The paper addresses the challenge of capturing angular information lost in conventional 2D imaging by proposing a lensless light-field imaging modality using a diffuser. Light-field imaging captures both spatial and angular information, enabling capabilities such as viewpoint shifting, post-capture refocusing, depth sensing, and depth-of-field extension. Existing systems include multi-camera arrays (bulky, expensive), single-sensor multi-exposure methods (high resolution but time-consuming and unsuitable for dynamics), and single-sensor single-exposure encoders (e.g., plenoptic cameras with microlens arrays). Classical microlens-based plenoptic imaging suffers from an intrinsic trade-off between spatial and angular resolutions. The study aims to overcome this trade-off by using a diffuser to encode angularly distinct sub-beams into a unique pseudorandom pattern, enabling lensless capture of 4D light fields and computational reconstruction with adjustable spatio-angular resolution beyond sensor pixel count limits.

The authors situate their work within light-field imaging approaches: early plenoptic concepts (pinhole or microlens arrays) and modern microlens-based cameras (e.g., Lytro, Raytrix) that directly sample light rays but face spatial–angular resolution trade-offs. Alternative computational approaches encode each light ray across multiple pixels using modulation masks (attenuation masks and diffuser plates), avoiding the classical trade-off by reconstructing light fields computationally. Imaging through scattering media has shown that object information is encoded (not lost) in speckle-like patterns and exhibits lateral shift invariance and depth scaling properties. The proposed method leverages diffuser-based encoding and these scattering-media properties for 4D light-field capture without lenses.

System concept: A thin phase diffuser with random surface statistics serves as an angular encoder. A temporally incoherent point source within the field of view produces on the sensor a high-contrast pseudorandom pattern composed of non-overlapping sub-images, each arising from an angularly distinct sub-beam incident on a small diffuser region. Due to the diffuser’s lateral shift invariance, translating the point source in a plane parallel to the diffuser shifts the pattern accordingly, enabling a two-plane light-field parameterization l(s,u). Modeling: The mapping from 4D light fields to the 2D sensor image is modeled linearly. For a single point source i, the vectorized pattern f_i is given by f_i = H_i g_i, where g_i represents the single-point light field (angular samples), and H_i stacks sub-image response vectors t_ij corresponding to the j-th light ray. For multiple points, the sensor measurement d is d = T l, where T = (H_1, H_2, …, H_n) is the light-field transmission matrix encoding contributions from all spatial–angular samples. A single sensor pixel can integrate contributions from multiple light rays, allowing the total number of reconstructable rays to exceed the pixel count. Calibration: A key step is determining the transmission matrix T via a calibration using a point-source-generated pattern. Because the diffuser response is shift-invariant and uniquely encodes each incident angle/position pair, one captured pattern (from a single point source at a known plane) suffices as a basis to flexibly construct T for adjustable spatial and angular sampling grids. Depth scaling of patterns (a property of scattering-media imaging) can be used to adapt calibration across object depths by appropriately rescaling the base pattern when necessary. Reconstruction: Given a raw single exposure d and the calibrated T, the 4D light field l(s,u) is computationally decoupled via iterative optimization (200 iterations, approximately 2.4 s per iteration reported in one setup). Digital refocusing is then applied to the reconstructed light field to generate focal stacks and analyze resolution and depth. Experimental protocol included capturing 12 calibration patterns by axially translating a point source; one (Pattern 6) served as the calibration base for most reconstructions. Demonstrations were performed for sparse distributed point objects and for area objects (USAF-1951 target), with evaluations under different spatial and angular sampling configurations such as 512×512×6×6, 512×512×12×12, and 1024×1024×6×6.

- Lensless diffuser encoding effectively maps 4D light fields to 2D sensor images and allows computational decoupling of more light rays than sensor pixels (breaking sensor-resolution limitations). - Single-point calibration suffices to determine the transmission matrix; shift invariance and depth scaling enable flexible calibration across planes. - Demonstrated reconstruction of approximately 9.4 million light rays (512×512×6×6) from a 0.26-megapixel image. - Runtime and resolution trends for point objects: - Resolution 1 (512×512×6×6): runtime 8.00 min (200 iterations, ~2.4 s/iteration); resolved size 10.09 (normalized width metric). - Resolution 2 (512×512×12×12): runtime 32.56 min; resolved size 5.51. - Resolution 3 (1024×1024×6×6): runtime 32.85 min; resolved size 19.99 (apparent doubling due to denser spatial sampling; real resolved sizes comparable to Resolution 1 when accounting for sampling). - Runtime scales approximately with the total number of decoupled light rays. - Increasing angular sampling (e.g., 6×6 to 12×12) effectively reduces the resolved size for point objects; increasing spatial sampling primarily increases runtime with limited benefit for very small objects (15 µm pinhole used). - Multi-point scene (simulated from shifted patterns): Digital refocusing yields in-focus slices at correct depths; resolved size increases with the axial distance between the measured point and the calibration point-source depth. - Area objects (USAF target): Increasing angular sampling (6×6, 8×8, 12×12) did not improve reconstruction appreciably, unlike the point-object case. Depth maps were computed via focal stacks. - Depth adaptation via scaling: For a different working distance (target at 25 mm), acceptable reconstructions required rescaling the calibration pattern (Pattern 6) to emulate a calibration at the new depth, consistent with the scaling property of scattering-media imaging.

The findings show that diffuser-based encoding can uniquely and linearly map spatial–angular light-field information to sensor intensities without lenses, enabling single-shot 4D capture and computational reconstruction. By calibrating a transmission matrix from a single point-source pattern, the system decouples light rays at adjustable spatio-angular sampling densities that are not constrained by the sensor’s pixel count. This addresses the classical spatial–angular trade-off seen in microlens-based plenoptic imaging. Performance analyses clarify operational regimes: higher angular sampling enhances resolution for sparse point-like scenes, while gains for extended area objects are limited under the tested conditions. The dependence of resolved size on axial offset from the calibration depth and the need for depth scaling highlight the role of accurate depth-aware calibration. The successful digital refocusing, depth estimation, and multi-depth separation of sparse points validate the model and suggest applications in lensless 3D/4D imaging using scattering media.

The study introduces a lensless light-field imaging modality that uses a diffuser to encode angular information and a calibrated transmission matrix to reconstruct 4D light fields from a single 2D exposure. Contributions include: (1) a diffuser-encoding transmission model mapping 4D light fields to 2D images, (2) a practical calibration strategy based on a single point-source pattern, and (3) demonstrations of high light-ray counts reconstructed beyond sensor pixel limits, with digital refocusing and depth recovery for both sparse points and area targets. The proof-of-concept establishes scattering media as viable encoders for high-dimensional optical signal acquisition. Future work could explore improved and depth-robust calibration strategies, faster and more robust reconstruction algorithms for dynamic scenes, optimized diffuser designs, and quantitative resolution/throughput trade-off analyses across diverse object classes.

- Reconstruction quality depends on calibration depth; resolved size degrades as the axial distance between the measured object and the calibration point-source plane increases. - Acceptable reconstructions at different depths may require rescaling or re-acquiring calibration patterns (depth scaling adaptation). - Computational cost is significant; runtime scales with the number of reconstructed light rays (e.g., ~8–33 minutes for tested configurations with 200 iterations). - Increasing spatial sampling substantially increases runtime and did not improve resolution for very small point objects in the tested setup. - For extended area objects, increasing angular sampling (from 6×6 to 12×12) did not yield noticeable improvements under the reported conditions.