Single-shot 3D imaging with point cloud projection based on metadevice

Three-dimensional imaging captures and reconstructs spatial information and is foundational to applications in AI, VR, robotics, heritage conservation, and industrial inspection. Structured light techniques deliver fast, accurate, and short- to mid-range measurements but conventional projectors with refractive optics are bulky and complex to align, posing challenges for compact systems. Diffractive optical elements (DOEs) are limited in field of view due to large pixel sizes relative to wavelengths. Moreover, robust single-shot 3D imaging requires algorithmic designs tailored to the hardware for accuracy, speed, and data capacity. Metasurfaces, with subwavelength control over amplitude, phase, and polarization, offer miniature size, large numerical aperture, wide field of view, and multifunctionality, making them attractive for 3D imaging. However, image-formation metalenses face constraints in FOV, depth of field, and resolution. Active 3D imaging using metasurfaces has shown large FOV advantages over DOEs. This work addresses the need for a compact projector and a matching reconstruction algorithm enabling single-shot 3D imaging by using a single-layer metasurface to project a uniquely coded point cloud and a triangulation-based reconstruction with a tailored matching strategy.

Prior work demonstrates metasurfaces’ capabilities in holography, conformal optics, and beam shaping, with advantages of large NA and compactness. Metalens arrays and bifocal metalenses have enabled passive 3D positioning but suffer from limited FOV, depth of field, and resolution. DOE-based projectors typically provide smaller FOVs due to large pixel sizes. Metasurface-based periodic point cloud generators and Dammann grating analogs increase FOV but often provide limited diffraction orders or rely on VCSEL arrays. Integration of metasurfaces with laser sources promises compact, scalable on-chip devices. The present study leverages Fourier holography via geometric-phase metasurfaces to create a projection pattern with local uniqueness, enabling reliable correspondence without multiple patterns, and couples this hardware with a dedicated feature- and area-based matching pipeline and multiresolution search.

System concept: A single-layer geometric-phase metasurface projects a pseudorandom, locally unique point-cloud pattern in the Fourier (far-field) domain. Depth is recovered via triangulation by comparing the captured deformed pattern on the scene against calibrated reference and auxiliary planes using projective transform invariants (cross-ratio) to relate depth to pattern shifts/deformations. Pattern design: An M-array pseudo-random code generates a pattern with local uniqueness: any sub-window appears only once. The designed pattern has 1201 spots with 50% bright-spot area density. Local uniqueness is quantified by Hamming distance histograms (window size n=4), showing no zero-distance cases and <5% below 4, ensuring robust label discrimination. Metasurface hologram computation and encoding: A modified Gerchberg–Saxton (GS) algorithm computes the phase hologram to reconstruct the target dot pattern in the far field. The phase is discretized into eight levels for fabrication tolerance. A geometric (Pancharatnam–Berry) metasurface encodes the phase via nanopillar orientation. Metasurface design and fabrication: Material: amorphous silicon nanopillars on fused silica. Period: 316 nm; height: 600 nm. Operating wavelength for design/RCWA optimization: 633 nm. Chosen nanopillar lateral dimensions: length 180 nm, width 80 nm for high polarization-conversion transmission. Fabrication uses electron-beam lithography and reactive-ion etching, yielding a 1578 × 1578 nanopillar array. SEM validates structure quality. The projected holographic image closely matches the design with some speckle. Calibration and similarity: Practical illumination differs from ideal due to speckle and fabrication noise. Calibration uses one reference plane and two auxiliary planes to record the actual pattern and establish depth–shift relationships via cross-ratio. Inner-spot speckle textures exhibit high similarity across depths; zero-normalized sum of squared differences (ZNSSD) across labels exceeds 0.9, supporting fine correspondence within labels. Matching algorithm: A two-stage correspondence scheme is employed: - Initial feature-based label matching leverages spatial uniqueness. Handcrafted feature descriptors for each label are matched between deformed and reference images using cosine distance, with neighbor-label constraints defining a process path and ensuring surface continuity. - Fine area-based matching refines correspondences using speckle details within labels. With initial deformation parameters p from the coarse match, a local shape function W(x,y;p) is optimized to minimize dissimilarity between regions (adaptive subareas Ωi) using an inverse-compositional Gauss–Newton (IC-GN) solver, achieving pixel/sub-pixel accuracy while enforcing geometric continuity and outlier rejection via adaptive region selection. Multiresolution search: A pyramid strategy (I1 low, I2 medium, I3 high resolution via wavelet transform) and multiple z-spaced reference images (CR planes) enable coarse-to-fine depth estimation. Low-resolution images yield coarse depth maps to select the nearest candidate reference planes for subsequent higher-resolution refinement, improving accuracy and reducing uncertainty with limited computational overhead. Experimental setup: For depth-accuracy tests, a camera is mounted at 30° relative to the baseline, imaging objects at ~300 mm from the metasurface. Camera resolution: 2448 × 2048 pixels; lens focal length: 16 mm. Scenes include pairs of flat slabs with known thickness differences and deformable/gestural targets. Holographic reconstruction characterization uses circularly polarized illumination (linear polarizer + quarter-wave plate), objective (40×, NA=0.6), analyzer (opposite circular polarization), and CCD at the Fourier plane to capture k-space reconstructions.

- Achieved single-shot 3D reconstruction using a single-layer metasurface projector and tailored correspondence algorithms. - Depth accuracy: Maximum error approximately 0.2 mm; peak-valley (PV) error on planes less than 0.24 mm at 300 mm working distance. In five measurement groups (nominal steps 1.69, 2.00, 2.74, 3.69, 4.00 mm), absolute errors were 0.19, 0.01, 0.20, 0.12, and 0.02 mm, respectively. - Planeness metrics: Maximum PV = 0.24 mm; maximum RMS = 4.4 × 10^-4 mm, indicating low point-wise noise and high sub-pixel matching precision. - Pattern properties: Initial design with 1201 dots at 50% density; Hamming distance statistics confirm local uniqueness. ZNSSD similarity within spot interiors exceeds 0.9 across depths. - Robust reconstruction on challenging scenes: Successful 3D reconstruction of deforming, low-texture cardboard and discontinuous, variable-reflectivity gesture scenes (fingers/hands and background) without depth fuzziness due to adaptive matching constraints. - Spatial resolution scalability: Higher dot densities improve surface continuity and detail. Two additional metasurface samples (1 mm × 1 mm) with 6609 and 14768 projection dots enabled denser point clouds; Sample #2 (14768 dots) produced more continuous reconstructions than Sample #1. - Device efficiency: Experimental polarization conversion efficiency of the metadevice reaches 51% at 820 nm.

The study demonstrates that a single metasurface can project a uniquely coded point cloud enabling reliable, single-shot triangulation-based 3D imaging. By exploiting local pattern uniqueness and speckle features, the combined initial and fine matching algorithms deliver dense, accurate correspondences even on low-texture and discontinuous objects with varying reflectivity, addressing key limitations of passive stereo and traditional structured light systems. The multiresolution coarse-to-fine strategy improves depth accuracy and reduces uncertainty while maintaining computational efficiency. Compared to DOE-based projectors, the metasurface approach affords larger FOV, compact form factor, and flexible dot-density/FOV design. The quantitative accuracy (≤0.24 mm PV at 300 mm) validates the feasibility of sub-millimeter precision on a compact platform. These results suggest strong potential for applications in gesture recognition, surface inspection, and embedded depth sensing, where reduced alignment complexity and improved robustness are valuable.

This work introduces a compact, single-shot 3D imaging approach using a single-layer geometric-phase metasurface to project a locally unique point-cloud pattern in the Fourier domain, paired with a two-stage correspondence algorithm and multiresolution search. The system achieves sub-millimeter depth accuracy at 300 mm, robustly reconstructing low-texture and discontinuous scenes. The metasurface platform enables flexible FOV and dot-density design and can be integrated with light sources to realize compact metadevices. Future directions include on-chip integration with lasers for further miniaturization, multiplexing additional channels to increase dot counts and spatial resolution, algorithmic acceleration for higher throughput, and extending calibration and accuracy across broader depth ranges and scene scales.

- Current spatial resolution in some gesture reconstructions is limited by the number of projected dots; increasing dot density improves continuity but requires higher metasurface pixel counts and may increase computational load. - The multiresolution strategy trades some speed for accuracy and reduced uncertainty; although low-resolution stages are efficient, overall processing involves iterative optimization (IC-GN) and multiple reference planes. - Calibration requires one reference plane and two auxiliary planes, and performance depends on accurate system alignment and calibration stability. - Speckle and fabrication imperfections introduce pattern noise; while leveraged in fine matching, they can contribute to measurement drift on planar regions. - Depth accuracy is experimentally validated at a working distance of ~300 mm; generalization to wider ranges and different geometries requires further characterization. - The demonstrated setup uses separate illumination and metasurface components; full integration into a single compact source–metasurface module is proposed but not realized here.