Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field

The study addresses the long-standing trade-off in light-field imaging between depth-of-field (DoF) and spatial resolution. Conventional microlens-array-based light-field cameras either provide large DoF with low lateral resolution or higher resolution with reduced DoF. Inspired by the bifocal compound eyes of the trilobite Dalmanitina socialis, which simultaneously focus to near and far points, the authors propose a light-field camera that uses a spin-multiplexed bifocal metalens array to capture light-field information from both near and far depths concurrently. The purpose is to achieve extreme, continuous DoF without sacrificing spatial resolution. The work is significant for computational photography and imaging science, promising snapshot imaging that spans centimeter to kilometer scales while maintaining high image quality.

Early light-field cameras placed a microlens array at the primary lens focal plane to map angular information but suffered from low spatial resolution. Shifting the microlens array away from the focal plane improved lateral resolution at the expense of resolvable directions and DoF. Multifocal microlens arrays extended DoF by interlacing lenses of different focal lengths, but also reduced spatial resolution. Conventional DoF extension techniques (stopped-down apertures, focal sweeping, wavefront coding, stacked photodetectors) trade light throughput, temporal resolution, color fidelity, and imaging quality for DoF. Metasurface optics has enabled new sensing and imaging modalities (depth sensing, polarization imaging, quantitative phase imaging, angle-sensitive detectors, and achromatic metalenses for full-color light-field imaging). However, metasurfaces also face chromatic dispersion, limited aperture, and efficiency constraints. The proposed approach leverages spin-multiplexed metasurfaces and neural post-processing to overcome these trade-offs.

Design and theory: A photonic spin-multiplexed bifocal metalens array is designed using TiO2 nanopillars on SiO2. Each submetalens provides two independent phase profiles for left- and right-circularly polarized (LCP/RCP) light. The metasurface is modeled by a Jones matrix J(x,y) with eigenphases φL(x,y) and φR(x,y) that independently focus LCP and RCP components to different focal lengths fL and fR. The continuous phase for each channel follows φL,R(x,y)=2π/λ(√(x^2+y^2+fL,R^2)−fL,R). Unit cells are rectangular TiO2 nanopillars with optimized major/minor axes (D2/D1) and orientation θ to realize both propagation and geometric phase contributions for the two decoupled channels. Unit cell parameters: lattice period Px=Py=450 nm, height h=600 nm. Device and fabrication: A 39×39 close-packed array of square submetalenses (side d=150 μm; ≈110,000 nanopillars per sublens) is fabricated on fused silica. Design wavelength is 530 nm (green) to match the sensor’s Bayer filter and spectral sensitivity. Target focal lengths at 530 nm are fL=900 μm and fR=1250 μm. Fabrication uses e-beam lithography (positive resist, 600 nm thick; 2 nA, 100 kV), atomic-layer deposition of TiO2 at 90 °C, ICP-RIE etch (Cl2/BCl3), and liftoff in NMP. Optical characterization: A circularly polarized 530 nm beam illuminates the metasurface; focal lengths are measured as f1=(895±6) μm (LCP) and f2=(1243±9) μm (RCP). Measured FWHM of focal spots: (2.86±0.04) μm (LCP) and (3.96±0.06) μm (RCP), close to diffraction limits (2.83 μm and 3.92 μm). Broadband operation from 460–700 nm exhibits expected diffractive chromatic dispersion (wavelength-dependent focal shifts). Average transmission efficiency over the visible band is (72±1.5)% for unpolarized light. Average focusing efficiencies: (43.6±1.6)% (LCP) and (42.8±1.2)% (RCP), measured as power through a 10 μm pinhole at focus over incident power on a sublens. Camera construction: The metalens array is placed behind a primary imaging lens to form a light-field camera. The system parameters are optimized to yield two connected DoFs: the far boundary of the LCP DoF connects to the near boundary of the RCP DoF, forming a continuous DoF. Nominal distances: primary lens to metalens array L=47.5 mm; metalens array to sensor l=0.83 mm. Primary lens focal length F=50 mm; aperture D=6 mm. The forward imaging is modeled via Rayleigh–Sommerfeld diffraction with free-space propagation functions and phase terms for the primary lens and metalens. The polychromatic PSF is computed by integrating over the visible band weighted by the system’s spectral response and polarization conversion efficiencies. A PSF rank metric (PSFrank=Σσiωi/Ks) guides optimization subject to constraints: (i) focusing at infinity, (ii) angular sampling (repetition rate ≥3) for disparity estimation, and (iii) seamless LCP/RCP DoF touching. PSF measurement: A 100 μm pinhole illuminated with white light serves as a point source translated from 3 cm to infinity (infinity realized via collimation). PSFs are recorded for LCP, RCP, and unpolarized light across depths. Neural reconstruction pipeline: To correct spatially nonuniform chromatic/comatic and assembly-induced aberrations from the singlet metalens, a lightweight multiscale convolutional neural network (CNN) is trained. Training data generation involves measuring PSFs at multiple depths and field positions, augmenting the PSF set by rotations/resizing and linear combinations of phase-retrieved wavefront errors to create a diverse PSF space, and convolving clear images with augmented PSFs to form uniformly degraded pairs. The multiscale CNN aggregates features over different receptive fields to emulate biological neural compensation and correct diverse aberrations in a semiblind manner. After correction, standard light-field processing retrieves disparity maps and refocused images. The method is robust to reassembly and perturbations, requiring only an initial calibration. Test scenes and evaluation: 1) Matryoshka dolls placed at 0.3, 0.5, 1.0, 1.5, 2.3, and 3.3 m validate aberration correction and all-in-focus reconstruction. 2) USAF 1951 charts at depths from 3 cm to 5 m quantify resolution. 3) An outdoor scene includes objects at 3 cm, 0.35 m, 2 m, 10 m, 360 m, 480 m, and ≈1.7 km to demonstrate extreme DoF under natural light.

- Spin-multiplexed bifocal metalens array achieved two independent focal lengths at 530 nm with measured values fL=(895±6) μm and fR=(1243±9) μm, matching design (900 and 1250 μm). - Measured focal spot FWHM: (2.86±0.04) μm (LCP) and (3.96±0.06) μm (RCP), close to diffraction limits (2.83 and 3.92 μm), confirming high-quality focusing. - Broadband operation from 460–700 nm with average transmission efficiency (72±1.5)% for unpolarized light; focusing efficiencies (43.6±1.6)% (LCP) and (42.8±1.2)% (RCP). - The camera provides polarization-dependent DoFs that connect seamlessly: LCP yields high image quality from 3 cm to ≈2 m; RCP from ≈2 m to infinity. Combined (unpolarized) operation produces small PSF rank across 3 cm to infinity. - Deep CNN reconstruction effectively removes severe, spatially nonuniform aberrations, producing sharp, aberration-free all-in-focus light-field images and robust performance without retraining after reassembly. - Resolution across depth: USAF 1951 smallest resolvable line pairs are 5.04 lp/mm at 3 cm (group 2, element 3) and 0.89 lp/mm at 5 m (group −1, element 6). The angular resolution matches theoretical diffraction-limited performance across the visible (per Supplementary Fig. S13). - Extreme DoF demonstration: clear imaging of an outdoor scene covering 3 cm to ~1.7 km with high-quality all-in-focus reconstruction, simultaneously capturing macro and telephoto content in a single shot.

The findings demonstrate that spin-multiplexed bifocal metasurface optics can decouple near and far focusing via orthogonal circular polarizations, enabling a continuous extreme depth-of-field without sacrificing spatial resolution—addressing the core trade-off of conventional light-field cameras. Neural post-processing compensates for chromatic/comatic and assembly-induced aberrations typical of singlet metasurfaces, substantially relaxing design constraints that otherwise require complex achromatic meta-atoms. The integration yields full-color, snapshot light-field imaging with centimeter-to-kilometer DoF and diffraction-limited resolution, validating the trilobite-inspired concept. The approach opens avenues for compact, high-performance imaging in consumer photography, microscopy, and machine vision, where simultaneous macro and telephoto capabilities and robust computational correction are advantageous.

This work introduces a trilobite-inspired, spin-multiplexed bifocal metalens array integrated with a multiscale CNN reconstruction pipeline to realize a light-field camera exhibiting record continuous DoF from 3 cm to ~1.7 km with near-diffraction-limited resolution. The metasurface provides two decoupled polarization channels for near and far focusing, while neural processing removes severe, spatially varying aberrations, enabling high-quality full-color imaging and light-field rendering (disparity estimation and refocusing). Future research could focus on increasing focusing efficiency through improved nanofabrication and machine-learning-driven meta-atom design, enhancing achromatic performance without complex unit cells, scaling aperture and field-of-view, and integrating additional computational functions for broader applications in consumer and scientific imaging.

- Chromatic dispersion inherent to diffractive metasurfaces leads to wavelength-dependent focal shifts; color performance relies on computational correction. - Focusing efficiencies (~43%) and overall transmission (~72%) leave room for improvement; efficiency is sensitive to fabrication quality and unit-cell design. - Optical aberrations are significant due to singlet metasurface design and assembly tolerances, necessitating PSF calibration and neural correction (though the approach is robust post-training). - Performance depends on training data diversity and PSF augmentation; out-of-distribution aberrations may degrade results. - The approach leverages polarization channels; while natural light is un/partially polarized, splitting information between LCP/RCP can impact per-channel SNR depending on scene polarization and illumination.