Multi-functional imaging inspired by insect stereopsis

Stereopsis enables perception of spatial and temporal variations and underpins functions such as prey capture, navigation, and predator evasion in insects. While apposition and superposition compound eyes often yield limited depth accuracy due to few photoreceptors per facet, the unusual Xenos peckii eye achieves multi-view stereopsis with higher acuity via chunk-sampled images. Technologically, multi-aperture and compound-eye-inspired cameras capture arrays of scenes with parallax for stereoscopic imaging, but are constrained by limited depth-of-field, bulk, and especially low resolution, limiting exploration of insect-like functions. This study addresses these gaps by proposing an ultrathin microlens array camera (MAC) that emulates insect stereopsis to achieve multifunctional imaging across object distances—microscopy at near plane, 3D depth at mid plane, and HDR and high-speed imaging at far plane—within a compact, insect-eye-scale form factor.

Prior work on stereopsis and insect vision has established behavioral and neural bases for depth perception and motion processing in insects, including mantis stereopsis and optic flow for collision avoidance. Artificial compound eye cameras provide wide field-of-view and optic flow imaging but suffer from low resolution and limited DOF. Light-field and camera array systems enable stereoscopic reconstruction and refocusing yet are bulky and not insect-scale. Ultrathin arrayed cameras inspired by Xenos peckii demonstrated improved contrast and resolution on flat sensors. Achromatic metalens arrays and hyperspectral imaging approaches have expanded computational imaging capabilities. Remaining challenges include achieving high-resolution, multifunctional imaging across distances in compact, simple hardware, motivating the MAC approach.

System design: The MAC integrates a window glass, multilayer aperture arrays (MAAs), inverted microlens arrays (iMLAs), and alumina spacers, all packaged on a flat CMOS image sensor (Sony IMX219) with an image processing board (Raspberry Pi 3 B+). iMLAs with MAAs set viewing angle and enable short focal length and ultrathin packaging. The demonstrated lens diameter choice was 150 µm (F/1.7), balancing illumination, inverse MTF, and track length (total track length ~810 µm). Fabrication: MAAs were formed via repetitive photolithography of negative-tone black resin (Gersteltec GMC 1040) and SU-8 on a 4-inch borosilicate wafer with atmospheric plasma treatment to enhance adhesion. Positive-tone AZ9260 cylinders were patterned and reflowed upside-down to form F/1.7 spherical microlenses with higher curvature. The lens plate was diced and assembled onto the IMX219 using a flip-chip bonder. Epoxy adhesive was dispensed, alumina spacers (precision ground and micro-sawn) stacked to match focal length, and the lens plate aligned over the active pixel area. Final cure was at 120°C for 30 min. Optical characterization: Confocal laser scanning microscopy (532 nm) optically sectioned the microlens focus, yielding focal spot FWHM diameter 1.32 µm and DOF 19.8 µm. Packaging tolerance from spacer/glue thickness error was ~12 µm, within DOF. MTF and relative illumination were measured across microlens diameters at constant F-number (F/1.7). A 1951 USAF chart assessed resolution. Imaging setups: For distance-dependent imaging, a displayed flower on an LED panel was imaged at varying object distances and compared to a commercial single-lens camera (CSLC; Raspberry Pi V2, F/2, DLENS 1.5 mm). For microscopy, specimens (pine tree stem, pumpkin stem, mouse small intestine, red onion epidermis) were placed at the MAC top glass (500 µm window thickness). For 3D, multi-channel images were captured at different depths and disparities computed across channels. For HDR at far plane, channel images were merged. For high-speed imaging, a rotating fan and a 6-color disc were imaged to evaluate rolling shutter distortion and effective frame rate. Image reconstruction: 3D depth maps were generated via stereo matching with fast cost-volume filtering: constructing a disparity cost volume, smoothing with bilateral/guided filters, assigning minimum-cost disparities, rectifying, and filling with a weighted median filter; grayscale depth maps were pseudo-colored. HDR reconstruction used exposure fusion (Chasys Draw IES): channel image registration, brightness-based weight maps, reduced weights for under/overexposed regions via Laplacian/Gaussian blending, and pixel-wise weighted fusion. Processing ran on a desktop (Intel i5-6600K, GTX 960); depth reconstruction ~10 s, HDR merge ~1 s.

• All-in-focus imaging across distances: Due to short focal length and ultracompact track length, MAC maintains focus from near to far planes and reduces minimum object distance. Compared to CSLC, MAC achieved higher MTF50 for object distances <130 mm.
• Resolution: Resolved 1951 USAF Group 6, Element 2 (6.96 µm) with contrast >0.3; microlens diameter 150 µm selected (iMTF50 = 9.63 µm; total track length ≈810 µm). Microscopy demonstrated clear cellular/tissue structures when the specimen contacted the top glass.
• 3D depth imaging (mid plane): Pixel disparity increased as object distance decreased and as channel period increased. Depth resolution defined where disparity change equals a pixel: examples include ~3.7 µm at 2.5 mm object distance and ~790 µm at 30 mm. Red–cyan anaglyphs and reconstructed 3D depth maps validated multi-view stereopsis in both x and y axes.
• HDR imaging (far plane): Channel images (same viewing direction) merged via HDR produced a single high-contrast image with grayscale dynamic range increased by 1.61× and color gamut expanded by 2.13× relative to a single channel. Post-merge contrast reached 0.72, 1.09× higher than a single-channel image and 86% of a comparable CSLC.
• High-speed imaging: Using rolling-shutter frame fragmentation across channels, MAC captured sequential moments without motion artifacts. In a rotating fan test, channels along a column sampled every ~2.89 ms (≈25° rotation between captures); estimated fan speed 1441 rpm. Effective frame rate scaled with microlens diameter: up to 345 fps (100 µm lens diameter), achieving 11.5× the CSLC’s 30 fps and >90% reduction in rolling shutter distortion. Distortion (arc length differences) decreased as lens diameter decreased.

The MAC leverages insect-like stereopsis with multiple microlens channels to extract spatial parallax and temporal sampling while remaining ultrathin. Its short focal length and constant per-channel FOV enable all-in-focus capture from near to far, addressing depth-of-field limitations of prior systems. At near distances, fragmented yet sharp channel views can be stitched for wide-field microscopy; at mid distances, inter-channel disparities yield accurate 3D depth maps; at far distances, redundant views facilitate HDR fusion and mitigate rolling shutter artifacts for high-speed scenes. The quantitative gains in depth resolution, dynamic range, color gamut, and motion distortion reduction substantiate the approach. There is an inherent trade-off between lens diameter and performance across planes (illumination/MTF, frame rate, distortion), suggesting application-specific optimization. Overall, the results advance understanding of insect-inspired vision principles and demonstrate practical, compact multifunctional imaging.

An ultrathin microlens array camera inspired by insect stereopsis was demonstrated to deliver multifunctional imaging—near-plane microscopy, mid-plane 3D depth, and far-plane HDR and high-speed imaging—using a single compact device. The channel-array configuration provides clear, all-in-focus images with variable visual disparities, enabling reconstruction-based functionalities with low-cost hardware. This approach offers insights into insect visual processing and a path to compact imaging tools for healthcare, mobile, and surveillance applications. Future work includes re-optimizing microlens dimensions and packaging for specific imaging planes, improving CRA matching and color fidelity, accelerating depth reconstruction, and integrating on-sensor or embedded processing for real-time operation.

• Trade-offs across imaging planes: Performance depends on microlens diameter; optimizing for one plane (e.g., higher frame rate with smaller lenses) can impact illumination and MTF.
• Chief ray angle (CRA) mismatching across lens diameters caused slight color differences in high-speed tests.
• Reconstruction latency: 3D depth reconstruction required ~10 s on a desktop GPU/CPU; real-time processing was not demonstrated.
• Depth resolution degrades with increased object distance and depends on channel period and overlap.
• HDR contrast, while improved over single-channel MAC, remained below that of a comparable CSLC (86% of CSLC contrast in tests).
• System parameters (e.g., window glass thickness) influence magnification and may require customization per application.