Full-colour 3D holographic augmented-reality displays with metasurface waveguides

The study addresses key limitations of current AR glasses—bulky projector optics and the inability to present accurate 3D depth cues—by proposing a compact holographic optical-see-through (OST) system. Waveguide image combiners are the leading approach for near-eye displays due to their compactness, but traditional systems require projection optics with thickness proportional to lens focal length and are largely limited to 2D images at a fixed focal distance, leading to vergence–accommodation conflict and reduced realism. Holography promises perceptually realistic 3D imagery via thin optical films, yet prior attempts have not achieved both compact form factors and high-quality full-colour 3D imagery suitable for AR. The authors propose a co-designed system: a lensless holographic light engine tightly integrated with a dispersion-compensated, inverse-designed metasurface waveguide and AI-driven holography algorithms to deliver full-colour 3D content in a glasses-like form factor.

The paper situates its contribution within AR/VR display research, referencing foundational work on AR, holography, and head-worn displays. Waveguide combiners have become industry standard for AR but face challenges in depth presentation and optical bulk. Prior holographic near-eye displays and waveguide holography showed promise but lacked compactness or suffered poor image quality, particularly for see-through settings. Metasurfaces, empowered by inverse design and deep learning, have demonstrated superior diffraction efficiency, spectral selectivity, Q-factor, and transmittance across optical applications, including AR eyepieces, but device-level dispersion-engineered metasurfaces may lack sufficient degrees of freedom for full-colour, uniform AR performance. Recent camera-in-the-loop (CITL) neural holography approaches accurately learn system aberrations for benchtop VR displays; this work extends such learned propagation modeling to full-colour, coherent wavefront propagation through an OST waveguide, integrating it with a co-designed metasurface waveguide.

Hardware/optical co-design: The authors design an OST metasurface waveguide that maximizes compactness, dispersion correction, transmission efficiency, and angular uniformity. They select a high-index all-glass metasurface platform (e.g., SF6, n > 1.8, k ≈ 0) to ensure total internal reflection (TIR) for RGB at high diffraction angles, enabling single-layer in/out couplers and minimizing boundary reflections/interference. System-level dispersion compensation is achieved via geometric design and k-vector matching: in- and out-couplers share equal magnitude but opposite momentum to suppress observable dispersion. Waveguide geometry (thickness d_wg and coupler spacing) is engineered so RGB replicas laterally displace to meet at the out-coupler after different numbers of internal reflections (R/G/B = 1/3/5), using period Λ = 384 nm and thickness d_wg = 5 mm. Metasurface inverse design: Using rigorous-coupled-wave analysis (Fourier modal method) with gradient descent (Adam), the team optimizes 1D grating profiles (assuming x-symmetry) for TE polarization to maximize first-order diffraction efficiencies at 638/521/445 nm and to minimize efficiency variance over ±5° incident angles, with Gaussian blur to model fabrication tolerances. The optimized double-lined grating geometry (period Λ = 384 nm, height H = 220 nm) efficiently steers high-angle diffraction for RGB with improved angular uniformity. Fabrication: All-glass metasurface couplers are patterned directly on high-index glass (SF6) to mitigate sensitivity to surface irregularities/contamination. A reverse-patterning, etch-based process avoids lift-off residues and etch-induced irregularities: deposit 30 nm Cr (e-beam), spin PMMA (950 A4), e-beam lithography (50 kV), use ICP-RIE to etch Cr with PMMA mask, RIE to etch glass with Cr mask (Cl2, O2, CHF3, CF4, Ar; He backside cooling), then remove Cr by ICP-RIE. SEM imaging confirms nanoscale fidelity. Physical and learned propagation model: The field coupled into the waveguide u_ic is formed by the SLM phase, converging illumination of focal length f_um, and in-coupler aperture. A physically motivated waveguide transfer function H_WG (from RCWA-simulated first-order reflection and coupler translation) and out-coupler aperture a_oc model the out-coupled field u_oc. Free-space propagation models the image at target depth d_target. To bridge model–hardware gaps (aberrations, fabrication error, SLM/electro-optic nonidealities), the authors augment the model with learned components: complex-valued coupler efficiencies, and CNNs at the in-coupler and target planes (modified U-Net: 32 channels, 5 down/5 up, instance norm; LeakyReLU down, ReLU up; skip connections). This yields a fully differentiable, camera-calibrated waveguide model for CGH. Experimental setup: Prototype uses HOLOEYE LETO-3 phase-only SLM (1080×1920, 6.4 µm pitch), FISBA READYBeam RGB lasers (638/521/445 nm), slight SLM/illumination tilt to avoid pre-coupled stray light, and a FLIR Grasshopper3 12.3 MP color camera with Canon EF 35 mm lens (Arduino-driven focus) to capture calibration and results. Holograms are synthesized via gradient descent CGH incorporating the learned propagation model. Partially coherent operation multiplexes a few coherent modes in time to reduce speckle and improve 3D depth-of-field realism. Training: Dataset pairs SLM phase patterns with captured intensity images at four planes: 0 D (∞), 0.33 D (3 m), 0.67 D (1.5 m), 1.0 D (1 m). Initial phase sets are generated from the physical model to render DIV2K images at various virtual distances, with randomized CGH parameters (learning rates, initial phases, propagation distances) to broaden angular spectra. 10,000 patterns per RGB channel captured; split 8:1:1 for train/val/test. Training on NVIDIA RTX A6000 (48 GB), batch size 1, learning rate 3e-4. Two refinement rounds (self-synthesized data) further improve experimental fidelity. Post-training, the model enables hologram synthesis without additional camera feedback.

- Compact full-colour 3D OST waveguide holography: Demonstrates high-quality full-colour multiplane 3D holography through a single metasurface waveguide with a lensless light engine. - High see-through efficiency and angular uniformity: Inverse-designed all-glass metasurface couplers achieve ~78.4% visible transmittance (see-through efficiency). Uniformity (min/max over viewing angles) for the inverse-designed metasurfaces: Red 61.7%, Green 91.2%, Blue 98.3%, outperforming conventional gratings (Red 58.9%, Green 47.7%, Blue 88.8%). - Efficient high-angle diffraction: Double-lined metasurface gratings with Λ = 384 nm and H = 220 nm steer RGB first-order diffraction at high angles suitable for TIR in high-index glass; high-index design (n > 1.8) supports broadband TIR and improved light–matter interaction. - Dispersion-compensated waveguide geometry: With d_wg = 5 mm and Λ = 384 nm, in-coupled RGB waves meet at the out-coupler after 1/3/5 internal reflections, respectively, enabling full-colour out-coupling with minimal dispersion. - Learned propagation model accuracy: Camera-calibrated, physics-informed CNN model improves captured 2D hologram quality by ~3–5 dB PSNR over free-space and purely physical baselines, and yields superior full-colour 3D results (in- and out-of-focus regions) including optical-see-through scenes combining real and virtual content. - Field of view: Current prototype field of view is 11.7°, comparable to many commercial AR systems. - Practical prototype: HOLOEYE LETO-3 SLM; RGB lasers at 638/521/445 nm; partial coherence multiplexing; calibration and runtime CGH leveraging the differentiable model.

The co-designed metasurface waveguide and AI-driven holography address key AR constraints by eliminating bulky projection optics (placing the phase-only SLM close to the in-coupler) and by producing accurate, multiplane 3D content that mitigates vergence–accommodation conflict. The inverse-designed all-glass metasurface couplers deliver high see-through transmission and uniform angular response, crucial for OST AR. The physics-informed learned propagation model captures system-specific aberrations and nonidealities, enabling significantly improved hologram quality over free-space and purely analytic models and establishing a practical path for compact, high-fidelity 3D AR. Future enhancements could broaden the field of view beyond 11.7° by adopting higher index materials or integrating a metasurface eyepiece into the out-coupler. Reducing waveguide thickness is tied to SLM size; emerging SLMs with ~1 µm pixel pitch could enable ultrathin waveguides. Étendue constraints imposed by SLM space–bandwidth product may be alleviated by adapting étendue expansion; integrating an illumination waveguide could further compactify the light path. Algorithmically, accelerating CGH via learned real-time inversion could deliver interactive operation.

This work demonstrates, for the first time to the authors’ knowledge, a compact, full-colour 3D optical-see-through AR display via a co-designed, dispersion-compensated metasurface waveguide and a camera-calibrated, physics-informed learned propagation model. The system achieves high see-through efficiency, excellent angular uniformity, and substantial image quality improvements, producing realistic full-colour 3D content in a wearable form factor. The approach charts a path toward practical holographic AR glasses. Future research should target larger field of view, thinner waveguides via smaller-pitch SLMs, étendue expansion compatible with full-colour 3D waveguide holography, integration with illumination waveguides, and real-time CGH synthesis using machine learning.

- Field of view limited to 11.7°, necessitating higher-index materials or added metasurface eyepieces for expansion. - Waveguide thickness is directly proportional to SLM size; current SLM pixel pitch (6.4 µm) constrains thickness; smaller-pitch SLMs are not yet commercially available. - Computational latency: CGH generation currently takes several minutes per phase pattern; no runtime optimization implemented. - Étendue limited by SLM space–bandwidth product; étendue expansion techniques for full-colour 3D waveguide holography remain to be demonstrated. - Diffraction efficiency for the red channel is harder to optimize due to very large diffraction angles, leaving room for improvement. - Sensitivity of holographic waveguides to surface irregularities/contamination necessitates careful fabrication; while addressed via all-glass process, it remains a practical consideration.