Augmented reality (AR) systems overlay digital information onto the user's view of the real world, offering transformative potential across various fields. However, widespread AR adoption is hindered by limitations in current display technologies, such as bulky projection optics and the inability to accurately represent 3D depth cues. This leads to visual discomfort and limits the realism of virtual content. Waveguide image combiners represent a promising solution for compact AR glasses, but existing designs still require bulky projection optics and are restricted to 2D images at a fixed distance. Holographic displays offer a potential solution, providing perceptually realistic 3D content using ultra-thin films. Previous attempts to integrate digital holography into AR displays, however, have not achieved the necessary compactness and image quality. This research presents a novel AR display system that addresses these limitations. The system's compact form factor, achieved through a unique combination of hardware and software components, addresses the need for comfortable and stylish all-day use AR glasses. The system aims to deliver visually compelling 3D AR experiences by resolving issues of bulk, limited depth cues and visual discomfort associated with current technologies. This will significantly impact several fields, including entertainment, education, communication, and training, broadening the applicability of AR technology.

The authors review existing AR display technologies, highlighting the limitations of current waveguide designs, which often require bulky projection optics and are limited to 2D image display at a fixed distance. They discuss the potential of holographic displays to generate realistic 3D content using thin optical films, but note that previous attempts to adapt digital holography to AR configurations have fallen short in terms of compactness and image quality. The use of metasurfaces for enhanced diffraction efficiency, spectral selectivity, Q-factor, and transmittance in AR applications is also considered. Previous work on waveguide holography for non-see-through virtual reality settings is acknowledged, but its poor image quality is highlighted as a key challenge.

The researchers developed a novel AR display system combining a lensless holographic light engine with a metasurface waveguide optimized for full-color optical-see-through (OST) AR applications. The system uses inverse-designed metasurface grating couplers to achieve high uniformity and see-through efficiency in relaying full-color 3D holographic images. The high-index glass waveguide material (n > 1.8) is chosen to ensure total internal reflection (TIR) for all visible wavelengths, minimizing boundary reflections and interference. Chromatic dispersion is corrected at the system level through geometric design of the waveguide and k-vector matching of the input and output couplers, enabling propagation of broadband light with TIR. The metasurface grating geometry is optimized using a rigorous-coupled-wave-analysis solver, maximizing diffraction efficiency and uniformity of angular response. The fabrication process, involving electron beam lithography on high-index glass, is described, emphasizing steps taken to avoid contamination or surface damage. A novel wave propagation model is developed, incorporating both physically accurate modelling techniques and artificial intelligence components (Convolutional Neural Networks or CNNs). The learnable parameters of this model are automatically calibrated using camera feedback. This combined physical-AI model accurately accounts for the nuances of the optical system, including nanoscopic variations and fabrication errors, ultimately improving the quality of synthesized holograms. The experimental setup involves a phase-only spatial light modulator (SLM), a fiber-coupled RGB laser diode light source, and a high-resolution color camera for calibration data acquisition. Holograms are computed using a gradient descent computer-generated holography (CGH) algorithm incorporating the camera-calibrated wave propagation model.

The inverse-designed metasurface waveguide demonstrated high see-through efficiency (approximately 78.4% in the visible spectrum) and uniform transmittance regardless of the angle of incidence. The experimental results show that the AI-augmented wave propagation model significantly outperforms baseline models (free-space propagation and a purely physical waveguide model), achieving a 3-5 dB improvement in peak signal-to-noise ratio for 2D images. Full-color 3D holographic images were successfully generated, demonstrating high image quality in both in-focus and out-of-focus regions, mitigating the vergence-accommodation conflict. Optical see-through AR operation with digitally overlaid content on real-world scenes was also demonstrated. The prototype achieved a field of view of 11.7°, comparable to many commercial AR systems. The relationship between waveguide thickness, SLM size, and field of view was derived, suggesting potential for further miniaturization using smaller SLMs.

The co-design of the metasurface waveguide and AI-driven holography algorithms resulted in a compact, full-color 3D holographic OST AR display system, surpassing existing waveguide designs in both form factor and image quality. The achieved image quality significantly exceeds that of related non-see-through applications. The current field of view limitation (11.7°) could be improved by using higher refractive index materials or an additional metasurface eyepiece. Further miniaturization of the waveguide is possible by employing smaller SLMs with smaller pixel pitches. The authors discuss potential improvements such as étendue expansion techniques, integration with illumination waveguides, and optimization of the CGH algorithm for real-time operation using machine learning approaches.

This research demonstrates a significant advance in holographic AR technology. The co-design of nanophotonic hardware and AI-driven algorithms enabled a compact, high-quality, full-color 3D holographic OST AR display. The superior image quality and compact form factor represent a substantial step towards the realization of practical and comfortable 3D holographic AR glasses. Future research could focus on expanding the field of view, further miniaturizing the waveguide, implementing étendue expansion techniques, and optimizing the CGH algorithm for real-time performance.

The current prototype has a limited field of view (11.7°), although this is comparable to some commercial systems. The hologram generation process currently takes several minutes per phase pattern, limiting real-time performance. The authors acknowledge the need for further optimization of the CGH algorithm for real-time capabilities, as well as the exploration of étendue expansion techniques to address the space-bandwidth product limitations of the SLM.