Adaptative machine vision with microsecond-level accurate perception beyond human retina

Current machine vision systems struggle with adapting to varying brightness levels, often requiring complex circuits and algorithms. This research aims to overcome this limitation by developing a bio-inspired visual sensor that mimics the human retina's adaptive capabilities but with significantly improved speed. The human retina achieves visual adaptation through a combination of feedforward excitation and feedback inhibition pathways involving rod and cone cells and horizontal neurons. However, the feedback inhibition mechanism is relatively slow, leading to issues like delayed response times in changing light conditions. This study proposes a novel approach using avalanche tuning as feedforward inhibition in a 2D bionic transistor to achieve much faster adaptation speeds than the human retina or existing bionic sensors. This faster adaptation is crucial for applications like autonomous driving and robotics where rapid and accurate perception in varying light conditions is essential. The use of a 2D material-based transistor offers advantages in terms of miniaturization, flexibility, and potentially lower power consumption compared to traditional silicon-based approaches.

Existing 2D bionic vision sensors have shown promise in bio-inspired photonic and scotopic adaptation, but they suffer from slow adaptation times (minutes) and require manual gate voltage configurations. Methods like using charge trapping/detrapping mechanisms or interfacial defects for feedback inhibition result in sluggish adaptation. The research highlights that utilizing feedforward inhibition, unlike the commonly employed feedback inhibition, can significantly enhance the speed of visual adaptation. This review contrasts the slow response time of existing systems with the proposed solution, emphasizing the need for faster adaptation in applications requiring real-time image processing in dynamic environments.

The researchers fabricated a bionic visual sensor using a MoS2/WSe2 van der Waals heterostructure. The device functions as a field-effect transistor (FET) with avalanche properties. The fabrication process involved mechanical exfoliation of few-layer MoS2 and WSe2, followed by stacking them on a SiO2/Si substrate using a dry transfer technique. Ti/Au electrodes were deposited using ultraviolet maskless lithography and electron beam evaporation. The device's electrical characteristics were measured using a four-probe station and source meter. Noise spectral density was determined through Fourier transformation of dark current traces. The photoresponse was tested using a 635 nm laser, with response times evaluated using an electric shutter system and oscilloscope. Kelvin probe microscopy (KPFM) was used to measure the work functions of WSe2 and MoS2 to characterize the heterojunction. The avalanche properties were characterized by measuring the output characteristics at different gate voltages and drain voltages. The light intensity-dependent avalanche effect was investigated by measuring the photocurrent and avalanche gain under varying light illumination. Finally, an adaptive machine vision system was implemented by combining the bionic transistor with a convolutional neural network (CNN) trained on the MNIST dataset. The CNN was trained to incorporate brightness as an explicit parameter, and the performance was evaluated under various brightness conditions. TCAD simulations were used to model the electric field and ionization rate within the device under different bias and light conditions.

The fabricated bionic transistor exhibited a high multiplication factor (5.29 x 10⁶) and low breakdown voltage (5.48 V). The device demonstrated spontaneous switching between linear and photoconductive effects, mimicking the behavior of rod and cone cells in the retina. The adaptation speed was significantly faster than that of the human retina and reported bionic sensors—108 μs for scotopic adaptation and 268 μs for photopic adaptation, representing an over 10⁴ times improvement. The device showed a large -3dB bandwidth (10.5 kHz at weak light). The responsivity varied greatly depending on light intensity, ranging from 7.6 × 10⁻⁴ to -1 × 10⁻³ A/W. Combining the bionic transistor with a CNN resulted in an adaptive machine vision system with over 98% image recognition accuracy under both dim and bright conditions. The system achieved this accuracy with microsecond-level response times. TCAD simulations validated the bias-and-light-tuned avalanche effect, confirming the operation mechanism.

The findings demonstrate a significant advancement in adaptive machine vision. The microsecond-level adaptation speed surpasses the capabilities of the human retina and existing technologies, enabling real-time image processing in dynamically changing light conditions. The combination of the bio-inspired design with deep learning techniques provides a robust and accurate solution for image recognition. This technology holds significant potential for applications in autonomous vehicles, robotics, and other fields requiring high-speed visual processing. The success of the feedforward inhibition approach suggests that exploring similar mechanisms in future bio-inspired sensor design could lead to further improvements in performance and efficiency.

This research successfully demonstrated a bio-inspired visual transistor with unprecedented adaptation speed, far exceeding the human retina. The integration with a CNN resulted in a highly accurate and rapid adaptive machine vision system. Future work could focus on exploring other 2D materials and device architectures to further enhance performance and explore the application of this technology in various real-world scenarios, such as improving low-light vision and enhancing the robustness of autonomous navigation systems.

The study primarily focused on image recognition using the MNIST dataset, which consists of simple handwritten digits. Further research is needed to evaluate the performance of the adaptive machine vision system on more complex and realistic datasets and scenes. The current device design and fabrication methods may have limitations in terms of scalability and manufacturability for mass production. Long-term stability and reliability testing of the bionic transistor under various operating conditions are also needed.