The field of machine vision is rapidly advancing, driven by the demand for real-time applications such as autonomous driving. Traditional machine vision systems, however, suffer from high latency and energy consumption due to the separation of image sensing and processing units. Data transfer between these units creates bottlenecks that limit performance in time-critical scenarios. Neuromorphic visual systems, inspired by biological vision, offer a promising solution by integrating sensing and processing, either near or within the sensor itself. This in-sensor computing architecture significantly reduces data transfer, enhancing speed and energy efficiency. Existing neuromorphic visual systems utilize various approaches to achieve tunable photoresponsivity in photosensors, including gating effects in 2D materials and ion migration in memristive materials. However, these methods have limitations. Gating requires additional power, while ion migration is slow and suffers from poor retention. Therefore, a novel photosensor with improved speed, energy efficiency, and reliability is crucial for advancing real-time machine vision. Ferroelectric photosensors (FE-PSs) are emerging as a superior solution, offering a gate voltage-free, self-powered approach to reconfigurable photoresponsivity. By leveraging the remanent polarization to modulate the photovoltaic response, FE-PSs can achieve both magnitude and sign changes in photoresponse, allowing a single device to represent both positive and negative weights, reducing hardware complexity. The nonvolatility, high controllability, and ultrafast switching speed of ferroelectric polarization, combined with high photosensitivity and fast response time, make FE-PSs ideal for high-speed and reliable image sensing. This research explores the potential of FE-PS networks (FE-PS-NETs) as a fast, low-power, and reliable hardware solution for real-time machine vision.

A variety of neuromorphic visual systems have been developed to perform image processing tasks such as contrast enhancement, noise suppression, adaptive imaging, recognition, and auto-encoding. Reconfigurable photosensor networks (PS-NETs) with in-sensor computing architecture are particularly promising because they act as artificial neural networks (ANNs), enabling simultaneous image sensing and processing. However, existing reconfigurable PS-NETs rely on photosensors with tunable photoresponsivity achieved through methods such as gating effects in 2D materials or ion migration in memristive materials. These methods have limitations in power consumption, speed, and reliability. Ferroelectric materials have been explored in other neuromorphic applications due to their desirable properties like non-volatility and fast switching speeds; however, a ferroelectric photosensor network integrating sensing, memory, and computing capabilities has not been extensively explored.

The study focuses on demonstrating a prototype FE-PS-NET for integrated image sensing and processing. The FE-PS consists of a Pt/Pb(Zr0.2Ti0.8)O3 (PZT)/SrRuO3 (SRO) heterostructure epitaxially grown on a SrTiO3 (STO) substrate. High-quality epitaxial PZT film was chosen due to its large remanent polarization and strong, controllable photoresponse in the ultraviolet (UV) spectrum. The fabrication involved pulsed laser deposition (PLD) for the PZT/SRO bilayer and Pt top electrode deposition. The ferroelectric properties of the Pt/PZT/SRO device were characterized by measuring polarization-voltage (P-V) hysteresis loops using bipolar and monopolar triangular pulses. The photovoltaic response was assessed by measuring current-voltage (I-V) characteristics under 365 nm UV illumination for various polarization states. Piezoresponse force microscopy (PFM) was used to investigate the formation mechanism of intermediate polarization states. The stability and endurance of the FE-PS were evaluated through long-term retention and cycling tests. To demonstrate synaptic behavior, long-term potentiation (LTP) and long-term depression (LTD) were characterized by measuring the photoresponsivity changes upon applying successive positive and negative pulses. A 1x2 FE-PS-NET was initially used to demonstrate the in-sensor multiply-accumulate (MAC) operation, where the output current is the sum of the photocurrents generated by each FE-PS. This was then extended to a 2x2 FE-PS-NET to evaluate the immunity to sneak path issues. The FE-PS-NET's capabilities were further evaluated through real-time image processing tasks: binary classification of 'X' and 'T' patterns and edge detection of an arrow sign. For pattern classification, a 1x9 FE-PS-NET was used, and for edge detection, a 2x9 FE-PS-NET with Sobel kernels was employed. The performance of these tasks was assessed using accuracy (for classification) and F-Measure (for edge detection). The theoretical framework for polarization-controlled photovoltaic behavior was based on Schottky barrier modulation at the Pt/PZT and PZT/SRO interfaces.

The study demonstrated several key findings: 1. **Tunable and Nonvolatile Photoresponsivity:** The fabricated FE-PS exhibited tunable, nonvolatile, and multilevel photovoltaic responses controlled by the remanent polarization. The photoresponsivity could be switched symmetrically, with opposite signs and similar magnitudes for opposite polarization states. This allowed a single FE-PS to represent both positive and negative weights. 2. **High Reliability and Endurance:** The FE-PS demonstrated high reliability with long retention times (>24 hours) and excellent endurance (10⁶ cycles). Device-to-device variation was minimal (~3.2%). The linear dependence of photocurrent on light intensity validated the multiplication aspect of the device operation. 3. **Synaptic Behavior:** The FE-PS exhibited typical synaptic behaviors, including LTP and LTD, with gradual and controllable weight changes. Multiple intermediate photoresponsive states were accessible (at least 25). 4. **In-sensor MAC Operations:** The FE-PS-NET successfully performed in-sensor MAC operations, validating the summation and multiplication operations of the network even in larger configurations (2x2). The network showed good immunity to sneak path issues. 5. **Real-time Image Processing:** The FE-PS-NET demonstrated real-time image processing capabilities, achieving 100% accuracy in binary classification and an F-Measure of 1 in edge detection. The weights showed minimal changes after these operations. 6. **Ultra-low Latency and Energy Consumption:** The FE-PS-NET demonstrated potential for ultra-low latency (~2.6 µs for a 10-million-pixel image) and zero energy consumption for inference, with low energy consumption for programming (~3.1 nJ per device, potentially reduced to ~0.1 pJ per bit per operation with scaling).

The findings address the research question of creating a robust and efficient hardware solution for real-time machine vision by demonstrating the feasibility and advantages of the FE-PS-NET. The successful implementation of in-sensor MAC operations, binary classification, and edge detection shows the potential for the FE-PS-NET to perform complex image processing tasks in real-time. The superior characteristics of FE-PSs, including self-powered operation, nonvolatility, and fast switching speeds, significantly improve upon existing technologies that rely on gate voltages or suffer from slow ion migration. The ultrafast operation and minimal energy consumption for inference make FE-PS-NET a compelling alternative to traditional Von Neumann architectures, offering significant advantages for energy-efficient and high-speed applications. This work also demonstrates a first-of-its-kind ferroelectric neuromorphic device integrating sensing, memory, and computing capabilities. This novel paradigm opens up new avenues for research in neuromorphic computing and real-time machine vision.

This research presented a proof-of-concept demonstration of a ferroelectric photosensor network (FE-PS-NET) for real-time machine vision. The FE-PS-NET, built using multiple two-terminal Pt/PZT/SRO heterostructures, showcased multilevel nonvolatile photoresponses controlled by remanent polarization. The system demonstrated high reliability, low device-to-device variation, and excellent endurance. The ability of a single FE-PS to represent both positive and negative weights, combined with in-sensor MAC operation, enabled real-time image processing tasks with high accuracy and an F-Measure of 1. The extremely low latency and zero energy consumption for inference highlight the potential of FE-PS-NET for developing high-speed and energy-efficient hardware for real-time machine vision. Future research should focus on scaling up the network to process larger images and explore different applications.

The current study utilizes 365 nm UV light for illumination. The effectiveness of the FE-PS-NET with visible light needs further investigation. While the network showed good immunity to sneak path issues in the tested configurations, further research is needed to evaluate the scalability of the approach to large-scale networks. The training process was performed ex-situ, meaning that a software-based algorithm was used to determine the weight matrix and subsequently program the FE-PS-NET. In-situ training techniques should be explored to fully realize the potential of this technology. Finally, while the energy consumption for programming was shown to be low, improvements in fabrication and circuit design may further reduce energy usage.