Ferroelectric photosensor network: an advanced hardware solution to real-time machine vision

The paper addresses the challenge of achieving real-time machine vision with low latency, high energy efficiency, and strong reliability. Conventional von Neumann vision pipelines incur significant latency and energy due to data shuttling between separate sensing and processing units. Neuromorphic visual systems with near- or in-sensor computing architectures mitigate this bottleneck by fusing sensing and computation. However, many reconfigurable photosensors to date rely on gate voltages (increasing power) or ion migration (slow kinetics and poor retention). The authors propose a ferroelectric photosensor network (FE-PS-NET) that integrates sensing, memory, and computation in hardware. Leveraging polarization-controlled photovoltaic effects in epitaxial PZT, each FE-PS provides multilevel, nonvolatile, and sign-reversible photoresponsivity, enabling signed weights in a single device and enabling in-sensor multiply-accumulate operations for real-time machine vision tasks.

The authors review neuromorphic visual systems that implement functions such as contrast enhancement, noise suppression, adaptive imaging, recognition, and auto-encoding using reconfigurable sensor networks based on 2D materials and memristive devices. Prior reconfigurable photosensors achieved tunable responsivity mainly via gate modulation in 2D materials and ion migration in memristors. These approaches suffer from additional power consumption for gate biasing and slow, relaxation-prone ionic processes, limiting speed and retention. Ferroelectric photosensors (FE-PS) using remanent polarization to modulate photovoltaic response offer self-powered operation, fast switching (<1 ns), nonvolatility, and high readout speed, with the additional advantage of sign-reversible photoresponse enabling single-device signed weights. Prior works on ferroelectric photovoltaic and synaptic devices establish polarization’s reliable control and fast kinetics, motivating the present sensing-memory-computing integrated FE-PS-NET, which had not been experimentally demonstrated before this study.

Device structure and fabrication: Two-terminal ferroelectric photosensors (FE-PSs) were fabricated with Pt/Pb(Zr0.2Ti0.8)O3 (PZT)/SrRuO3 (SRO) heterostructures epitaxially grown on SrTiO3 (STO) (001) substrates via pulsed laser deposition (PLD). SRO (~40 nm) was deposited at 680 °C, 15 Pa O2; PZT (~120 nm) at 600 °C, 15 Pa O2; laser: KrF (λ=248 nm), 0.9 J/cm², 5 Hz. Films were cooled at 10 °C/min in 1 atm O2. Pt top electrodes (~10 nm, ~200 µm diameter via shadow mask) were deposited ex situ by PLD in vacuum. Individual FE-PSs were wired to form FE-PS-NETs. Characterization: XRD and TEM confirmed epitaxial perovskite phases and high crystalline quality; AFM showed RMS roughness ~470 pm; high-resolution TEM showed well-aligned PZT lattice. Ferroelectric properties were measured via bipolar and monopolar polarization–voltage (P–V) hysteresis using triangular pulses (width 0.15 ms). PFM assessed domain switching and stability (retention up to 18 days). Photovoltaic current–voltage (I–V) characteristics were captured under 365 nm UV illumination (150 mW/cm²; optical power scaled by illuminated electrode area). Short-circuit photocurrent (Isc) and open-circuit voltage (Voc) were extracted for various polarization states written by monopolar triangular pulses; preset pulses (±3 V) prepared initial states. Synaptic testing (LTP/LTD): FE-PS initialized to complete Pup with a +3 V/0.15 ms preset. Sequences of 25 positive triangular pulses (1.65–1.89 V, 10 µs, 0.01 V steps) followed by 25 negative pulses (−1.70 to −1.94 V, 10 µs) were applied without preset between, to gradually switch polarization below coercive voltage (±1.9 V). After each pulse, short-circuit photocurrent under UV illumination was measured to compute photoresponsivity R=I/P (signed), where P is optical power. Multi-cycle measurements assessed reproducibility. Network architecture and MAC: The FE-PS-NET comprises N pixels each split into M subpixels; each subpixel is an FE-PS. Devices with the same subpixel index m are connected in parallel under short-circuit for inference. Under illumination, each FE-PS multiplies local optical power Pn with its photoresponsivity Rmn; Kirchhoff’s law sums currents from N FE-PSs to produce Im=Σn Rmn Pn. MAC was demonstrated on: (i) a 1×2 network (M=1,N=2) with controlled illumination sequences and programmed signs of R; (ii) a 2×2 network (M=2,N=2) to evaluate sneak-path immunity by independently toggling illumination on different devices. Pattern classification: A 1×9 FE-PS-NET implemented a single-layer perceptron for binary classification of 3×3 ‘X’ versus ‘T’ patterns (including noisy variants). Pixel value 1 (black) corresponded to illumination (~4.5 µW) on the matching FE-PS; 0 (white) meant no illumination. Weights were trained ex-situ in software, mapped to target photoresponsivities, and programmed by write-and-verify to minimize discrepancies. The output current Ii was read and fed to a sigmoid activation f(x)=1/(1+e^{-a x}), with scaling a=3 nA^{-1} (activation implemented in software; could be CMOS in practice). Edge detection: An 11×11 binary arrow image was decomposed into 81 overlapping 3×3 sub-images (stride 1). Two 3×3 Sobel kernels were implemented on a 2×9 FE-PS-NET; kernel coefficients were mapped to FE-PS photoresponsivities (signed). Each sub-image was sequentially presented via illumination, and the two kernel outputs I1 and I2 were recorded. The edge map Ig=|I1|+|I2| was normalized to [0,1] and binarized with threshold d=0.6 to yield the final edge image. Electrical measurements and instruments: P–V loops on Radiant Precision Multiferroic; I–V and photocurrents on Keithley 6430; 365 nm LEDs with tunable intensity as light sources. Endurance assessed with ±3 V/10 µs switching pulses up to 10^6 cycles. Device-to-device variation evaluated across 11 devices. Linearity of photocurrent versus light intensity verified across polarization states. Latency estimation considered photocurrent generation (<1 ns reported; <100 ms upper-bound limited by instrumentation in this setup) and RC limits for large arrays. Programming energy per pulse measured at ±2 V/10 µs for ~0.0314 mm² devices; scaling projection provided for ~1 µm devices.

• High-quality epitaxial PZT films (∼120 nm) exhibited large remanent polarization (~80 µC/cm²) with negligible imprint; symmetric coercive voltages enabled balanced, reversible photovoltaic response control.
• Polarization-tuned photovoltaic response produced multilevel, nonvolatile Isc and Voc with sign reversibility. Under ~150 mW/cm² 365 nm illumination: complete Pup and Pdown states yielded Isc ~ +10 nA and ~ −10 nA, respectively; Voc ~ −0.5 V and ~ +0.5 V, respectively. Intermediate polarization states mapped one-to-one to intermediate photocurrent levels.
• Nonvolatility and reliability: photoresponsive states retained ≥24 h; ON/OFF cycling stable; endurance to 10^6 switching cycles with slight change; device-to-device photocurrent variation ~3.2%; LTP/LTD cycle-to-cycle variation ~3% with 25 distinct responsivity levels per sweep; nearly linear photocurrent scaling with light intensity across states.
• Physical mechanism: polarization-modulated Schottky barriers at Pt/PZT and PZT/SRO interfaces tune built-in fields; domain-state proportion sets magnitude and sign of photocurrent.
• In-sensor MAC: 1×2 FE-PS-NET experimentally performed ΣRmnPn with correct multiplication (photocurrent scales with optical power) and summation (currents add). Sign of output matched programmed signed responsivities. A 2×2 network showed good immunity to sneak paths due to short-circuit operation and optical selection.
• Image processing demonstrations: single-layer perceptron (1×9) achieved 100% accuracy classifying 3×3 ‘X’ vs ‘T’ patterns (including noisy variants); kernel weights remained stable post-inference. Edge detection using Sobel kernels (2×9) on an 11×11 arrow image achieved F-Measure = 1 with excellent agreement between measured and theoretical currents and negligible weight drift.
• Performance outlook: Inference is self-powered (photovoltaic mode) with zero energy in principle; programming energy measured ~3.1 nJ per operation for ~0.0314 mm² devices; projected ~0.1 pJ per bit per operation for ~1 µm devices. Estimated total sensing+processing latency for a 10-megapixel image ~2.6 µs (dominated by circuit RC), orders of magnitude faster than von Neumann pipelines.

The study validates a sensing–memory–computing integrated architecture in which ferroelectric photosensors provide both storage (nonvolatile, multilevel, signed weights) and computation (in-sensor MAC) while simultaneously sensing optical inputs. By exploiting polarization-controlled photovoltaic effects in epitaxial PZT, the FE-PS-NET addresses key bottlenecks of real-time machine vision: it removes data shuttling between sensors and processors, reduces latency via analog parallelism, and minimizes energy by self-powered operation. Demonstrations of MAC operations, binary classification, and edge detection confirm functional correctness and robustness. Symmetric, sign-reversible photoresponsivity in a single device reduces hardware overhead compared with differential pair implementations. Reliability metrics (retention, endurance, low variability) and linear photocurrent scaling support practical deployment. The architecture also shows inherent resilience to sneak-path currents due to short-circuit, illumination-selected operation, differentiating it from memristor crossbars. Remaining challenges include scaling to large arrays while maintaining RC-limited speed and integrating peripheral circuitry (amplifiers, activation functions) for fully on-chip operation and visible-spectrum sensitivity for broader applications.

This work experimentally demonstrates the first ferroelectric photosensor network (FE-PS-NET) that integrates sensing, storage, and computation for real-time machine vision. Epitaxial Pt/PZT/SRO FE-PS devices deliver multilevel, nonvolatile, and sign-reversible photoresponsivity with strong reliability, enabling direct in-sensor MAC. The network correctly performs binary pattern classification (100% accuracy) and Sobel edge detection (F-Measure = 1) under 365 nm illumination. The paradigm offers ultralow inference energy (self-powered), high speed (µs-level system latency projected), scalability, and reduced hardware overhead (single-device signed weights). Future research should focus on scaling to large, dense arrays; integrating CMOS front-ends for amplification and activation; extending sensitivity to visible wavelengths; optimizing RC and interconnect design; on-chip learning schemes; and comprehensive system-level benchmarks in realistic machine-vision tasks (e.g., dynamic scenes, autonomous navigation).

• Spectral limitation: The demonstrated PZT devices operate under 365 nm UV due to PZT’s bandgap (~3.6 eV), limiting applicability under visible light without additional engineering.
• Measurement time resolution: While ferroelectric photovoltaic response can be sub-ns, the experimental confirmation here was limited to <100 ms by instrument resolution; overall system latency will depend on RC constants in large arrays.
• Output magnitude and peripherals: Photocurrents are on the order of nA, requiring low-noise amplification and activation circuits (implemented in software here) for full hardware systems.
• Scale and topology: Demonstrations used small arrays (up to 2×2 and 1×9/2×9). Sneak-path immunity appeared good, but behavior in large-scale, densely interconnected networks requires further validation.
• Training methodology: Weights were trained ex-situ and programmed by write-and-verify; in-situ/online learning was not implemented.
• Energy estimates: Zero inference energy refers to device-level photovoltaic operation; system-level energy (control, readout, amplification) was not included. Programming energy was measured on large-area devices; scaling projections assume ideal scaling to ~1 µm.
• Environmental factors and long-term stability: Retention was shown for ≥24 h and domain states up to 18 days; longer-term stability, temperature variability, and environmental robustness were not fully characterized.