Visualized In-Sensor Computing

Artificial nervous systems utilizing ion conduction are crucial for advancements in AI, robotics, and bio-hybrid interfaces. Reconfiguring synaptic weight is key, enabling autonomous spatiotemporal information encoding. However, current systems primarily rely on conductivity changes, offering limited information compared to biological systems. The ambiguity and complexity of ion doping/de-doping lead to challenges in processing time-series signals. Color, a widespread biological communication method, presents an opportunity for enhanced functionality in artificial neural systems. Existing neuromorphic electronics largely neglect color-based information, limiting their ability to process a broader range of environmental signals. This research addresses these limitations by introducing an electrochromic neuromorphic transistor that integrates color-based alterations with adaptable electrical properties for visualized in-sensor computing.

Existing literature highlights the importance of ion conduction in artificial nervous systems for applications in AI, robotics, and prosthetic devices. However, the limitations of using conductivity changes alone to represent synaptic weight are widely acknowledged, with studies demonstrating challenges in processing time-series signals due to ambiguous relaxation times. The research also reviews the prevalence of color-based communication in biological systems, and the lack of neuromorphic electronics capable of utilizing color changes to convey information. The all-or-none law in neuro-reflex processes and the limitations of existing neuromorphic electronics that focus primarily on electrical signals are also discussed. The potential benefits of integrating color-based information into artificial neural systems for enhanced accuracy and efficiency in in-sensor computing are highlighted.

The researchers designed and fabricated an electrochromic neuromorphic transistor (ENT) composed of an ion gel, a Nafion-modified layer (NML), a poly(3-hexylthiophene) (P3HT) channel, and source/drain electrodes on a flexible substrate. The ion gel contains EMIM cations, TFSI anions, and H+ protons. The ENT mimics neurotransmitter release: presynaptic spikes cause ion migration, TFSI anion accumulation at the ion gel/NML interface triggers hole carriers in the P3HT channel (EPSC), and H+ introduction into P3HT initiates electrochromism. The P3HT nanowire thin film was engineered for synaptic weight modulation. Atomic force microscopy (AFM), scanning electron microscopy (SEM), high-resolution transmission electron microscopy (HRTEM), electrochemical impedance spectroscopy (EIS), ultraviolet-visible (UV-Vis) spectroscopy, X-ray photoelectron spectroscopy (XPS), and Kelvin probe force microscopy (KPFM) were used to characterize the device and its properties. The ENT's response to electrical impulses mimicking Morse code was tested, and its application in a visualized pattern-recognition network and a visualized bionic reflex system (integrated with an artificial whisker) was demonstrated. The artificial whisker included a vibration-sensing layer (CNT/PDMS composite film) and a bionic actuation layer (ionic polymer-metal composite). The flexibility of the ENT was evaluated using a stress-strain test.

The ENT successfully integrates color weight updates with electrical weight updates, offering a novel approach to synaptic weight representation. The device demonstrated rapid reset times (less than 1 second for both conductivity and chromaticity), crucial for efficient signal transmission. The ENT effectively encodes information using both electrical signals (EPSC) and color changes, achieving high accuracy rates in electrochromaticity boost coding (ECBC) mode. A 3x3 ENT array successfully functioned as a visualized pattern-recognition network, capable of classifying images. The integration of the ENT with an artificial whisker created a visualized bionic reflex system, demonstrating real-time visualization of signal flow and mimicking the all-or-none law. The artificial whisker successfully sensed and responded to varying vibration intensities. The ENT showed high flexibility and strength, making it suitable for biointegration.

The findings demonstrate the feasibility and advantages of using color changes as a measure of synaptic weight in artificial neural systems. The ENT’s rapid reset time and ability to integrate both electrical and color information significantly enhance its performance compared to traditional systems. The visualized pattern-recognition network and bionic reflex system showcase the potential of this technology in various applications, including advanced in-sensor computing and bio-hybrid interfaces. The success of ECBC and the accurate classification of images using the ENT array highlight the potential for highly efficient and accurate information processing. The visualized bionic reflex system provides a new paradigm for designing artificial neural systems that mimic biological processes more closely.

This research presents a groundbreaking electrochromic neuromorphic transistor that uses color changes to represent synaptic weight. The device demonstrates rapid reset times, high accuracy in multimode signal coding, and successful applications in visualized pattern recognition and a bionic reflex system. Future research could focus on exploring further applications of the ENT in more complex neural networks, investigating different materials and architectures for enhanced performance, and developing more sophisticated bio-hybrid interfaces.

While the study demonstrates significant advancements, potential limitations include the current focus on a limited number of image types for pattern recognition and the relatively small scale of the artificial whisker system. Further research is needed to test the robustness and scalability of the system across a broader range of conditions and tasks.