Visualized In-Sensor Computing

Artificial neural systems leveraging ion conduction play a crucial role in advancing universal artificial intelligence, neural robotics, in vitro perception of motion prostheses, and the replacement of diseased nerves. Within such architectures, the reconfiguration of synaptic weight is key to autonomous encoding of spatiotemporal information in response to stimuli, enabling artificial neural reflex systems to adapt to dynamic environments. However, most artificial nervous systems indicate synaptic weight updates mainly via conductivity changes, which provide limited information compared to living organisms. The uncertainty and intricacy of ion doping and de-doping during action potentials result in ambiguous relaxation times for weight updates, potentially leading to challenges in processing time-series signals such as aliasing or misses. Exploring alternative forms of synaptic weight information could mitigate these issues. Color is a direct and widespread channel of communication among biological entities; integrating color information into artificial neural systems could enhance functionality. For instance, chameleons adjust pigment cell activity via neuromodulators (e.g., epinephrine), enabling rapid color changes for communication. Yet, neuromorphic electronics capable of conveying information through color changes are scarce. Biological neuro-reflexes adhere to the all-or-none law, triggering actions only when stimulus intensity exceeds a threshold. Existing neuromorphic electronics largely process purely electrical signals and may filter out weak yet important signals. Combining color-based alterations with adaptable electrical properties can visualize synaptic weight changes and monitor a broader range of environmental signals. Incorporating color information as an additional measure of synaptic weight in visualized artificial neural systems can enhance accuracy and efficiency of in-sensor computing by high-fidelity local processing, mirroring natural sensory systems. Here, we present a conceptual design and production of the inaugural electrochromic neuromorphic transistor (ENT) that incorporates color weight updates alongside electrical weight. It leverages electrochromism from proton doping in a crystallized polymer and anion-accumulation-induced charge effects. Using a Nafion ion-exchange membrane allows adaptive control of ion doping, enabling precise adjustment of chromaticity and conductivity. The device exhibits a swift reset time (conductivity to ~1/3000 in ~1 s; chromaticity resets to 0) ensuring rapid, unimpeded signal transmission. We demonstrate diverse bionic coding (combined conductivity and chromaticity), a visualized pattern-recognition network, and integration with an artificial whisker to realize a color-visualized reflex system that shows real-time signal flow within a reflex arc.

The paper situates its contribution within neuromorphic systems that primarily rely on ion conduction and represent synaptic weight updates via conductivity changes. Prior works highlight limitations: uncertain ion doping/de-doping dynamics can yield ambiguous relaxation times and complicate time-series signal processing (aliasing/misses). Although color is a common biological communication modality (e.g., chameleon pigment cell regulation by catecholamines), there is a notable scarcity of neuromorphic electronics using color changes to convey information. Existing reflex systems focus on electrical transmission, adhering to the all-or-none law, and may miss weak yet meaningful signals. Integrating color-based information alongside electrical properties is proposed as a means to visualize synaptic weight and broaden detectable environmental signals, improving in-sensor computing fidelity.

Device concept and architecture: The electrochromic neuromorphic transistor (ENT) comprises a poly(3-hexylthiophene) (P3HT) channel, a Nafion-modified layer (NML) acting as a proton-exchange/ion-selective membrane, an ion gel gate dielectric, and Au source/drain electrodes on a flexible polyethylene naphthalate (PEN) substrate. The ion gel contains EMIM+ cations, TFSI− anions, and H+ protons. Presynaptic spikes applied to the ion gel drive ion migration: TFSI− accumulation at the ion gel/NML interface induces hole carriers in P3HT (EPSC), while H+ passes through Nafion into P3HT, triggering electrochromism. Active material engineering: A crystallized P3HT nanowire (NW) thin film was prepared via low-temperature solvent engineering (CB:DCM 1:1, 2 mg mL−1; heated at 60 °C for 2 h then slowly cooled) to form ~12 nm interconnected NW networks with efficient charge transport. Nafion (0.5 wt%) was dispersed in water:ethanol (1:1). Ion gels were made by dissolving PVDF-HFP and EMIM-TFSI (mass ratio 1:3) in acetone, stirred at 40 °C for ≥30 min, drop-cast and cured to form films. Device fabrication: Top-gate bottom-contact structure. Au S/D (60 nm) thermally evaporated on PEN, preheated 120 °C for 10 min. P3HT spin-coated (2000 rpm, 30 s) and annealed 120 °C for 10 min to crystallize NW film. Nafion spin-coated (2000 rpm, 30 s). Ion-gel top gate film transferred onto channel area. Characterization: AFM (tapping mode) and SEM characterized morphology; HRTEM confirmed P3HT NW self-assembly and Nafion lattice fringes indicative of ion-exchange capability. UV-Vis spectroscopy compared pristine P3HT, P3HT/Nafion (H+ undoped), and H+-doped films; XPS probed doping species. KPFM measured contact potential difference (CPD) under applied tip biases to map ion/doping distribution. Electrochemical impedance spectroscopy (EIS) (Nyquist plots) evaluated ion-transport and EDL resistance changes. Electrical tests used a Keithley 4200A in N2; optical micrographs captured electrochromism. Mechanical testing assessed stress–strain on ENT for biointegration suitability. Experimental protocols: Electro-chromaticity boost coding (ECBC) and Morse-code experiments applied gate spikes of different durations/amplitudes (e.g., −4 V pulses; VDS −0.5 V) to evaluate EPSC and chromaticity responses, define thresholds for EPSC coding (EC), chromaticity coding (CC), and their fusion (ECBC). Pattern recognition used a 3×3 ENT array configured as a crossbar, where input pixels were mapped to VDS levels (−0.1 to −0.5 V), and device conductance states were programmed by pulse numbers (1–100) to perform matrix multiplication. A bionic reflex arc integrated an ENT with an artificial whisker featuring a CNT/PDMS vibration sensor and an ionic polymer–metal composite (IPMC) actuator; afferent spikes derived from sensed vibrations drove the ENT; output EPSC controlled the actuator; chromaticity was monitored to visualize signal flow.

• First demonstration of an electrochromic neuromorphic transistor (ENT) that encodes synaptic weight in both conductivity and color (chromaticity), enabling visualized in-sensor computing.
• Nafion ion-exchange membrane adaptively regulates ion doping: TFSI− induces hole accumulation (EPSC), while H+ penetrates Nafion into P3HT to drive electrochromism.
• Rapid reset/absolute short-term plasticity: after a 1.5 s spike, ΔEPSC decays to ~1/3000 of its initial value in ~1 s, and chromaticity (ΔGray) resets to 0, enabling high-rate, interference-free processing.
• Spectroscopic and electrical evidence: UV-Vis shows a new broad peak at ~780 nm upon H+ doping with concomitant reduction of the 540 nm P3HT peak; optical images confirm visible color change consistent with polaron/bipolaron formation. EIS indicates reduced EDL resistance under positive bias. XPS corroborates doped species intermixing.
• KPFM mapping of surface potential (φs) under bias: initial −53.3 mV; −2 V → −75.2 mV; −4 V → −92.2 mV (H+ doping); +2 V → −43.7 mV; +4 V → −37.8 mV (H+ de-doping).
• Short-term plasticity mimics: spike-duration-dependent plasticity (SDDP) with near-linear increase in response versus duration; spike-number-dependent plasticity (SNDP) where EPSC and chromaticity scale with spike count.
• Electro-chromaticity boost coding (ECBC) for Morse code: EPSC coding (EC) accuracy 88.5% (threshold VEC=2); chromaticity coding (CC) accuracy 96.2% (threshold γCC=100); fused ECBC achieves 100% accuracy (threshold VECBC=120) across 26 letters.
• Visualized pattern-recognition network: 3×3 ENT array performs voltage-controlled matrix multiplication. Recognizes 3×3 pixel images (X, L, T, Y); increasing maximum pulse number (nmax=3→10→100) expands output ranges and improves accuracy. Real-time visualization of synaptic weights via RGB/gray chromaticity cards linked to EPSC mapping.
• Frequency-dependent filtering: ENT exhibits low-frequency suppression analogous to synapses with low neurotransmitter release probability, enabling dynamic filtering/encoding of real-time signals.
• Biohybrid reflex system: ENT integrated with an artificial whisker (CNT/PDMS vibration sensor; IPMC actuator) forms a neuro-electronic reflex arc. Afferent spikes (V1–V5, weak→strong) produce increasing EPSC and chromaticity. Consistent with all-or-none behavior: V1–V2 subthreshold (no actuation), V3–V5 trigger whisker deflection; ENT exhibits high-frequency color blinking even at V1, visualizing subthreshold information.
• Mechanical robustness for biointegration: ENT shows elongation at fracture 73.77%, maximum force tolerance 391.72 N, and tensile strength 217.62 MPa.

The study addresses the limitation of traditional ion-conducting neuromorphic systems that encode synaptic weight solely through conductivity, by introducing chromaticity as an additional, immediately perceivable modality. The ENT’s Nafion-enabled selective ion control yields fast, reconfigurable electrical behavior and synchronized color changes, providing a multidimensional, visualizable representation of synaptic weight. This dual-modality improves coding robustness and interpretability: EC and CC each provide partial accuracy, while their fusion (ECBC) reaches perfect decoding in the Morse-code test. The fast, absolute STP enables high-throughput in-sensor processing without interference from residual weights. The 3×3 array demonstrates that ENT-based cores can perform local matrix operations for pattern recognition with real-time weight visualization, aligning with biological visual memory principles to enhance efficiency. The biohybrid reflex system further illustrates practical relevance, where color blinking visualizes signal flow and subthreshold stimuli during actuation, combining sensory processing and motor response. Collectively, these capabilities expand the biomimetic coding paradigm and suggest broader applicability for neuromorphic interfaces that benefit from interpretable, visual feedback.

This work introduces an electrochromic neuromorphic transistor that simultaneously updates electrical and color-coded synaptic weights, enabled by Nafion-mediated adaptive ion regulation in a crystallized P3HT channel. The device exhibits rapid recovery (<1 s), absolute short-term plasticity in both EPSC and chromaticity, and supports robust multimodal coding; an ECBC scheme achieves 100% decoding accuracy for Morse letters. A visualized pattern-recognition network and a biohybrid reflex arc with an artificial whisker demonstrate in-sensor computing, real-time weight visualization, and action control. These advances pave the way for more sophisticated bioinspired computing systems and bio-hybrid interfaces that leverage color-based expressions for enhanced functionality, interpretability, and adaptability.