Tactile sensory coding and learning with bio-inspired optoelectronic spiking afferent nerves

Biological somatosensory systems efficiently sense, transmit, and process tactile information using distributed receptors, neurons, and synapses that encode stimuli into spikes for neural coding and learning. Neuromorphic devices aim to emulate such spike-based sensing and processing to achieve robust, low-power functions that are difficult for conventional architectures. Emulating human-like tactile sensing and processing is vital for intelligent robotics and human–machine interfaces. Prior bio-inspired spiking afferent nerves combined flexible pressure sensors, oscillators, and synaptic transistors to detect and integrate pressure, but lacked plasticity for learning and memory. The research question addressed here is whether an artificial, bio-inspired spiking afferent nerve can integrate tactile sensing with neural coding, plastic synaptic processing, and hardware-level learning/memory to recognize complex temporal and spatial tactile patterns. The purpose is to design an optoelectronic system that senses pressure, converts it into coded optical spikes, integrates them in a synaptic photomemristor, and demonstrates perceptual learning for tasks such as Morse code, braille, motion detection, and handwriting recognition. The significance lies in advancing e-skin and neurorobotics with bio-realistic coding and in-memory computation at the sensor edge.

Neuromorphic devices have demonstrated spike-based processing for images, vision, speech, sensing, and computing using memristive and synaptic elements. A recent flexible organic artificial afferent nerve combined resistive pressure sensors, ring oscillators, and a synaptic transistor to mimic SA-I afferent behavior, but its non-plastic architecture limited learning and memorization. MXenes (e.g., Ti3C2Tx) have shown promise for flexible, sensitive piezoresistive pressure sensing. Memristive and optoelectronic synapses provide plasticity, persistent photoconductivity, and activity-dependent weight changes essential for neural computation. Building on these, the present work integrates MXene sensors with LED-coupled ADCs and a synaptic photomemristor to realize optical spike communication, multiple neural coding schemes (rate and temporal), multi-input integration, and hardware-based feature extraction/learning for tactile tasks.

System architecture: The artificial afferent nerve emulates biological SA‑I pathways. It comprises (1) flexible MXene (Ti3C2Tx)-based piezoresistive pressure sensors; (2) an analog-to-digital conversion (ADC) stage built from a ring oscillator and edge detector that drives a light-emitting diode (LED) to generate optical spikes; and (3) an optoelectronic synapse (synaptic photomemristor) based on an ITO/ZnO/Nb-doped SrTiO3 (NSTO) stack that integrates optical spikes into post-synaptic currents (PSCs) with activity-dependent weight changes.

Neural coding: The ADC converts pressure-dependent voltages into optical spikes with constant amplitude/duration and a pressure-dependent frequency (rate coding). The system also encodes information in spike timing and quiescent intervals (temporal coding). The optical spike frequency was designed to span 0–100 Hz for 0–100 kPa pressure inputs, with spike amplitude/duration fixed.

Optoelectronic synapse behavior: The ZnO/NSTO interface shows persistent photoconductivity (PPC). The photomemristor responds to optical spikes with PSC spikes and gradually increasing baseline (facilitation). Spike-rate-dependent plasticity (SRDP) modulates synaptic weight (PSC level) as a function of spike frequency. A brief negative voltage pulse resets the PSC to baseline for re-use.

Multiple-input integration: Optical communication enables a single photomemristor to receive and combine multiple optical spike trains from spatially separated sensors/ADCs without complex wiring. Simultaneous pressures produce superposed PSCs whose Fourier spectra reveal distinct frequency components corresponding to each pressure input.

Motion detection: A 2×2 sensor array (each sensor coupled to its own ADC-LED and photomemristor) detects movement direction via the sequence of PSC spiking across synapses and estimates speed via latency to first spike between sensors. A flexible 4×4 array extends motion tracking over larger area.

Handwriting recognition with dimensionality reduction: A 5×5 sensor array is wired so that each row’s five sensors connect to a single ADC-LED and photomemristor (five photomemristors total). For a handwritten letter, each photomemristor’s spiking proportion P = tspiking / twriting is computed, yielding a 5D feature vector P for that letter. An alphabet feature dictionary is formed by averaging features over training inputs. Recognition is performed by nearest-vector matching (smallest difference norm) to the learned dictionary. Photomemristor weight changes Δω = (PSC2−PSC1)/PSC1 encode memory correlated with P, enabling feature learning and storage.

Word recognition with enhanced dimensionality reduction: For short words (e.g., APPLE), 25 features (five per letter) are reduced to 15 by combining adjacent letters’ features (e.g., A+P, P+L, L+E). A simple artificial neural network (ANN) with 15 inputs and six outputs (word classes) is trained via repeated handwriting to classify words.

Materials synthesis and device fabrication: Ti3C2Tx MXene was derived by etching Ti2AlC in LiF/HCl, followed by sequential washing/centrifugation to pH 4–5, redispersion, N2 deaeration, sonication, and sedimentation to collect the supernatant. MXene pressure sensors used PI flexible substrates with sputtered Ta/Au electrodes (5/50 nm), and a PDMS capping layer; MXene solution was drop-cast on selected areas and dried before PDMS alignment. Synaptic photomemristors used NSTO substrates (bottom electrode), sputtered ZnO (60 nm; 5.8×10−3 mbar, Ar 16 sccm, O2 4 sccm, 60 W) forming a Schottky barrier, and sputtered ITO top electrodes (3.4×10−3 mbar, Ar 10 sccm, 50 W) patterned by shadow mask, with 100 µm × 100 µm active area.

Characterization: Pressures (up to ≥200 kPa) were applied via a force stand and gauge; sensor I–V measured with Keithley 4200; ring oscillator/edge detector characterized with Keithley 2400 and Keysight DSO1024A. Photomemristor PSCs were recorded with Agilent B1500A under 375 nm LED pulses (0.65±0.06 mW mm−2). System-level PSCs were recorded during single/multi-input, motion, braille (3D-printed blocks), and handwriting tasks. Data analysis and decoding were implemented in Mathematica 12 and Matlab.

• MXene pressure sensor performance: Wide pressure response up to 200 kPa; clear resistance change vs. pressure. Sensor I–V demonstrates pressure-dependent resistance suitable for tactile range.
• Optical ADC behavior: Converts sensor voltage to optical spikes with frequency increasing from 0 to ~100 Hz over 0–100 kPa while maintaining constant spike amplitude/duration; outputs robust to voltage degradation issues inherent to amplitude coding.
• Optoelectronic synapse: Exhibits PSC spikes and persistent photoconductivity enabling temporal integration and facilitation (paired-pulse facilitation). High sensitivity and working speed up to 250 kHz, exceeding biological synapse ranges. Spike-rate-dependent plasticity: after 100 light pulses, higher spike frequencies produce larger weight changes.
• End-to-end input-output mapping: Increasing pressure increases PSC spike frequency and synaptic weight. Example: 100 kPa yields PSC frequency ~86 Hz and weight change ~50% after 1 s stimulation; device can be rapidly reset by a negative pulse.
• Temporal coding applications: Morse code recognized using timing of spiking and quiescent intervals; boundaries between short/long spikes and spaces learned from PSC statistics. Demonstration correctly decodes “AALTO.” Spike-counting method also distinguishes letters; ambiguous cases improved by group-wise counting.
• Multi-input integration: A single photomemristor integrates simultaneous pressures (e.g., 35 kPa and 90 kPa). The combined PSC approximates the sum of individual PSCs; Fourier spectra show distinct peaks (e.g., ~59 Hz and ~80 Hz) corresponding to each pressure, enabling decomposition of concurrent stimuli.
• Braille reading: Two-sensor scanning over braille characters produces PSCs whose frequency and timing encode patterns; with a trained threshold (F0) to separate left/right dot frequencies and a braille dictionary, the system reads words like “HELLO.”
• Motion detection: In a 2×2 array, direction is inferred from the sequence of PSC activations; speed computed from latency between first spikes across sensors using known spacing. Extended to 4×4 array for larger-area trajectory tracking and simultaneous extraction of pressure (frequency) and motion path from PSCs.
• Handwriting recognition with dimensionality reduction: 5×5 array reduced to five photomemristors (one per row). Each letter yields a 5D vector of spiking proportions P forming a feature dictionary. Supervised averaging over training updates the dictionary. Recognition accuracy is ~68% after first cycle and improves to ~84% after 10 training cycles. Weight changes Δω across photomemristors during training correlate strongly with P, evidencing hardware memory of features.
• Word classification with enhanced reduction: Combining adjacent letters reduces 25D word vectors to 15D. A 15-input/6-output ANN trained by repeated handwriting correctly recognizes six fruit words (APPLE, ORANGE, BANANA, PEAR, CHERRY, GRAPE) after only four training cycles; post-training, each output neuron selectively responds to its target word.

The work demonstrates a bio-inspired tactile sensing pipeline in which pressure is transduced to spike-coded optical signals and integrated by a plastic optoelectronic synapse, enabling both rate and temporal coding. By leveraging optical communication, the system integrates multiple inputs without complex wiring, mimicking synaptic convergence. The persistent photoconductivity-driven plasticity provides in-memory computation that supports perceptual learning: the device-level weight updates mirror extracted features, enabling recognition and memory of handwritten letters and words. These capabilities directly address the goal of combining sensing, coding, and learning in a single hardware framework, moving beyond prior non-plastic afferent nerve emulators. The significance spans e-skin, neurorobotics, and human–machine interfaces where robust spike coding, multi-input fusion, temporal pattern recognition (Morse, braille), motion tracking, and on-device learning reduce computational and communication burdens. The demonstrated dimensionality reduction shows how hardware feature extraction can simplify downstream classifiers while retaining discriminative information.

This study introduces an optoelectronic spiking afferent nerve that unifies tactile sensing, spike-based optical communication, synaptic integration with plasticity, and hardware-level learning/memory. The system: (1) uses MXene-based flexible sensors to detect pressure; (2) converts signals into optical spikes with rate/temporal codes; (3) integrates and learns via a photomemristive synapse. It recognizes Morse code and braille, detects motion and speed, and performs handwriting and word recognition with dimensionality reduction and simple classifiers, achieving up to ~84% letter recognition after training and accurate six-word classification after four training cycles. Future directions include scaling to larger, higher-density arrays; integrating lower-power or wavelength-optimized optoelectronics; enhancing stability and endurance; combining multiple coding schemes adaptively; expanding vocabularies and training sets; and embedding more advanced on-chip learning rules and networks for richer, closed-loop sensorimotor tasks.

The demonstrations are proof-of-concept on relatively small arrays (2×2, 4×4, 5×5) and limited vocabularies, which may constrain generalizability to complex, real-world tactile scenes. Letter recognition accuracy (~84% after 10 cycles) indicates room for improvement and may be sensitive to handwriting variability (stroke, speed, path). Optical communication employs 375 nm illumination and device plasticity via persistent photoconductivity, necessitating periodic electrical resets and potentially impacting long-term stability or energy consumption in continuous-use scenarios. Training datasets were limited, and environmental factors (lighting, temperature, mechanical wear) were not extensively evaluated. Integration and packaging for large-area, flexible e-skins remain to be addressed.