The human somatosensory system's remarkable ability to sense and process tactile information stems from the intricate interplay of mechanoreceptors, neurons, and synapses. This complex biological system efficiently encodes and processes environmental stimuli, enabling diverse tasks from fine motor control to object recognition. Inspired by the efficiency and adaptability of biological systems, researchers are actively developing neuromorphic devices to emulate these functions. These devices offer potential advantages over conventional computing architectures, particularly in energy efficiency when dealing with complex, unstructured real-world data. Existing neuromorphic systems have shown promise in areas like image and speech recognition, but emulation of human-like tactile sensing and processing remains a significant challenge. Prior work on bio-realistic spiking afferent nerves using organic electronics has demonstrated pressure detection and conversion, but lacked the crucial element of learning and memory. This paper addresses this gap by presenting a novel optoelectronic spiking afferent nerve system capable of sensing, coding, learning, and memorizing tactile information. This system aims to significantly advance bio-inspired sensory systems by directly emulating the complex neural coding principles of biological systems, paving the way for more sophisticated applications in robotics and human-machine interaction.

Numerous studies have explored the development of neuromorphic devices inspired by biological systems. These devices have shown success in various pattern recognition tasks, including image recognition (Wang et al., 2018; Sheridan et al., 2017), visual information processing (Seo et al., 2018; Chen et al., 2018), and speech recognition (Truong et al., 2014; Wu et al., 2018). The development of smart sensing and sensorimotoric systems has also seen progress, utilizing bio-inspired approaches (Tee et al., 2015; Wu et al., 2018; Wan et al., 2018; Lee et al., 2018). Furthermore, advancements in neuromorphic computing have leveraged memristive devices (Wang et al., 2018; Prezioso et al., 2015; Xia & Yang, 2019) to create systems that more closely mirror biological neural networks. However, the emulation of the complex tactile processing, including the learning and memory aspects observed in biological somatosensory systems, has remained limited. Kim et al. (2018) presented a bio-inspired flexible organic artificial afferent nerve capable of detecting and converting pressure information, but it lacked the plasticity crucial for learning and memory. This study builds upon this previous work, incorporating learning capabilities to further bridge the gap between artificial and biological tactile sensing.

This research utilized a bio-inspired optoelectronic spiking afferent nerve system. The system consists of several key components: 1. **MXene-based Pressure Sensors:** Flexible pressure sensors were fabricated using high-crystalline quality Ti3C2Tx MXene, chosen for its lamellar structure and pressure-sensitive conductivity. These sensors provided a wide-range pressure response (up to 200 kPa), covering the typical range of biological receptors. The MXene was synthesized using a process involving LiF, HCl, and multiple washing cycles to achieve a pH of 4-5, followed by dispersion in deionized water and deaeration with N2. The sensors were constructed by depositing patterned Au/Ta electrodes on polyimide (PI) substrates and encapsulating the MXene solution between the electrodes and a PDMS capping layer. 2. **Analog-to-Digital Converter (ADC):** A LED-coupled ring oscillator and edge detector functioned as an optical ADC. This circuit converted the pressure-dependent voltage signals from the MXene sensors into optical spikes, improving robustness compared to voltage amplitude coding. The design ensured an output frequency range of 0-100 Hz for a pressure input of 0-100 kPa, maintaining constant amplitude and duration of optical spikes. 3. **Optoelectronic Synapse:** A synaptic photomemristor based on ITO/ZnO/NSTO was used as an artificial synapse. This device integrated and processed the optical spikes, generating post-synaptic currents (PSCs) and exhibiting persistent photoconductivity, neural facilitation (paired pulse facilitation), and spike-rate-dependent plasticity. The synaptic photomemristor was fabricated on Nb-doped SrTiO3 (NSTO) substrates, with ZnO film grown by magnetron sputtering and ITO film deposited as a top electrode. 4. **System Integration:** Optical communication between the pressure sensors, the ADC, and the synaptic photomemristor allowed for non-contact integration, mimicking the biological process of multiple action potentials converging at synapses. Multiple sensor arrays (2x2 and 4x4) were used to test for the ability of the system to handle multiple inputs and motion detection. 5. **Data Analysis:** The experimental data, obtained using force stands, Keithley instruments, and oscilloscopes, were processed and analyzed using Wolfram Mathematica and MATLAB. Custom algorithms were developed for tasks such as decoding Morse code and recognizing Braille. Dimensionality reduction techniques were used for handwritten character and word recognition. Detailed characterization methods for each component are provided in the supplementary information.

The optoelectronic spiking afferent nerve demonstrated several key capabilities: 1. **Pressure Sensing and Encoding:** The MXene-based sensors accurately detected pressure changes, and the ADC effectively converted these into optical spikes. The system utilized both rate coding (frequency of spikes reflecting pressure magnitude) and temporal coding (timing and duration of spikes carrying information). 2. **Morse Code and Braille Recognition:** The system successfully recognized Morse code characters using both temporal coding (interpreting the duration of spikes and pauses) and spike counting. The ability to process different time scales effectively is shown through a combination of rate and temporal coding. The system also recognized Braille characters by integrating information from multiple sensors. 3. **Object Motion Detection:** Using a 2x2 and later a 4x4 sensor array, the system could detect the direction and speed of object movement across the sensor surface. The direction was determined by which sensor was activated first, and the speed was calculated from the latency between activation of different sensors. 4. **Handwritten Character and Word Recognition:** Through dimensionality reduction, the system processed inputs from a 5x5 sensor array, extracting features from the spiking patterns of five parallel synaptic photomemristors. This created a 5D feature vector for each character. The system was trained on sets of these feature vectors, demonstrating learning and memorization capabilities through the persistent photoconductivity of the photomemristors, which altered the synaptic weights during training. The recognition accuracy improved with each training cycle. This feature learning allows the system to convert spiking patterns into memorized weight changes, essentially creating a learned dictionary for character recognition. A proof-of-concept demonstration of word recognition, using a letter-by-letter approach and later a combined letter approach for dimensionality reduction, was also presented. This latter approach used an artificial neural network to classify short words after training cycles. This further shows the system's adaptability and ability to learn and process complex information.

This research successfully demonstrates a bio-inspired optoelectronic spiking afferent nerve that replicates several key aspects of biological tactile sensing and processing. The use of optical communication and spike coding enhances the system’s robustness, efficiency, and scalability. The system’s ability to integrate multiple inputs, recognize patterns, and learn through activity-dependent plasticity opens up new possibilities for artificial tactile sensing applications. The dimensionality-reduction technique used for handwritten character and word recognition is particularly significant. This approach simplifies the computational burden while preserving essential information, making the system more efficient and suitable for real-world applications. The demonstration of word recognition, while using short words as a proof of concept, hints at the potential for extending this approach to more complex language processing. The successful integration of the various components of the system shows the feasibility of creating advanced bio-inspired sensory systems. These systems could be crucial for developing the next generation of prosthetics, robotics, and human-machine interfaces that are more sensitive, responsive, and adaptable to various tasks.

This study presents a novel optoelectronic spiking afferent nerve that successfully mimics biological tactile sensing, including neural coding, perceptual learning, and memorization. The system’s ability to recognize Morse code, Braille, object motion, and handwritten characters and words represents a significant advance in bio-inspired sensory systems. Future research could focus on increasing the scale and complexity of the sensor arrays, exploring more sophisticated learning algorithms, and investigating applications in advanced prosthetics, neurorobotics, and more complex human-machine interaction scenarios.

The current system's ability to recognize handwritten words is limited to short words, due to the dimensionality reduction techniques employed. Scaling the system to handle longer words or more complex patterns would require further algorithmic improvements and potentially larger sensor arrays. Additionally, the training process for handwritten character recognition relies on a supervised learning approach. Developing methods for unsupervised or reinforcement learning could further enhance the system's adaptability and reduce training time. While the MXene-based pressure sensors exhibit good performance, the long-term stability and durability of the sensors and the synaptic photomemristors under various environmental conditions need further investigation.