Deep learning-assisted object recognition with hybrid triboelectric-capacitive tactile sensor

The increasing demand for advanced human-machine interaction necessitates sophisticated sensors for comprehensive information acquisition. Tactile sensors, crucial for object recognition in robotics and human-machine interaction, convert mechanical stimuli into electrical signals reflecting properties like temperature, humidity, shape, texture, and softness. Existing tactile sensors are categorized into piezoresistive, piezoelectric, capacitive, and triboelectric types, each with limitations in complex environments. Single-function sensors often struggle with diverse objects, requiring multi-parameter analysis for accurate recognition. Two approaches exist: multi-sensor systems (complex and less integrated) and hybrid sensors (offering streamlined integration and multidimensional information). Previous research has explored using tactile sensors for object recognition based on shape features, surface texture, and hardness. However, solutions for single-sensor, multi-signal acquisition remain scarce. This research proposes a hybrid tactile sensor integrating triboelectric and capacitive sensing units to accurately capture diverse object information within a unified framework. The triboelectric unit detects surface material and texture, while the capacitive unit measures hardness. Deep learning is employed to improve recognition accuracy.

Existing literature demonstrates various approaches to tactile sensing for object recognition. Researchers have employed tactile sensors to distinguish objects based on inherent physical properties. Common methods involve recognizing object shape using shape features, characterizing surface texture through sliding and pressing actions, and assessing object hardness. However, single-function sensors face challenges in complex environments. Multi-sensor systems, while providing comprehensive information, suffer from low integration and high complexity. Hybrid tactile sensors offer a more efficient alternative, simultaneously acquiring multiple signals for improved object recognition. Lee's group developed a biomimetic piezoelectric sensor for surface feature recognition using machine learning, and Ding's group presented a hybrid triboelectric and piezoresistive sensor with a real-time sensing system for texture and material recognition using a parallel residual convolutional neural network (PR-CNN). The authors' previous work demonstrated object recognition using a triboelectric-inductive hybrid tactile sensor and machine learning. While current research enables multidimensional information acquisition, methods for obtaining various signals with a single sensor are relatively limited.

A hybrid tactile sensor was designed, integrating a triboelectric and a capacitive sensing unit. Both units utilized a porous PDMS sensitive film and copper electrodes, simplifying the fabrication process. The triboelectric unit resides on the upper layer, and the capacitive unit on the lower layer, separated by a PI layer to prevent signal interference. The triboelectric unit's operation relies on charge transfer between contacting surfaces. Differences in electron gain/loss and contact area due to texture variations lead to distinct triboelectric signals, providing information about both surface material and texture. The capacitive unit’s operation involves capacitance change due to compression of the porous PDMS when gripping objects. Different levels of compression with the same force applied indicate varying object hardness. Fabrication employed a salting-out molding method using a double-sided flexible copper clad laminate (FCCL), PDMS mixture, and NaCl particles. The mass ratio of PDMS-to-NaCl particles was optimized, finding 1:2 to provide a balance between sensitivity and measurement range. Sensor characterization involved assessing capacitive variations under different pressures, triboelectric voltage outputs with various materials (PTFE and PI), and sensor response at different frequencies (1 Hz, 5 Hz, 10 Hz). Deep learning-assisted object recognition utilized a platform with a robotic grip, oscilloscope, and impedance analyzer for signal acquisition. The optimal gripping depth was determined by testing three samples with varying Shore hardness (PTFE ball, hollow rubber ball, and balloon). Twelve samples with diverse shapes, materials, and hardness levels were selected for object recognition experiments, with tactile information extracted from the bi-channel signals to form the dataset.

The optimized PDMS-to-NaCl mass ratio for the sensor was found to be 1:2, balancing sensitivity and measurement range. The capacitive sensing unit demonstrated a consistent increase in output signal with rising pressure, exhibiting good repeatability. The triboelectric sensing unit generated distinct signals when in contact with different materials, with significant differences in peak-to-peak voltage observed between materials like PTFE and PI. The sensor showed stable signals across various frequencies (1 Hz, 5 Hz, 10 Hz), with the triboelectric unit demonstrating a faster, more instantaneous response than the capacitive unit. The optimal gripping depth was determined to be 2mm, balancing sensitivity and accuracy in tactile perception. The sensor effectively distinguished among objects with varying hardness levels based on differences in both triboelectric and capacitive signals. For example, PTFE and billiard balls, both with high Shore hardness, were easily distinguishable due to opposite triboelectric signal polarities, while hollow and solid rubber balls showed significant differences in both triboelectric and capacitive signals. Testing with 12 diverse samples resulted in a 98.46% object recognition accuracy.

The results demonstrate the effectiveness of the proposed hybrid tactile sensor for object recognition. The integration of triboelectric and capacitive sensing units, along with deep learning, allows for accurate and comprehensive characterization of objects, considering both surface properties and material hardness. The high recognition accuracy (98.46%) validates the sensor's ability to differentiate among a wide range of samples. The sensor's ability to distinguish between objects with similar hardness levels based on contrasting triboelectric signals highlights the importance of multi-modal sensing. The findings address the research question by presenting a practical and effective solution for robust object recognition in complex environments. This approach could significantly advance robotics and human-machine interaction by enabling more intelligent and adaptable systems.

This paper successfully developed a novel hybrid triboelectric-capacitive tactile sensor for object recognition. The integration of two sensing mechanisms and deep learning resulted in a highly accurate system (98.46% accuracy for 12 samples). This research demonstrates the feasibility and effectiveness of using hybrid sensors for advanced tactile perception in robotic applications. Future research could focus on improving sensor miniaturization, exploring different materials for enhanced performance, and integrating more sophisticated deep learning algorithms for improved robustness and real-time processing.

The current study focused on a limited number of samples (12). While the recognition accuracy was high, further testing with a more extensive dataset would strengthen the generalizability of the results. The fabrication process, while simplified compared to multi-sensor systems, still requires optimization for mass production. The sensor's performance might be influenced by environmental factors such as temperature and humidity, which were not extensively investigated in this study. Further research is needed to assess the long-term durability and stability of the sensor under various operating conditions.