Deep learning-assisted object recognition with hybrid triboelectric-capacitive tactile sensor

The study addresses the need for robust object recognition in human–machine interaction and robotic intelligence, where tactile sensing complements auditory and visual sensing. Tactile sensors convert mechanical stimuli into electrical signals to detect object properties such as surface texture and softness, which are valuable for object recognition. Prior approaches typically leverage single features (shape, texture, hardness), but single-function sensors can struggle with diverse objects in complex environments. Hence, integrating multiple tactile features can improve recognition accuracy. Multidimensional information can be obtained by (i) multi-sensor systems that integrate independent sensors at the expense of complexity and low integration, or (ii) hybrid tactile sensors that acquire multiple signals within a single device. To address the scarcity of single-sensor solutions that provide diverse signals, this work proposes a hybrid tactile sensor that integrates a triboelectric sensing unit (sensitive to surface material and texture) and a capacitive sensing unit (responsive to object hardness) within a unified porous-PDMS-based framework. Deep learning is employed to fuse bi-channel signals for accurate recognition of multiple samples.

The introduction surveys tactile sensing mechanisms—piezoresistive, piezoelectric, capacitive, and triboelectric—and their use in preliminary tactile perception and object recognition. It highlights common object features used for recognition: shape, surface texture (via sliding/pressing), and hardness. The literature also contrasts multi-sensor systems (providing comprehensive information but with higher complexity and low integration) with hybrid tactile sensors that can simultaneously acquire multiple signals. Examples include biomimetic piezoelectric tactile sensors for surface roughness recognition via machine learning, and a hybrid triboelectric–piezoresistive sensor enabling real-time texture/material recognition using a PR-CNN on a robot manipulator. The authors also reference their prior triboelectric–inductive hybrid tactile sensor for recognizing fruits with different packages with machine learning. The gap identified is the relative scarcity of single-sensor solutions that can obtain diverse signal types for robust object recognition, motivating the presented hybrid triboelectric–capacitive design.

Design and fabrication: The hybrid tactile sensor comprises two stacked units separated by a PI layer to avoid interference: (i) an upper triboelectric sensing unit and (ii) a lower capacitive sensing unit. Both use porous PDMS films and copper electrodes. The triboelectric friction layer and the capacitive dielectric are fabricated via the same porous PDMS process, simplifying manufacturing. Working principles: Triboelectric signals arise from contact–separation with objects; differences in material and contact area (texture) yield distinct polarities and amplitudes. The capacitive unit measures capacitance changes due to compression of porous PDMS under grip; for a given force, deformation depends on object hardness, enabling hardness inference. Fabrication process (salting-out molding): Starting from a double-sided flexible copper clad laminate (FCCL), the FCCL is embedded in upper/lower molds. A PDMS–NaCl particle mixture is poured and cured. After mold removal, a salting-out process creates porous PDMS films. The FCCL is folded to form the complete hybrid sensor. The PI interlayer separates the units. Characterization setup and parameters: Mass ratios of PDMS:NaCl were optimized by measuring capacitance change under pressure; 1:1, 1:2, and 1:3 ratios were compared, with 1:2 selected for high sensitivity over a larger range. An indenter (15 mm × 15 mm contact area) applied forces from 5 N to 45 N; capacitive signals were recorded to assess linearity, consistency, and repeatability. Triboelectric responses to different materials were tested at 0.5 Hz (e.g., PTFE vs PI showing opposite polarity and different peak-to-peak voltages). Frequency response was evaluated at 1, 5, and 10 Hz using PMMA contact on a tensile testing platform. Fatigue tests involved 3000 contact–separation cycles at 0.5 Hz to assess robustness of both channels. Robotic gripper platform and protocol: The hybrid sensor was mounted on a robotic gripper. Triboelectric and capacitive signals were acquired in real time using an oscilloscope and an impedance analyzer, respectively. Gripping depth optimization used three samples of varying hardness (PTFE ball, hollow rubber ball, balloon) at depths of 0, 1, 2, and 3 mm to balance sensitivity and discrimination; 2 mm was selected for subsequent experiments. A dataset was constructed from bi-channel signals for 12 samples with diverse shapes, materials, and Shore hardness. Deep learning was used to assist object recognition by fusing bi-channel triboelectric and capacitive signals; classification performance was evaluated across the 12 samples.

- A single hybrid sensor simultaneously captures triboelectric (material/texture) and capacitive (hardness) information, enabling richer object characterization from one mechanical stimulus. - Fabrication unifies the triboelectric friction layer and capacitive dielectric via the same porous-PDMS salting-out process, simplifying manufacturing. - Optimal porous structure: PDMS:NaCl mass ratio of 1:2 provided relatively high sensitivity over a larger measurement range compared with 1:1 and 1:3. - Capacitive unit exhibited steady, repeatable increases with pressure from 5 N to 45 N using a 15 mm × 15 mm indenter. - Triboelectric unit distinguished materials by polarity and amplitude; e.g., at 0.5 Hz, PTFE contact produced negative voltage on contact and positive on separation, opposite to PI, with significant peak-to-peak differences. - Stable operation across frequencies of 1, 5, and 10 Hz; triboelectric showed instantaneous contact/separation responses, while capacitive provided slower, continuous deformation responses indicative of hardness. - Robustness: After 3000 cycles at 0.5 Hz, both channels maintained resilient performance (fatigue data in supplementary figures). - Gripping depth optimization identified 2 mm as optimal, providing stable grasp and clear discrimination among objects of differing Shore hardness; 0–1 mm gave weak/insufficient discrimination, while 3 mm reduced sensitivity, especially for harder objects. - Application to 12 diverse samples achieved an object recognition accuracy of 98.46% using deep learning on fused bi-channel signals. Case examples: PTFE vs billiard balls (similar hardness) are distinguishable via opposite triboelectric polarity; hollow vs solid rubber balls show clear differences in both channels.

The proposed hybrid triboelectric–capacitive tactile sensor directly addresses the challenge of accurate object recognition with a single, integrated device by capturing complementary features: triboelectric signals encode surface material and texture, while capacitive signals quantify hardness. This bi-channel fusion reduces ambiguity inherent in single-feature sensing and improves robustness across diverse objects and object states. Employing the same porous-PDMS fabrication for both units streamlines production and enhances integration, facilitating deployment on robotic grippers. Experimental results demonstrate consistent, frequency-stable, and fatigue-resilient performance, and optimizing gripping depth further enhances discrimination. The deep learning classifier effectively leverages fused signals to reach 98.46% accuracy over 12 varied samples, validating that combining multi-modal tactile cues in one sensor substantially improves recognition performance for robotic perception and tactile intelligence.