Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions

The paper addresses the need for somatosensory interfaces to enable immersive interactions in virtual reality and metaverse environments beyond visual and auditory channels. Hands and fingers, with dense sensory and motor innervation, are ideal for fine interaction, yet existing systems often rely on camera/IMU-based rigid solutions or flexible resistive/capacitive/optical-fiber sensors, which can be power-hungry and bulky. Self-powered sensing modalities such as triboelectricity, piezoelectricity, thermoelectricity, and pyroelectricity offer pathways to reduce power consumption. Concurrently, haptic feedback—vibrotactile and thermal—is essential for realism, but many glove-based devices have complex structures and require large/external power. The study proposes a compact, highly integrated ring-based human–machine interface (ATH-Ring) that combines self-powered triboelectric tactile sensing and pyroelectric temperature sensing with low-voltage-driven vibro- and thermo-haptic feedback, all operated by a wireless IoT platform. The core research questions are how to realize continuous, high-resolution finger motion sensing with triboelectric sensors suitable for mobile use, how to fuse multimodal sensing with haptic feedback for realistic VR interactions, and whether such a system can support accurate AI-based gesture/sign-language and object recognition to enable cross-space perception in the metaverse.

The authors review prior approaches to finger motion tracking and haptics: (1) Rigid, camera- or IMU-based solutions provide tracking but can be cumbersome. (2) Flexible/stretchable sensors use resistive, capacitive, or optical fiber mechanisms to capture deformation, offering better wearability but often needing continuous power. (3) Self-powered sensing using triboelectric, piezoelectric, thermoelectric, and pyroelectric effects reduces power consumption and has been applied in wearables for biomedical monitoring and HMIs. For haptic feedback, vibrotactile devices include electromagnetic, linear resonance, piezoelectric, and voice-coil actuators; recent wireless skin-integrated systems can display programmable vibrotactile patterns. Thermal feedback has been achieved via Joule heating (metal/carbon nanowire heaters), thermoelectric devices, and electrocaloric effects; wire heaters offer flexibility, easy integration, and localized heating. Multimodal glove systems integrating sensing with vibro-/thermo-haptics have been demonstrated but often require external power and complex structures. A compact, ring-based, multimodal, self-powered, and IoT-compatible solution had not been realized prior to this work.

System design: Five ATH-Rings, one per finger, connect to a custom wireless IoT module mounted on the back of the hand. Each ring integrates: (i) a triboelectric nanogenerator (TENG) tactile sensor for bending detection, comprising a silicone (Eco-flex) film with micro-pyramids (negative triboelectric layer), finger skin as the positive tribolayer, and an aluminum electrode; (ii) a PVDF pyroelectric temperature sensor (poled PVDF film with silver electrodes, PET packaging with grounded outer electrodes for noise reduction and thermal conduction); (iii) an eccentric rotating mass (ERM) vibrator for vibro-haptic feedback; and (iv) a nichrome (NiCr) wire heater embedded beneath the silicone layer for thermo-haptic feedback. TPU (85A) 3D-printed soft structures house and connect components. The IoT module includes analog front-end, ADC, MCU, PWM drivers for actuators, wireless transmission, and battery power. Signal processing: To obtain continuous motion information from the TENG, the authors compute the time integral of the load voltage, proportional to transferred charge Q = ∫i(t)dt = ∫v(t)/R dt. This “voltage integration” yields bending-angle-related signals largely invariant to bending speed, enabling continuous tracking and improved robustness versus using pulse amplitudes alone. Integration is performed on-board in the IoT module. Machine learning: Datasets were collected for 14 American Sign Language gestures. Signals from five fingers were normalized to mitigate inter-finger and inter-user variability. Features were reduced by PCA and classified using an SVM with linear kernel (C=1.0). For continuous sign sentences, the final stabilized integration values per finger were used to avoid dependence on preceding gestures. Object recognition used 5 grasped shapes (120 samples each; 80 train/40 test) and a separate dataset of 8 daily items. Classification used PCA+SVM on concatenated multi-finger time-series (e.g., length 300 per channel, 1500 features total). Haptic feedback control: Vibration amplitude is modulated via PWM voltage to the ERM; amplitude and frequency (130–230 Hz) vary with supply voltage. Control logic ties vibration intensity to object stiffness and deformation in VR: for soft objects, intensity increases with deformation up to a maximum; for rigid objects, intensity peaks at contact. Thermal feedback on the thumb ring uses PWM power to the NiCr heater to track target temperatures from virtual objects; a two-stage drive (fast warm-up then maintain) can reduce response time. Cross-space demo fuses shape (TENG) and temperature (PVDF) sensed by user 1 to reconstruct a virtual object that user 2 touches; collision events trigger vibro-/thermo-feedback. Fabrication and characterization: The TENG/Heater unit is fabricated by molding Eco-flex with pyramid grooves, attaching NiCr wire on the flat area, and encapsulating with Eco-flex. TPU components are 3D printed per Supplementary parameters. Electrical signals were measured with a high-impedance oscilloscope and electrometer; vibration characterized with piezo sensor and laser vibrometer; thermal response measured with IR camera and thermistor; heat flux via a commercial sensor. Hand pose in space used HTC Lighthouse tracking.

- Continuous bending sensing via voltage integration decouples bending angle from speed: integration values correlate with angle and are largely invariant to bending speed; slope encodes speed. Step and continuous bending are tracked with high resolution (1° increments distinguishable). Transferred charge trends match integration values. - Multi-finger robotic control: Five TENG sensors enable real-time continuous control of a robotic hand with minimal inter-channel interference; both abrupt and incremental motions are faithfully reproduced. - Gesture/sign recognition: Using integration signals with PCA+SVM achieves 99.821% accuracy on 14 ASL gestures (8 principal components), outperforming pulse-like signals (97.143%). Integration-based features improve clustering and robustness to human-induced speed variations. Durability tests show no output decay after thousands of cycles; accuracy remains stable. - Continuous sign sentences: Using final stabilized integration values enables reuse of single-gesture data in sentences without performance loss; 99.821% accuracy maintained. - Object recognition: Five grasped shapes (cube, cylinder, tri-pyramid, big ball, small ball) recognized with 94% accuracy; a separate dataset of 8 daily items exceeds 96% accuracy, indicating feasibility of shape inference from finger-bend patterns. - Vibro-haptic feedback: ERM vibration amplitude scales approximately linearly with drive voltage; frequency varies ~130–230 Hz with voltage, enhancing perceptual contrast versus fixed-resonance actuators. Demonstrations include squeezing a soft virtual ball (3-stage intensity logic) and multi-finger VR piano playing with per-finger feedback. - Thermo-haptic feedback: NiCr heater achieves target temperatures with low voltage; on-skin scenario reaches ~61.9 °C within ~9.4 s under ~3.5 W (IoT platform max). Heater temperature versus power is characterized; two-stage drive reduces time-to-target. Heat flux increases with power and is linearly related to heater temperature, enabling controlled thermal stimuli. - Pyroelectric temperature sensing: PVDF outputs a positive thermal peak upon contacting warmer objects; pressure artifact is a small preceding negative peak. Temperature-induced peak varies approximately linearly from 30–70 °C; pressure dependence introduces ~1.34 °C error (10–40 kPa), acceptable for applications. - Cross-space metaverse demo: User 1’s real object’s shape and temperature are sensed and reconstructed in VR; user 2 remotely perceives via vibro-/thermo-feedback. Measured feedback closely matches preset virtual temperatures; system shows good power efficiency and response with negligible interference between sensing and feedback units. - Integration and portability: All functions are battery-driven by a compact wireless IoT module; the ring form factor offers higher integration and portability than glove-based counterparts requiring external/large power.

The study demonstrates that combining self-powered triboelectric and pyroelectric sensing with low-power vibro- and thermo-haptics in a compact ring architecture can deliver rich, continuous somatosensory interaction for VR/metaverse applications. The voltage integration method addresses a key limitation of TENG-based sensing by providing a speed-invariant continuous measure of bending, improving both tracking and machine-learning-based interpretation. The AI pipeline (PCA+SVM) leverages the enhanced separability of integration features to achieve near-perfect gesture recognition and robust continuous sentence decoding, while grasp-induced finger-bend signatures enable practical object recognition. The haptic feedback architecture provides adaptive vibro- and thermal stimuli that correlate with virtual object properties (stiffness, temperature), enhancing realism. System-level demonstrations confirm cross-space perception: real-world sensations can be captured, transmitted, reconstructed, and felt remotely. This contributes a pathway toward more immersive social, educational, and training experiences in the metaverse. The ring-based, battery-powered, and IoT-compatible design underscores feasibility for long-term portable use, although thermal module refinements are needed to broaden sensations (including cooling) and reduce response/cooling times.

The authors present ATH-Rings, a highly integrated, battery-powered ring platform that fuses self-powered triboelectric tactile and pyroelectric temperature sensing with programmable vibro- and thermo-haptic feedback under an IoT framework. A key contribution is the voltage integration signal-processing method enabling continuous, speed-robust TENG sensing, which elevates AI-based gesture/sign recognition to 99.821% accuracy and supports reliable continuous sentence decoding. Object recognition from grasp patterns is validated (94–96%+), and adaptive vibro-/thermo-haptics simulate realistic interactions. A metaverse demo achieves cross-space perception where a remote user feels another’s real object in real time. Future work should focus on advancing thermal actuators (faster response, active cooling for cold sensation, reduced power), extending feedback to multiple fingers with optimized energy use, integrating additional modalities (force, texture), and broader user studies to assess ergonomics, generalization across users, and long-term wearability.

- Thermo-haptic module: relatively long cooling time; inability to generate cold sensations compared to thermoelectric cooling; response time increased by silicone encapsulation; higher power draw when scaling heaters to multiple fingers. - Thermal-on-skin performance requires higher power and longer time to reach target temperature than in-air measurements. - External positional tracking (HTC Lighthouse) is required for global hand pose in demonstrations; full self-contained tracking is not yet integrated. - As with many ML systems, performance may depend on normalization and dataset coverage; cross-user generalization, long-term drift, and extensive real-world variability warrant further study. - ERM placement atop the ring slightly attenuates vibration at the skin, though still perceptible; detailed psychophysical calibration across users remains to be explored.