The metaverse, a network of 3D virtual worlds using VR technology, promises immersive experiences across various sectors. While head-mounted displays provide visual stimuli, wearable devices capable of sensing human motion and simulating sensations are crucial for full-body interaction. The hand, with its complex sensory and motor nerves, is particularly important for VR interactions. Existing finger motion tracking methods include camera-based rigid solutions and flexible solutions using various materials and sensing mechanisms (resistivity, capacitivity, optical fibers). Self-powered sensing (triboelectricity, piezoelectricity, thermoelectricity, pyroelectricity) offers advantages in low power consumption. Current VR sensing/feedback devices are often glove-based, complex, and require substantial power. This research aims to develop a highly integrated, multifunctional ring-shaped manipulator with a compact design, self-powered sensors, and low-voltage driven feedback units, compatible with IoT platforms for long-term VR use.

The paper reviews existing technologies for finger motion tracking and haptic feedback. It discusses conventional rigid and flexible sensing solutions, highlighting the advantages of self-powered sensing mechanisms for reducing power consumption. The literature also explores existing wearable devices for thermo-haptic feedback, using Joule heating, thermoelectric effects, and electrocaloric effects. The authors point out limitations of current glove-based VR devices, emphasizing the need for a more compact and energy-efficient solution, like a ring-based system.

The authors propose ATH-Rings, integrating triboelectric sensors (TENGs) for tactile sensing, pyroelectric sensors for temperature detection, eccentric rotating mass (ERM) vibrators for vibro-haptic feedback, and nichrome (NiCr) wire heaters for thermo-haptic feedback. These components connect to a wireless IoT module. The TENG sensor uses a silicone rubber film with pyramid structures as a negative triboelectric material and finger skin as the positive layer, with an aluminum film as the output electrode. The pyroelectric sensor detects temperature changes. The ERM vibrators and NiCr heaters are controlled by the IoT module using pulse width modulation (PWM). A voltage integration processing method is introduced to enable continuous finger motion detection from the TENGs, improving accuracy in gesture recognition. Machine learning (ML), specifically principal component analysis (PCA) and support vector machines (SVM), is used for gesture/object recognition. The system transmits sensory data wirelessly to an AI-enabled cloud server for analysis and feedback.

The voltage integration method for processing TENG signals proves superior to using only voltage amplitude for continuous motion detection, unaffected by bending speed. High-resolution continuous finger motion tracking is achieved. The ML-enabled gesture recognition system achieves 99.82% accuracy for 14 American Sign Language gestures using voltage integration signals, outperforming results based on pulse-like signals (97.14%). The voltage integration approach also excels in continuous sign language interpretation. The vibro-haptic feedback system provides adjustable vibration intensity, simulating the feeling of squeezing objects of varying hardness. The thermo-haptic feedback system uses NiCr wire heaters to accurately deliver temperature feedback, simulating grasping objects at different temperatures. Object recognition, combining TENG and PVDF sensor data, achieves 94% accuracy for five objects and over 96% for eight everyday items. A metaverse platform demonstration shows cross-space perception, where actions and sensations are shared between users in real and virtual spaces.

The ATH-Rings address the need for a compact, energy-efficient, and multimodal human-machine interface for immersive VR experiences. The superior accuracy of gesture recognition using voltage integration highlights the method's effectiveness. The successful demonstration of cross-space perception via the metaverse platform showcases the potential of the system for interactive virtual social experiences. The results contribute to advancements in human-computer interaction, particularly in areas like virtual training, virtual meetings, and virtual social interactions.

This research successfully developed a highly integrated and portable ATH-Ring system, offering multimodal sensing and haptic feedback for immersive VR interactions. The voltage integration method improved gesture recognition accuracy, and the metaverse platform demonstrated cross-space perception capabilities. Future work could explore full-body integration, improved haptic feedback mechanisms (cold sensation), and advanced AI algorithms for enhanced object recognition and interaction.

The current thermo-haptic unit has a relatively long cooling time and cannot provide cold feedback. The system uses the HTC Lighthouse tracking system for hand positioning, adding external dependency. Power consumption increases when multiple fingers are equipped with thermal heaters. Further research is needed to address these limitations and enhance the system's capabilities.