Hand gesture interaction is becoming increasingly popular in virtual reality (VR), augmented reality (AR), and human-machine interaction (HMI) systems. Wearable devices, particularly electronic skins (e-skins), offer a promising avenue for capturing hand motion data for gesture recognition. However, single-mode sensors (e.g., those only measuring shape or position) lack comprehensive information, hindering accurate gesture interpretation. Furthermore, limited computing power in wearable devices restricts the fusion of multi-modal sensing data and the application of complex deep learning models. This research addresses these limitations by proposing a novel electronic skin capable of simultaneously capturing shape and positional data and employing an autonomous learning algorithm for efficient model deployment.

Existing wearable sensing platforms, while offering various advantages, often suffer from limitations in measuring finger joint shape and position. This lack of comprehensive information leads to an "intention gap" between user gestures and their accurate recognition. Magnetosensitive e-skins have shown promise but may still be limited in their ability to distinguish nuanced hand shapes, leading to ambiguities in gesture interpretation. This research builds upon the existing literature by developing a novel sensor design that overcomes these limitations through shape and position fusion.

The PFES utilizes a magnetostrictive Co70Fe30 alloy film as its sensing element. This film is sensitive to both strain (reflecting joint curvature) and magnetic fields (providing positional information). The PFES is designed with a hierarchical structure, including a magneto-elastic (ME) layer and a magneto-inductive (MI) layer. The ME layer generates an alternating magnetic field, while the MI layer measures changes in magnetic flux resulting from both bending and magnetic field variations. The output signal is a composite reflecting both curvature radius and magnetic field strength. The device fabrication process involves electrodeposition of the Co70Fe30 film. The performance of the PFES was characterized by measuring its response to different curvature radii and magnetic fields under various conditions (different sensor lengths, magnet sizes, MS layer thicknesses, and temperatures). A control circuit was designed to process the sensor output, which includes amplification, peak detection, analog-to-digital conversion, and Bluetooth transmission to a PC for data analysis. Haptic feedback is provided through a linear resonance actuator (LRA). An autonomous learning algorithm, based on Deep Reinforcement Learning-based Knowledge Distillation (DRL-KD), is implemented. This method uses multiple pre-trained models as "teachers" to train a smaller, more deployable "student" model. The student model dynamically selects knowledge from the teacher models that best suits the user's hand movement characteristics.

The experimental results demonstrate the effectiveness of the PFES in sensing both curvature and magnetic fields. The sensor exhibits good linearity in response to magnetic fields and reasonable linearity in response to curvature within a specific range. The sensitivity of the PFES was found to be optimal with a 20 µm thick Co70Fe30 film. The sensor shows good stability across a temperature range of 25-100 °C, with less than 4.2% measurement error. The response time was measured to be 17 ms for the maximum response and 39 ms for recovery. The PFES demonstrated excellent dynamic stability, with less than 1.6% change in sensitivity at frequencies of 1-4 Hz. The DRL-KD method enabled efficient compression and deployment of the gesture recognition model on wearable devices. The system effectively combines shape and position information for accurate gesture recognition and provides haptic feedback. The authors provide detailed characterization of the PFES including detailed figures demonstrating the sensor's response to different stimuli.

The PFES successfully addresses the limitations of existing single-mode sensors by integrating shape and position information. The use of a magnetostrictive material allows for a compact and flexible sensor design. The DRL-KD algorithm effectively reduces the computational burden, making the system suitable for deployment on wearable devices. The high sensitivity, fast response time, and temperature stability of the PFES make it suitable for various human-machine interaction applications. The results suggest that the integration of autonomous learning significantly improves the performance and efficiency of the gesture recognition system compared to traditional methods.

The developed PFES with autonomous learning demonstrates a significant advancement in wearable gesture interaction technology. The fusion of shape and position information, coupled with the efficient DRL-KD algorithm, offers improved accuracy and reduced computational requirements. Future work could focus on exploring more sophisticated machine learning algorithms for enhanced gesture recognition and expanding the range of applications beyond those presented.

While the PFES shows promising results, certain limitations exist. The sensor's nonlinearity at large strains could be further addressed through advanced signal processing techniques. The current study focused on a limited set of gestures; further research is needed to evaluate the system's performance with a broader range of hand movements. The long-term stability and durability of the device also require further investigation.