Behavioral biometric optical tactile sensor for instantaneous decoupling of dynamic touch signals in real time

Behavioral biometrics, identifying individuals based on their interaction with devices, is a growing field. Analyzing writing speed, pressure, and movement offers unique authentication capabilities, reducing fraud. Dynamic pressure sensing is crucial for this, requiring accurate measurement of various forces. Optical tactile sensors, a cost-effective solution, are limited by their inability to accurately measure in-plane motion, often relying on imprecise multi-image averaging. This leads to "time-averaged" velocity estimations that may not reflect rapid directional changes. To address this, an optical tactile sensor capable of instantaneous velocity measurement (highly correlated with shear force) is needed for accurate behavioral biometric analysis. Existing optical sensors, using polymers, photonic crystals, or quantum dots, lack the speed and resolution for applications like handwriting analysis. This research aims to develop a highly sensitive optical tactile sensor that quantitatively decomposes applied force into normal and shear components from a single image in real-time, advancing behavioral biometric technologies.

Existing literature highlights the potential of behavioral biometrics for authentication, particularly using touchscreen input and dynamic pressure sensing. However, current optical tactile sensors struggle with dynamic force measurements, relying on multi-image processing leading to inaccuracies. While various materials have been explored for optical tactile sensors (optical polymers, photonic crystals, quantum dots), these often suffer from slow response times, high detection limits, and low spatiotemporal resolution, hindering their application in high-speed dynamic processes such as handwriting analysis. This work builds upon this existing research by focusing on developing a novel sensor capable of real-time, single-image analysis of dynamic forces, overcoming the limitations of previous approaches.

The researchers developed an optical tactile sensor consisting of a stress concentration layer, an anti-reflective platinum layer, and a signal-generating tactile layer containing upconversion nanocrystals (UCNs). The stress concentration layer, mimicking human skin's interface, amplifies and transfers forces. The UCNs, embedded in PDMS micro-hemispheres, generate unique luminescence signals based on contact with a total internal reflection (TIR) dove prism. Static normal forces produce axisymmetric luminescence, while dynamic shear forces generate non-axisymmetric signals. The UCNs offer advantages such as high spectral resolution, low optical errors, large anti-Stokes shifts, negligible autofluorescence, and optical stability. The sensor's response to normal and shear forces was systematically analyzed. The relationship between normal force and luminescence intensity, the hysteresis, stability, and response time were evaluated. The temporal changes in luminescence under lateral shear forces were studied, analyzing the transition from axisymmetric to non-axisymmetric profiles based on the indenter's velocity and direction. The friction force was quantified using a surface forces apparatus (SFA). A machine learning (ML) framework, using support vector regression (SVR) and linear discriminant analysis (LDA), was implemented for real-time force decoupling and handwriting analysis. Finite element analysis (FEA) was used to model stress distribution within the sensor and confirm the effect of the undulated interface on force transmission. The sensor's performance was evaluated through surface profiling of objects (golf balls, screws), fingerprint recognition, and Braille-to-speech translation.

The developed sensor successfully decoupled dynamic touch signals into normal and shear force components from a single image in real-time. The sensor demonstrated a high sensitivity to normal forces (as low as 0.05 N), showing a linear relationship between luminescence intensity and force magnitude. The response time was 9.12 ms. Applying shear force resulted in non-axisymmetric luminescence, with the degree of asymmetry correlating with the shear force magnitude and direction. The stress concentration layer significantly enhanced shear force transmission. A machine learning algorithm accurately decoupled normal and shear forces from single-frame images (r² values of 0.965 and 0.947, respectively). The sensor achieved high spatial resolution (100 µm), enabling detailed surface texture recognition (fingerprints, screws, golf ball dimples). A Braille-to-speech translation system was successfully implemented. Importantly, the sensor's dynamic force measurements enabled highly accurate handwriting biometric recognition using linear discriminant analysis (LDA), outperforming analyses based solely on normal force or velocity. The LDA clustering using combined normal and shear force features exhibited a Silhouette coefficient of 0.867 and a Calinski-Harabasz index of 5.76 × 10², indicating excellent clustering performance. Finite element analysis confirmed the enhanced shear force transmission due to the undulated interface in the stress concentration layer.

This research successfully addressed the challenge of instantaneous decoupling of dynamic touch signals in optical tactile sensors. The sensor's unique design, mimicking human skin's architecture and utilizing upconversion nanocrystals, provides superior sensitivity and spatiotemporal resolution compared to existing technologies. The integration of machine learning allows for real-time analysis and application in advanced biometric systems. The high accuracy of handwriting recognition, achieved by incorporating shear force data, opens new avenues for improved security and authentication systems. The ability to analyze both normal and shear forces within a single frame significantly enhances the speed and efficiency of the biometric process.

This study introduces a novel upconversion nanocrystals-based optical tactile sensor that achieves real-time decoupling of normal and shear forces from single images. This technology exhibits high sensitivity, speed, and spatial resolution, enabling various applications including high-resolution surface texture recognition, fingerprint identification, Braille-to-speech translation, and importantly, accurate dynamic handwriting biometric analysis. Future research could explore integrating this sensor into wearable devices and exploring other behavioral biometric applications.

While the sensor demonstrates high performance, some limitations exist. The current design focuses on specific material choices, and exploring alternative materials might broaden the applicability and optimize performance. The machine learning models were trained on specific datasets, and further training with diverse data could enhance the robustness and generalizability of the biometric recognition system. The accuracy of the handwriting analysis could potentially be impacted by the writing speed and pressure variations of different individuals and environmental factors, requiring further research to overcome such limitations.