A deep-learned skin sensor decoding the epicentral human motions

The paper addresses the challenge of monitoring complex human body motions without dense sensor networks at each joint or muscle. Conventional approaches such as multi-site EMG and distributed strain/temperature sensors require many sensors, extensive wiring, and time-consuming preprocessing, and are affected by crosstalk and inter-user variability. Inspired by detection strategies for converging signals in large systems (e.g., seismology, radio astronomy), the authors hypothesize that a single, highly sensitive skin-like sensor placed at a location where motion information converges (e.g., wrist for fingers, pelvis for gait) can capture minute skin deformations that encode the epicentral motions of distant joints. Coupling this sensor with a deep neural network that learns temporal patterns should enable real-time decoding of complex motions (e.g., five fingers) from a single channel, improving efficiency and practicality over conventional methods.

Prior work directly measuring joint and muscle activity includes EMG-based gesture and intention decoding, which often requires many electrodes, suffers from signal coupling between neighboring muscles, and demands labor-intensive preprocessing. Soft, stretchable strain sensors and smart garments have been explored for motion tracking, typically necessitating multiple sensors across the body to cover curvilinear surfaces. Nanoscale crack-based sensors have shown ultra-high mechanosensitivity, with performance tuned via substrate thickness, modulus, annealing times, and stress concentrators; however, earlier studies lacked a clear mechanistic link between controllable fabrication parameters and crack/sensor performance. Additional works have inferred distal motions from proximal sites (e.g., foot from shin, knee from thigh, gait from pelvis), motivating the authors’ approach of exploiting converging information at select body locations.

Sensor design and fabrication: A crack-based, skin-like strain sensor is fabricated via digital laser processing. Colorless polyimide (CPI) is coated on glass, followed by spin-coating of silver nanoparticle (AgNP) ink. A 355 nm laser with high power (>100 mW) patterns the AgNP/PI bilayer into serpentine electrodes to enhance elasticity and conformal contact. Subsequently, lower laser power (6–13 mW) selectively sinters the AgNP layer to form a crack-inducible region. The patterned structure is peeled to obtain a free-standing sensor and embedded with adhesive PDMS (tuned with PEIE) for epidermal attachment. FEM analysis (COMSOL) of the serpentine geometry under 15% macroscopic strain shows effective electrode strain below ~2%, supporting durability and conformal sensing.

Crack mechanics characterization and modeling: Initial laser-annealed layers are subjected to quasi-static bending (linear stage at 0.05 mm/s) while recording resistance to study controlled crack initiation and propagation. A displacement-controlled bending model defines the projected cracked length and resistance ratio between cracked and non-cracked regions, linking measured resistance changes to local strain and crack growth. Using Irwin’s energy release rate framework, the team derives a resistance function dependent on void size (linked to porosity and grain structure from laser sintering), identifying three regimes by laser power: non-cracking (>~13 mW), stable cracking (optimal sensing window), and unstable cracking (<~6 mW). In the stable regime, higher laser power reduces void size and critical crack strain, enabling tunable sensitivity. A relation εc^2 ≈ 4 p b / L^2 links critical crack strain to propagated crack length p, indicating that larger grains (longer p) yield higher sensitivity. Gauge factor exceeds 2000 at ~0.55% strain in optimized sensors.

Data acquisition: Sensor signals are sampled at 40 Hz via a Keithley 7510 DMM for pretraining, recalibration, and real-time demos. Wrist strain mapping for validation uses digital image correlation (DIC) with applied speckle patterns.

Neural network for motion decoding: To decode five individual finger motions from a single wrist-mounted sensor, the authors define a 2D metric space: r encodes bend angle and ε encodes finger identity (spatial ordering of fingers). An encoding network of stacked LSTM layers processes 16-frame temporal windows to generate latent vectors capturing sequential patterns. A decoding network of two dense branches maps latent vectors to r and ε, respectively. Principal component analysis of latent vectors shows finger-specific cycles and alignment of straightened positions, indicating successful temporal feature learning. For noise-robust classification, the decoding network is adapted to a three-layer dense classifier outputting probabilities for eight classes (five finger bends plus touch, wrist bend, and twist).

Rapid Situation Learning (RSL): To address inter-user variability and sensor repositioning, the RSL pipeline uses transfer learning. Users follow on-screen prompts to collect ~8 s of data per finger at a new placement. A sliding window (size 16) forms inputs. Model weights (LSTM and dense layers) are initialized from a pretrained network and fine-tuned for ~5 minutes, substantially reducing data and time compared with training from scratch (~400 s data and >20 min training otherwise).

Additional demonstrations: The approach is extended to decode numpad typing (9 classes) combining wrist and finger signals, and to pelvis-mounted sensing for real-time ankle and knee position estimation (modified models described in supplementary notes).

• Single-sensor decoding: A single, laser-crack-based skin sensor on the wrist, combined with an LSTM-based deep network, decodes dynamic motions of five individual fingers in real time, mapping to a 2D metric space capturing bend magnitude (r) and finger identity (ε).
• Robustness to noise: With a modified classifier, the system distinguishes five finger motions from three non-finger motion noises (touching sensor, wrist twist, wrist bend) with an average classification accuracy of 96.2% and 92.9% in the worst finger class (little finger).
• Transfer learning (RSL): Rapid recalibration for new users or sensor positions requires only ~8 s per finger of data and ~5 minutes of fine-tuning using transferred parameters, reducing training time from >20 minutes to ~5 minutes for comparable loss (<0.1).
• Sensor sensitivity and tunability: In the stable cracking regime, sensors exhibit gauge factor >2000 at ~0.55% strain. Laser power controls porosity, void size, bonding energy, and thus critical crack strain and sensitivity, establishing a clear link between fabrication parameters and performance.
• Mechanics regimes: Identified non-cracking (>~13 mW), stable cracking (optimal sensing), and unstable cracking (<~6 mW) regimes via an energy release rate model; FEM shows serpentine patterns keep effective electrode strain under ~2% at 15% overall strain.
• Expanded applications: Demonstrated real-time decoding of 9-number numpad presses and gait estimation from a single pelvis-mounted sensor, generating ankle and knee positions in real time.

The study demonstrates that converging motion information can be captured at strategic skin sites with a single ultra-sensitive sensor and decoded via temporal deep learning, addressing the inefficiencies and crosstalk of multi-sensor EMG or dense sensor networks. The mechanistic model connects laser sintering parameters to crack behavior and sensitivity, enabling rational sensor optimization. The LSTM-based encoder-decoder exploits temporal patterns to distinguish finger identities and bend magnitudes even from a single channel, while transfer learning-based RSL mitigates inter-user and repositioning variability to enable practical deployment. The approach generalizes beyond fingers to tasks like keypress decoding and gait estimation from the pelvis, highlighting broad potential in rehabilitation, prosthetics, human–machine interfaces, and VR haptics. For broader adoption, ergonomic analysis to identify optimal placement sites on various body parts, strategies to determine minimal sensor counts, and integration with wireless platforms are important next steps.

This work introduces a deep-learned, laser-crack-based epidermal sensor system that decodes epicentral human motions from distant skin deformations using a single conformal sensor and temporal deep networks. It establishes a fabrication–mechanics–performance relationship for crack-based sensors and achieves real-time, noise-robust decoding of five finger motions, with rapid transfer learning-based recalibration for new users or positions. Extensions to typing and gait underscore the system’s versatility. Future research should focus on ergonomic mapping for optimal sensing locations across the body, minimal sensor configurations for complex tasks, long-term wearability and stability, and seamless wireless integration for everyday use.

• Generalization across users and placements still requires brief per-user/placement calibration (RSL), implying residual dependency on fine-tuning.
• The study primarily demonstrates single-finger bending and selected tasks (numpad, gait); broader, multi-DOF complex hand/arm activities and simultaneous multi-finger combinations are not comprehensively evaluated in the main text.
• Noise robustness was tested for three specific non-finger motions; other realistic activities and environmental perturbations were not exhaustively assessed.
• Practical deployment will require ergonomic analyses to select optimal sensing sites, determination of minimal sensor counts for whole-body applications, and wireless system integration.
• Long-term durability, biocompatibility over extended wear, and performance under sweat, temperature changes, and daily activities were not detailed in the main results.