Deep learning-enabled real-time personal handwriting electronic skin with dynamic thermoregulating ability

The study addresses the need for electronic skin (e-skin) to dynamically regulate temperature, akin to human skin’s sweat gland-mediated thermoregulation, while also functioning as a self-powered human–machine interface. Existing e-skin thermoregulation approaches rely on porous materials to enhance heat dissipation or thermal insulators to reduce heat transfer, which handle only supercooling or overheating and lack dynamic adaptability. The authors propose embedding microencapsulated paraffin as a phase change material (PCM) into the e-skin surface to achieve dynamic thermoregulation around comfortable skin temperatures (30–34 °C), preventing leakage via microencapsulation. They further integrate a triboelectric nanogenerator (TENG) sensing mechanism to capture handwriting-induced signals and apply deep learning for accurate and real-time letter recognition, aiming to overcome environmental variability of TENG signals and eliminate the need for bulky powered handwriting systems.

Background work highlights: (1) E-skin functions such as flexibility, tactile sensing, breathability, and thermoregulation; (2) Conventional thermoregulation techniques (porous cooling materials, thermal insulation) lack dynamic temperature adjustment; (3) Human sweat glands dynamically regulate heat dissipation; (4) PCMs absorb/release latent heat near a phase transition with near-constant temperature, and paraffin offers high melting enthalpy and is environmentally friendly; (5) Microencapsulation prevents PCM leakage and minimizes performance impact during phase change; (6) TENG-based sensors are appealing due to low cost, simple structure, and low power but signals are environment-sensitive; (7) Deep learning has proven effective in pattern recognition and has been applied to TENG signal recognition and handwriting recognition, though achieving accurate, real-time display on e-skin remains challenging. This study builds on these findings to combine microencapsulated paraffin-based thermoregulation with TENG sensing and deep learning for robust, real-time handwriting recognition.

Materials and device fabrication: Paraffin was microencapsulated via a urea–melamine–formaldehyde route. Stage 1: In alkaline pH≈8, addition reactions formed a water-soluble prepolymer. Stage 2: Paraffin emulsification formed small oil droplets dispersed in the prepolymer with emulsifier adsorption. Stage 3: Under acidic pH 2.5–3 and heating, further polycondensation formed a crosslinked shell, yielding microencapsulated paraffin (M-paraffin). For ME-skin fabrication, 0.2 g M-paraffin was dispersed in ethanol (stirred 5 min), drop-cast on glass, ethanol evaporated, then a 1:1 mixed silicone elastomer (Parts A:B, stirred 15 min) was poured atop and cured at 60 °C for 4.5 h. The film was stripped off with tweezers. Resulting ME-skin thickness was ~480–500 µm, with an ~60 µm M-paraffin-rich layer concentrated on the back surface by the stripping method. Characterization: SEM (GeminiSEM 500) imaged M-paraffin (~30 µm average diameter) and ME-skin front/back/cross-sections, confirming encapsulation and layer distribution. DSC (TA DISCOVER DSC250) determined phase transition temperature (~31.1 °C) and melting enthalpy (161.92 J g−1) and cycling stability (5 heating–cooling cycles with negligible enthalpy loss). Mechanical testing (JITAI-5KN) compared stress–strain of silicone with/without M-paraffin; similar performance with slight fluctuation due to microcrack formation around microcapsules during stretch. Adhesion stability was verified after 200 stretches at 120% strain with negligible surface change. Thermoregulation tests: Temperature sensors (UT325) were placed between two stacked films (5×5 cm) of ME-skin or control silicone. Hot condition: samples placed on a film heater at 35 °C; both flat and curled configurations tested. Cold condition: samples equilibrated to same initial temperature then exposed to 16 °C air conditioning on glass; flat and curled configurations tested. Temperature vs time recorded to assess dynamic regulation. Electrical/TENG setup: ME-skin structured as a three-layer film (silicone/M-paraffin tribo-layer, insulating layer, copper electrode) to ensure body insulation. Single-electrode mode TENG operation was analyzed under contact–separation and contact–sliding with skin. Electrical measurements: open-circuit voltage (Voc) via oscilloscope (Tektronix TDS 2012C/MDO34 for real-time PC connection), short-circuit current (Isc) via low-noise preamplifier (SR570), transferred charge via integrating current. Load resistance during studies: 1 GΩ. Environmental dependencies examined: contact frequency, contact force, temperature (25–50 °C). Durability assessed after 1000 vertical presses and 1000 bending cycles. Human motion sensing evaluated by tap, finger bending (30°, 45°, 90° and slow bends), wrist up/down (30°, 60°), and elbow bends (45°, 90°). Handwriting data acquisition and preprocessing: Volunteers wrote letters on the ME-skin by sliding a finger. For representative sets, letters A, B, C, D were recorded; extended set included A, B, C, D, X, J, T, U; a challenging set of similar letters included I/L, U/O, S/Z, U/V. Signals were acquired via oscilloscope and transferred to LabVIEW in real time. Time–frequency analysis used STFT (0–20 s window, frequency band 10–30 Hz). For deep learning input, each sample underwent 0–1 normalization (scaling to [0,1]) and was resampled to length 1024. Some visualizations used t-SNE on principal components. Deep learning model: A 1D-CNN was designed with input window size 50 and training data segment length 200 data points (then resampled for model). Architecture: three convolutional layers (with 32, 64, 128 filters respectively), each followed by max-pooling, then flatten and a dense fully connected layer outputting class probabilities for eight letters. Dataset: For each of the eight letters (A, B, C, D, X, J, T, U), 100 samples were collected; split 80/20 into training/testing. Training was conducted to classify letters from raw voltage time series. A LabVIEW-based real-time system connected the ME-skin, data acquisition, and a Python-trained model for online recognition and display.

- Thermoregulation: DSC showed M-paraffin phase transition at ~31.1 °C with high latent heat (161.92 J g−1), aligning with comfortable skin temperatures (30–34 °C). In hot (35 °C) tests, ME-skin averaged ~2 °C cooler than control silicone, with up to 4 °C maximum difference. In cold (16 °C) tests, ME-skin cooled more slowly and maintained ~1 °C higher average temperature than control. Curled configurations showed stronger regulation and less environmental influence. - Electrical performance: Under palm tapping of a 5×5 cm ME-skin, average peak Voc ≈ 228 V, peak Isc ≈ 3.82 µA, and transferred charge ≈ 20.3 nC. Output increased with contact–separation frequency and force; 25–50 °C ambient had minimal impact on output. Signals from body motions (finger/wrist/elbow bending) were clearly detected; larger bending angles produced higher voltages; minimum reliably detectable finger bending angle was ~30° due to fixation stability. Signals remained stable after 1000 pressing and 1000 bending cycles. - Handwriting sensing and AI recognition: Distinct voltage signals were produced by sliding finger handwriting; time-only features were insufficient, and STFT revealed discriminative time–frequency characteristics. The 1D-CNN achieved 98.13% accuracy on eight-letter classification (A, B, C, D, X, J, T, U) with 80/20 train/test split (100 samples per class). For more confusable pairs (I vs L; U vs O; S vs Z; U vs V), accuracy was 90.71%. A real-time demonstration displayed handwritten sequences (e.g., “X-J-T-U”) in LabVIEW as recognized by the trained model. Signal characteristics were robust across 25–35 °C and similar on-table vs on-arm writing (amplitude reduced on skin but features preserved).

Embedding microencapsulated paraffin into a silicone e-skin provides passive, bidirectional thermal buffering near human comfort temperatures, closely mimicking sweat gland thermoregulation. The high enthalpy and stable cycling performance of M-paraffin, combined with localization of the PCM layer to one side, enables dynamic regulation with minimal impact on mechanical properties. Integrating a TENG sensing layer leverages self-powered operation to capture rich triboelectric signals from handwriting and motions. While TENG outputs are often sensitive to environmental factors, the study shows temperature insensitivity over 25–50 °C and uses deep learning to robustly extract discriminative features from noisy, variable signals. The 1D-CNN yields high-accuracy letter recognition and enables a real-time HMI demonstration, indicating strong potential for IoT interfaces, wearable systems, and VR/AR applications where simultaneous thermal comfort and intelligent interaction are desired.

This work introduces a microencapsulated-paraffin-based electronic skin that delivers dynamic thermoregulation around skin-comfort temperatures and simultaneously functions as a self-powered triboelectric handwriting and motion sensor. The ME-skin maintains mechanical performance, prevents PCM leakage, and demonstrates tangible thermal buffering (up to 4 °C differential at 35 °C, ~1 °C retention at 16 °C). A 1D-CNN model trained on triboelectric signals achieves 98.13% accuracy across eight letters and supports real-time on-screen display, while still performing well (90.71%) on challenging, visually similar letters. The approach provides a generalizable route to thermoregulating e-skins and advances their use as intelligent HMIs for IoT. Future work could scale to larger vocabularies and continuous handwriting, improve user-independence and cross-user generalization, integrate flexible electronics for on-device inference, and expand to multimodal sensing for richer interaction in wearable/VR/AR scenarios.

- Generalization across users was only preliminarily explored; handwriting variability (strength, speed, habits) can affect signals, and broader cross-user training/validation was not detailed. - The demonstration covered a limited set of letters; performance on full alphabets, numerals, or cursive/continuous inputs remains to be validated. - Similar-shaped letters reduced accuracy (to 90.71%), indicating remaining confusability in complex sets. - Stable finger-bend sensing required a minimum angle (~30°) due to fixation; attachment mechanics may impact sensitivity and repeatability. - The TENG requires an insulating layer when contacting skin; integration and long-term wear comfort in real-world conditions were not fully assessed. - The system relied on high input impedance (1 GΩ) instrumentation and external data acquisition/LabVIEW; fully integrated, low-power wearable implementations were not demonstrated.