Topographic design in wearable MXene sensors with in-sensor machine learning for full-body avatar reconstruction

Full-body motion tracking and avatar reconstruction are important for applications in movement analysis, rehabilitation, human–machine interaction, and AR/VR. Camera-based systems, while accurate, are immobile, expensive, power-hungry, and raise privacy concerns, and typically require GPU-based external processing with high bandwidth and latency. Wearable strain sensors provide a conformal alternative for proprioceptive sensing across multiple joints; however, most existing devices have limited ability to customize the sensing working window to the deformation range of specific joints or muscles, resulting in erroneous signals and poor signal-to-noise ratios. Further, multi-channel sensor systems face challenges in continuous data transmission, storage, and processing. While Bluetooth-enabled streaming to external devices is common, near-/in-sensor processing can reduce bandwidth and power while improving latency and data security. The research addresses two core questions: (1) how to design wearable strain sensors with tunable, joint-matched linear working windows without sacrificing sensitivity; and (2) how to integrate edge computing to perform accurate, power-efficient in-sensor machine learning for multi-joint, multi-mode full-body motion monitoring and avatar reconstruction.

Prior motion monitoring uses camera-based systems that require stationary, costly hardware and GPU computation, with privacy and bandwidth constraints. Wearable approaches using strain or inertial sensors have been explored for motion tracking and gesture/sign recognition, but often focus on maximizing sensitivity via material composition rather than customizing linear working windows for specific strain ranges. Existing works typically stream data wirelessly to external devices or perform single-channel processing; multi-channel in-sensor/edge computing for full-body motion has been limited. Reviews highlight the need for power-efficient, local (near-/in-sensor) machine learning to handle large sensor data volumes and reduce communication overhead. This work builds on MXene-based piezoresistive sensing literature showing high conductivity, processability, and sensitivity, extending it with a scalable topographic design to tune working windows and an integrated edge ML pipeline for avatar reconstruction, which the authors note has not been previously realized for multi-joint full-body monitoring.

Materials and nanolayer preparation: Ti3C2Tx MXene nanosheets were synthesized by etching Ti3AlC2 MAX with LiF/HCl, followed by washing, ultrasonication, and centrifugation to obtain a ~5 mg/mL suspension. SWNTs were dispersed in SDS by probe sonication (~0.1 mg/mL). MXene, SWNTs, and PVA were mixed (typical mass ratios MXene/SWNT/PVA = 85/10/5 unless varied) and vacuum-filtered onto PVDF membranes to form ~400 nm composite nanolayers (ps-MXene). The freestanding films were released in ethanol. Topographic design via localized thermal contraction: ps-MXene nanolayers were transferred onto oxygen-plasma-treated uniaxial PS shrink films. Samples were thermally contracted at 100 °C for 120 s. By constraining different regions during contraction, distinct topographies were created: planar (Mp), fully wrinkled (Mw), center-wrinkled (Mp-w-p), and edge-wrinkled (Mw-p-w). After contraction, PS was dissolved in DCM to obtain freestanding textured films, which were transferred to VHB tapes. Copper leads with silver paste were added to fabricate strain sensors. Characterization and simulation: SEM and AFM characterized wrinkle morphology (wavelength ~8–10 µm). Four-point probe measured conductivity (e.g., ~2479 S/cm for Mp at 85/10/5). FEA simulated strain distribution under uniaxial stretching for different topographies and stretching directions, correlating with in situ reflection microscopy and SEM to observe crack initiation/propagation (long/continuous cracks in planar regions vs short/zigzag cracks confined to wrinkle valleys in textured regions). Cycling stability (up to 20,000 cycles), response times, hysteresis, Young’s modulus, and fabrication reproducibility (signal error) were evaluated. Tuning sensing performance: Effects of stretching direction (parallel vs perpendicular to wrinkle axes), areal percentage, and spatial distribution of wrinkled regions were studied. Under parallel stretching, linear working windows and gauge factors (GF) were determined for Mp (3–6%, GF ~3400), Mp-w-p (8–24%, GF ~1160), Mw (25–39%, GF ~1230), and Mw-p-w (35–50%, GF ~1470); εmax respectively ~6%, 24%, 39%, and 50%. By varying wrinkled area from 5% to 75%, linear working windows increased (e.g., 12–14% to 45–69%). Placing wrinkled regions at edges (p-w-p-w) reduced localized strain and increased εmax (e.g., up to 50%). Composition and thickness variations were compared; topographic design was most effective for tuning εmax (overall tunability of linear working windows from 6% to 84% while maintaining GF > 1000). Wireless sensor module: Seven sensors with joint-matched windows were selected based on measured joint strain ranges of a volunteer: back waist (~5%), shoulders (~10%), elbows (~30%), knees (~50%), using Mp (3–6%), Mp-w-p (8–24%), Mw (25–39%), and Mw-p-w (35–50%) respectively. Sensors were interfaced with a 16-bit ADC (AD7606), MCU (PCA9658), and Bluetooth module (HC-06). Each channel used a 100 kΩ series resistor and 5 V input; ADC voltage outputs were converted to sensor resistance. Wireless transmission performance included data rate ~400–450 bps over >80 m and >100 h, with transmission errors <4% using 100 kΩ series resistors. Machine learning for motion classification: Multi-channel time-series data from six motions (elbow/shoulder lifting L/R, squatting, stooping, walking, running) were streamed and used to train an ANN. Dimensionality reduction (t-SNE) visualized separable clusters. Independent test data were used to evaluate classification accuracy. Edge sensor module and in-sensor CNN: An edge module integrated seven sensors with a multi-channeled ADC and an ML chip (with Bluetooth for intermittent result transmission). A CNN was trained offline to map multi-channel sensor time series to 2D positions of 15 avatar joints. Ground-truth joint positions and body proportions were obtained from a prerecorded video using OpenPose; the trained CNN was deployed to the ML chip for real-time in-sensor inference. Avatar animations were reconstructed from inferred joint coordinates; determination error was computed against video-extracted ground truth. Power consumption for wireless versus edge architectures was calculated for the avatar reconstruction task.

- Programmable topographic design of MXene/SWNT/PVA piezoresistive nanolayers via localized uniaxial thermal contraction yielded homogeneous and heterogeneous wrinkle patterns that control crack propagation and localized strain. - Under parallel stretching, sensors exhibited tunable linear working windows with high sensitivities: Mp 3–6% (GF ~3400, εmax ~6%), Mp-w-p 8–24% (GF ~1160, εmax ~24%), Mw 25–39% (GF ~1230, εmax ~39%), Mw-p-w 35–50% (GF ~1470, εmax ~50%). Overall, linear working windows were tunable from 6% to 84% across designs while maintaining GF > 1000. - Increasing the areal percentage of wrinkled regions (5%→75%) broadened the linear working window (e.g., 12–14%→45–69%). Placing wrinkled regions at edges reduced localized strain (e.g., p-w-p-w average localized strain ~20% at 120% stretching) and increased εmax (up to ~50%). - Crack behavior: planar regions formed long/continuous fractures; textured regions exhibited short/zigzag cracks confined to wrinkle valleys, preventing complete conductive pathway failure and enabling wider working windows and improved reproducibility (signal error <10% for textured sensors vs >50% for planar-only sensors). - Mechanical and stability performance: Young’s modulus ~150 kPa (vs VHB ~106 kPa), stable cycling up to 20,000 cycles; measured hysteresis values increased with strain (e.g., 18–31 for different designs at 5–40% applied strains). - Wireless module performance: multi-channel data rate ~400–450 bps over >80 m and >100 h; wireless transmission error <4% with 100 kΩ series resistors; higher, strain-dependent errors with lower-value resistors (e.g., 5 kΩ). - Motion classification: ANN trained on seven sensors (joint-matched windows) achieved 100% accuracy classifying six full-body motions without any image/video data; t-SNE showed six distinct clusters. - Edge computing avatar reconstruction: In-sensor CNN on an ML chip reconstructed personalized avatars (15 joints) with an average determination error of 3.5 cm relative to video-derived ground truth. - Power efficiency: Edge module consumed ~9 mW for avatar reconstruction versus ~31.5 mW for the wireless streaming approach (~71% reduction), excluding terminal device compute power in the wireless scenario.

By engineering wrinkle-like microtextures through localized thermal contraction, the authors controlled crack initiation and propagation in MXene-based piezoresistive layers, enabling precise tuning of linear working windows across a wide strain range while preserving ultrahigh sensitivity. This customization allows matching sensor characteristics to specific joint deformation ranges, improving signal-to-noise and avoiding saturation or low-amplitude signals that arise from a one-size-fits-all sensor approach. The multi-type sensor suite captured proprioceptive signals across multiple joints and motions, supporting accurate ANN-based classification of six full-body activities. Integrating a CNN into an edge ML chip enabled real-time, in-sensor inference of avatar joint positions without continuous data streaming, addressing bandwidth, latency, and privacy concerns inherent to external processing. The average avatar determination error of 3.5 cm demonstrates high fidelity in reconstructing complex, continuous motions. The edge architecture substantially reduced system power, critical for wearable deployments with limited battery capacity. Overall, the findings show that topographic design coupled with edge ML bridges materials-level customization and systems-level efficiency, advancing full-body motion monitoring and avatar reconstruction.

This work introduces a scalable, controllable topographic design strategy for MXene-based wearable strain sensors that programmatically tunes linear working windows (6–84%) without compromising sensitivity (GF > 1000). Leveraging these joint-matched sensors, the authors demonstrated a wireless module for multi-channel data streaming and 100% accurate ANN-based motion classification, and an edge sensor module with an in-sensor CNN for real-time personalized avatar reconstruction with an average joint location error of 3.5 cm. The edge approach achieved 71% lower power consumption than continuous wireless streaming. These contributions showcase an end-to-end pathway from materials engineering to integrated systems for accurate, power-efficient full-body motion monitoring. Future research could generalize and validate the approach across diverse users and activities, enhance comfort and ergonomics of sensor modules, extend to 3D avatar reconstruction, and explore deployment in challenging environments (e.g., underwater, mobile, or privacy-sensitive settings) and applications in sports performance, rehabilitation, and human–robot interaction.

- Personalization and generalizability: Sensor window selection and CNN training were based on a single volunteer’s joint strain ranges and video-derived skeletons; performance across broader populations, body types, and activities remains to be validated. - Calibration data requirement: Avatar reconstruction requires initial video-based extraction of joint locations (OpenPose) for figure parameters and training labels, introducing a dependency on camera data during setup. - Activity scope: Motion classification was demonstrated on six activities; scalability to more complex or subtle motions (e.g., fine hand gestures, rapid transitions) requires further study. - Wireless module constraints: The Bluetooth-based wireless approach has modest data rates (~400–450 bps) and may face interruptions; although mitigated by edge processing, these factors can impact real-time performance in some scenarios. - Environment and durability: While cycling stability is shown up to 20,000 cycles, long-term wear, environmental factors (sweat, temperature, washing), and multi-user donning/doffing effects were not reported.