Deep learning enabled smart mats as a scalable floor monitoring system

The study addresses the need for privacy-preserving, low-cost, and scalable indoor monitoring systems for smart buildings and homes. Camera-based surveillance raises privacy concerns and optical approaches (e.g., laser scanning) are limited and costly. Floors offer a ubiquitous interface to capture rich information from human walking, such as position, activity, and identity, but prior floor sensors (resistive, capacitive, piezoelectric, triboelectric) often lack scalability, are expensive, power-hungry, and require complex wiring and signal processing for large areas. The authors propose combining triboelectric sensing with large-scale printing to create self-powered, low-cost mats, and reducing system complexity by parallel-connecting mats with distinct electrode coverage patterns that yield distinguishable signal magnitudes. Integrating deep learning enables extraction of identity-related gait features from minimal two-channel signals, aiming to realize indoor positioning, activity monitoring, and individual recognition in smart buildings while minimizing privacy concerns and computational resources.

Prior work includes various floor sensors and localization systems using resistive, capacitive, piezoelectric, and triboelectric mechanisms, often demonstrated at small scale with complex wiring and high cost. Triboelectric sensors provide self-generated signals via contact electrification and electrostatic induction, offering simple configurations, broad material compatibility, scalability, and low cost. Large-area printing techniques (roll-to-roll, inkjet, screen printing) enable scalable fabrication. Methods to reduce electrode count include edge-electrode ratio readouts, which suffer at large scales due to weak signals, and parallel connection of patterned electrodes producing fingerprint-like signatures; however, large-area floor applications remained unshown. The convergence of AI/ML with IoT (AIoT) has enabled advanced analytics for authentication and activity recognition from sensor data. Building on these, the study adopts triboelectric sensing with screen-printed, distinct electrode coverage patterns and applies deep learning (CNN) to two-channel signals for gait-based recognition, addressing prior limitations in scalability, system complexity, and privacy.

Device design and fabrication: DLES-mats are triboelectric floor mats with a PET friction layer (125 μm, tribo-positive), a screen-printed silver (Ag) electrode (~15 μm) on a primer-treated PET surface, and a PVC substrate (80 μm) laminated beneath. A 2 cm × 2 cm opening in PVC exposes a connector pad; a copper wire is attached with conductive paste and sealed with Kapton tape. Individual mats are 42 cm × 42 cm PET, assembled onto a woolen floor. Distinct electrode coverage patterns (0%, 20%, 40%, 60%, 80%, 100%) are achieved on a 20×20 grid (20 mm squares) by filling selected squares; grid lines provide interconnection. A single mask and successive orthogonal orientations enable fabrication of 20%, then 40%, 60%, and 80% coverage, reducing cost. Operation principle and connection: Under identical contact conditions, induced charge Q scales with electrode area; open-circuit voltage Voc = Q/C. Parallel-connected mats share C, so different coverage areas yield distinct output magnitudes, enabling identification. PTFE soles (negative relative to PET) were used in many characterizations. To reduce electrode count and avoid signal interference, an interval parallel connection scheme is used: two sets of six mats (0–100% coverage) are alternately connected to two output electrodes (E1, E2), minimizing overlap of on/off pulses during walking. Electrical characterization: Output voltages were measured with an oscilloscope at 1 MΩ and 100 MΩ input impedances. With 100 MΩ, pulses have higher amplitude but longer RC discharge, causing pulse overlap and baseline drift; 1 MΩ provides clearer, faster-decaying pulses. Peak-to-peak voltage is used for analysis. Users wore PTFE soles; tests also evaluated cotton socks and EVA shoes. One-dimensional 12-mat lines and a two-dimensional 3×4 array (two electrodes total) were built for trajectory and activity tests. Position and activity sensing: Walking trajectories along rows/columns of the 3×4 array were executed in two cycles (alternating lead foot per cycle). Relative peak magnitudes per electrode correlate with coverage rate of the mat stepped on, allowing trajectory reconstruction. Activities (slow, normal, fast walking, running, jumping) were performed on the middle row; differences in amplitude and frequency enabled activity classification. Energy harvesting tests measured V–R and P–R characteristics for the array (normal walking) and individual mats (100% coverage) of 40×40 cm and 30×12 cm; capacitor charging and intermittent wireless sensor powering demonstrations were conducted. Deep learning analytics: Two-channel signals (E1, E2) were acquired in real time via Arduino MEGA 2560 and streamed to a laptop. For identity recognition, each sample comprises 1,600 data points per channel (two channels). Dataset: 10 users × 100 samples each (1,000 total), split 80%/20% train/test. A CNN with four convolutional blocks (filters: 32, 64, 128, 256; window size 50; max-pooling after each conv; flatten; dense) was trained for 50 epochs using categorical cross-entropy loss and Adam optimizer; accuracy used for evaluation. Additional experiments classified passing statuses (normal, fast, running) per user (12 classes total) and user-only regardless of status. A virtual smart building demo used Python for peak detection and Unity 3D for a digital twin to control lights and door access via TCP/IP commands, with CNN-based validation of users for access control.

- Output magnitude scales with electrode coverage rate (0–100%): maximum, minimum, and peak-to-peak voltages all increase with coverage; trends hold across different users and footwear (cotton sock, EVA shoe, PTFE shoe). - Interval parallel connection of mats to two electrodes eliminates overlapping pulses from simultaneous on/off steps, yielding more stable increment–decrement peak trends than simple parallel connection at both 1 MΩ and 100 MΩ loads. - A 3×4 array connected to only two electrodes enables real-time position and trajectory detection from relative peak magnitudes across E1 and E2; repeated cycles show consistent trends. - Activity monitoring distinguishes slow/normal/fast walking, running, and jumping based on signal amplitude and frequency. - Energy harvesting: For the 3×4 array under normal walking, maximum output power is 8.57 μW at 1.96 MΩ. For a 40×40 cm single mat (100%): Vmax ≈ 55.0 V at 100 MΩ; Pmax ≈ 169.46 μW at 9.10 MΩ. For a 30×12 cm mat (100%): Vmax ≈ 144.0 V at 100 MΩ; Pmax ≈ 800.84 μW at 13.79 MΩ. A 27 μF capacitor charged to 8 V powered a wireless sensor for one operation cycle, indicating support for intermittent IoT devices. - Deep learning identity recognition: CNN achieved 96.00% average accuracy on 10-user identification (1,000 samples). Status-aware classification (4 users × 3 statuses = 12 classes) achieved 89.17% accuracy. Collapsing statuses per user yielded 91.47% accuracy, indicating robustness across walking speeds. - Smart building demo: Real-time peak detection controlled virtual lighting at positions; full walking sequence triggered CNN-based valid/invalid user decision for door access. Minimal two-channel sensing reduced computational load compared with image/multi-channel systems.

The system demonstrates that distinct electrode coverage patterns combined with interval parallel connection enable a minimal two-electrode output to reliably encode position information across a multi-mat array. This reduces wiring and readout complexity, addressing a core barrier to large-area floor sensing. Triboelectric sensing eliminates external power for sensing, aligning with sustainable smart building goals and privacy requirements by avoiding cameras. Deep learning on compact two-channel waveforms successfully captures gait signatures for identity recognition with high accuracy, supporting secure access control and personalized automation. Activity differentiation and limited energy harvesting further broaden applications to healthcare monitoring and powering intermittent IoT sensors. Together, these capabilities address the research goal of a scalable, low-cost, privacy-preserving floor monitoring system for smart buildings and homes, and illustrate relevance to AIoT by enabling intelligent, responsive environments using lightweight analytics.

The work introduces deep learning-enabled smart mats (DLES-mats) that integrate self-powered triboelectric sensing, scalable screen-printed electrode patterns with unique coverage-based identities, and an interval parallel connection to achieve two-channel outputs from a 3×4 array. The system provides real-time indoor positioning, activity monitoring, and gait-based identity recognition, achieving 96% accuracy for 10-user identification and robust performance across walking speeds. Energy harvesting demonstrations show feasibility for powering intermittent IoT nodes. The approach offers a cost-effective, highly scalable, and privacy-preserving alternative to camera-based systems, suitable for automation, healthcare, security, and AIoT applications. Future research could scale arrays to larger areas and complex layouts, enhance robustness across footwear and environmental conditions, expand datasets to more users for generalization, reduce processing latency with embedded ML, integrate on-mat power management for energy harvesting, and explore adaptive electrode designs or multi-modal sensing to further improve accuracy and functionality.

- Scale and layout: Demonstrations used a 3×4 array; performance on much larger, irregular floor plans and long-term durability were not reported. - User diversity and dataset size: Identity recognition trained on 10 users (100 samples each); generalization to larger, more diverse populations and varied gait conditions (injury, carrying loads) remains untested. - Footwear and materials: While multiple contact materials were tested, PTFE soles were emphasized; broader variation (humidity, dust, different floor coverings) may affect triboelectric performance and require calibration. - Signal distortion at high impedance: 100 MΩ measurements exhibited pulse overlap and baseline drift; system relies on connection schemes and load selection to mitigate interference. - Processing latency: Real–virtual demonstration showed small delays due to processing; embedded real-time deployment may require optimization or edge ML acceleration. - Privacy vs. identification: Although more privacy-preserving than cameras, gait-based recognition still entails biometric processing; operational policies must address privacy and consent. - Energy harvesting: Harvested power supports intermittent operation; continuous powering of devices was not demonstrated and may require optimization or supplementary power.