Silent speech recognition offers a versatile communication method, particularly beneficial for individuals with aphasia and in challenging environments (obscured, dynamic, quiet, dark, noisy). Unlike sign language or typing, silent speech is intuitive, requiring minimal training. This research focuses on developing a natural and all-weather silent speech recognition system (SSRS). The system addresses challenges related to acquiring high-fidelity sEMG signals from the face, accurately classifying them, and creating imperceptible wearable devices. Facial sEMG signals exhibit significant spatiotemporal variability, necessitating robust classification algorithms. Previous research explored various classifiers (SVM, deep learning, machine learning), but mostly relied on non-flexible electrodes and laboratory settings, lacking long-term performance evaluations. This study aims to improve upon existing methods by leveraging lightweight, bendable, stretchable tattoo-like electronics to conform to the dynamic nature of facial skin, combined with a cloud-based machine-learning algorithm to reduce the device's complexity and improve accuracy. The use of machine learning is preferred over deep learning due to its superior performance with small sample sizes and faster processing, suitable for real-time recognition.

Silent speech recognition using EMG has been explored since the mid-1980s. Early studies focused on classifying limited sets of vowels or words. Over time, the number of classifiable words has increased, with more recent research demonstrating high accuracy (>90%) in recognizing hundreds or even thousands of phrases. However, most previous work utilized rigid, uncomfortable electrodes and lacked extensive real-world testing. The advancement of machine learning and deep learning techniques has contributed to this progress. This work distinguishes itself by employing tattoo-like electronics, addressing the limitations of existing systems regarding comfort, flexibility, and real-world applicability. The cloud-based approach further streamlines the technology by reducing the computational demands on the wearable device itself.

The proposed SSRS integrates four components: four-channel tattoo-like electronics, a wireless data acquisition (DAQ) module, a server-based machine-learning algorithm, and a terminal display. The tattoo-like electrodes, approximately 1.2 μm thick and integrated within a skin-like patch, are designed with filamentary serpentine patterns for improved stretchability and conformability to the skin. Four pairs of electrodes are strategically placed on facial muscles (levator anguli oris (LAO), depressor anguli oris (DAO), buccinators (BUC), and anterior belly of digastric (ABD)) to enhance signal quality. An "electricity-preferred" method is used to maintain electrical conductivity even under significant stretch. The wireless DAQ module amplifies and filters sEMG signals, digitizes them, and transmits them to a cloud server via Bluetooth. The cloud server uses linear discriminant analysis (LDA), a machine-learning algorithm suitable for small datasets and multi-label classification, to recognize the silent speech. A dataset of 110 frequently used words, divided into 13 categories, was used for training and testing. The methodology includes assessment of electrode performance under large deformation, evaluation of long-term wearability, and extensive real-world testing across various scenarios. Signal processing steps involve active segment interception to differentiate between silent speech and other movements, wavelet packet decomposition for denoising, and extraction of time and frequency-domain features. The performance of the LDA classifier was compared to SVM and naive Bayesian models.

The tattoo-like electrodes demonstrated excellent conformability and stretchability, adapting to significant facial deformation (~45%) without compromising signal quality. Long-term wearability tests (10 hours) showed stable performance, with noise and impedance remaining low even after strenuous activity. The LDA algorithm achieved a high average accuracy of 92.64% in recognizing 110 words. Comparison with other machine-learning models (SVM, NBM) demonstrated LDA's superior performance in terms of accuracy, F1-score, training speed, and prediction speed. Real-world testing across various scenarios (greeting, exercise, dining, noisy environment, darkness) confirmed the system's robustness and applicability. The system maintained high accuracy (≥85%) even with the loss of up to one sEMG channel, demonstrating its resilience to potential failures. The recognition rate of ASR was compared to the proposed system. The result shows that ASR is vulnerable to noise while SSRS remains unaffected by noise and maintains high accuracy.

The results demonstrate the feasibility and effectiveness of using tattoo-like electronics and machine learning for all-weather, natural silent speech recognition. The system's high accuracy, robust performance in various environments, and ease of use represent significant advances over existing methods. The imperceptible nature of the tattoo-like electrodes provides a comfortable and unobtrusive user experience. The cloud-based approach reduces computational burden on the wearable device, facilitating portability and scalability. The system’s success in real-world scenarios highlights its potential applications for various user groups, including people with communication disabilities and those requiring hands-free communication in challenging environments. The results indicate a viable approach for various applications, including communication aids, human-machine interaction, and assistive technologies. Future work may explore expanding the vocabulary size, integrating other biosignals, and developing more sophisticated machine-learning models.

This study successfully demonstrates an all-weather, natural silent speech recognition system using innovative tattoo-like electronics and a cloud-based machine-learning algorithm. The system achieves high accuracy while maintaining comfort and ease of use. This work opens new possibilities for communication aids, human-machine interfaces, and assistive technologies for diverse populations. Future research directions include investigating alternative materials and fabrication processes for the electrodes, developing more sophisticated machine learning models, and expanding the vocabulary size to incorporate a wider range of words and phrases.

While the study demonstrates impressive results, some limitations exist. The dataset used for training and testing the algorithm was relatively small, potentially limiting the generalization of the findings. Further research with larger datasets and more diverse participants is necessary. Additionally, the long-term stability of the electrodes and the system’s performance in extreme environmental conditions could be further investigated. The current system is dependent on a cloud server; future development could focus on making the system work offline.