logo
ResearchBunny Logo
Decoding lip language using triboelectric sensors with deep learning

Engineering and Technology

Decoding lip language using triboelectric sensors with deep learning

Y. Lu, H. Tian, et al.

Discover an innovative lip-language decoding system developed by Yijia Lu, Han Tian, Jia Cheng, Fei Zhu, Bin Liu, Shanshan Wei, Linhong Ji, and Zhong Lin Wang. Utilizing self-powered triboelectric sensors and a cutting-edge dilated recurrent neural network model, this research achieves an impressive 94.5% accuracy, facilitating barrier-free communication for the voiceless.

00:00
00:00
Playback language: English
Abstract
This paper introduces a novel lip-language decoding system using self-powered triboelectric sensors and a dilated recurrent neural network model based on prototype learning. The system effectively captures lip motions for vowels, words, phrases, and silent/voiced speech, achieving a 94.5% test accuracy with 20 classes and 100 samples each. Applications like identity recognition and toy car control demonstrate the system's feasibility and potential for barrier-free communication for the voiceless.
Publisher
NATURE COMMUNICATIONS
Published On
Mar 17, 2022
Authors
Yijia Lu, Han Tian, Jia Cheng, Fei Zhu, Bin Liu, Shanshan Wei, Linhong Ji, Zhong Lin Wang
Tags
lip-language decoding
triboelectric sensors
dilated recurrent neural network
prototype learning
communication
voiceless

Related Publications

Explore these studies to deepen your understanding of the subject.

Design of optical meta-structures with applications to beam engineering using deep learning
Engineering and Technology

Design of optical meta-structures with applications to beam engineering using deep learning

R. Singh, A. Agarwal, et al.

Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array
Engineering and Technology

Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array

C. Jiang, W. Xu, et al.

Deep learning-assisted object recognition with hybrid triboelectric-capacitive tactile sensor
Engineering and Technology

Deep learning-assisted object recognition with hybrid triboelectric-capacitive tactile sensor

Y. Xie, H. Cheng, et al.

Unraveling the effect of collagen damage on bone fracture using in situ synchrotron microtomography with deep learning
Engineering and Technology

Unraveling the effect of collagen damage on bone fracture using in situ synchrotron microtomography with deep learning

M. Sieverts, Y. Obata, et al.

Listen, Learn & Level Up
Over 10,000 hours of research content in 25+ fields, available in 12+ languages.
No more digging through PDFs, just hit play and absorb the world's latest research in your language, on your time.
listen to research audio papers with researchbunny