Using Mobile Data and Deep Models to Assess Auditory Verbal Hallucinations

This paper explores the use of mobile data and deep learning models to assess auditory verbal hallucinations (AVH). Researchers collected ecological momentary assessments (EMA) four times daily for a month from 435 participants who experience hearing voices. Participants also recorded audio diaries and passively provided mobile sensing data. The study used VGGish and BERT for feature extraction from audio and text data respectively and a novel approach using VGGish to transform mobile sensing data. A hybrid neural network model, employing transfer learning and data fusion techniques, achieved a 54% top-1 and 72% top-2 F1 score in predicting AVH valence. The research demonstrates the potential of mobile technology for real-time AVH assessment.

Shayan Mirjafari, Andrew T Campbell, Subigya Nepal, Weichen Wang