Usability Comparison Among Healthy Participants of an Anthropomorphic Digital Human and a Text-Based Chatbot as a Responder to Questions on Mental Health: Randomized Controlled Trial

Background: The use of chatbots in mental health support has increased exponentially in recent years, with studies showing potential effectiveness in treating mental health problems. More recently, visual avatars (“digital humans”) with anthropomorphic features and facial expressions have been introduced. It is important to study differences in emotional response and usability preferences between text-based chatbots and digital humans in mental health contexts. Objective: To compare usability between a digital human interface and a text-only chatbot interface using BETSY (Behavior, Emotion, Therapy System, and You), and to explore how chatbot-generated conversations on mental health affected self-reported feelings and biometrics. Methods: Healthy participants (n=45) were randomized to a digital human with anthropomorphic features (n=25) or a text-only chatbot (n=20). Usability was measured with the System Usability Scale (SUS-10). Emotional reactions were explored via electroencephalography (EEG), and feelings of closeness were self-reported. Groups were compared using linear regression and t tests. Results: No demographic differences were observed between groups. Mean SUS-10 was 75.34 (SD 10.01; range 57-90) for the text-only chatbot versus 64.80 (SD 14.14; range 40-90) for the digital human. Both interfaces were rated average or above average in usability. Women were more likely to report feeling annoyed by BETSY. Conclusions: The text-only chatbot was perceived as significantly more user-friendly than the digital human, although EEG measurements did not differ significantly between groups. Male participants exhibited lower levels of annoyance with both interfaces, contrary to prior reports.

Almira Osmanovic Thunström, Hanne Krage Carlsen, Lilas Ali, Tomas Larson, Andreas Hellström, Steinn Steingrimsson