Response-act Guided Reinforced Dialogue Generation for Mental Health Counseling

Virtual Mental Health Assistants (VMHAs) have become a prevalent method for receiving mental health counseling in the digital healthcare space. An assistive counseling conversation commences with natural open-ended topics to familiarize the client with the environment and later converges into more fine-grained domain-specific topics. Unlike other conversational systems, which are categorized as open-domain or task-oriented systems, VMHAs possess a hybrid conversational flow. These counseling bots need to comprehend various aspects of the conversation, such as dialogue-acts, intents, etc., to engage the client in an effective and appropriate conversation. Although the surge in digital health research highlights applications of many general-purpose response generation systems, they are barely suitable in the mental health domain—the prime reason is the lack of understanding in the mental health counseling conversation. Moreover, in general, dialogue-act guided response generators are either limited to a template-based paradigm or lack appropriate semantics in dialogue generation. To this end, we propose READER—a REsponse-Act guided reinforced Dialogue genERation model for the mental health counseling conversations. READER is built on transformer to jointly predict a potential dialogue-act for the next utterance (response-act) and to generate an appropriate response. Through transformer-reinforcement-learning with Proximal Policy Optimization (PPO), we guide the response generator to abide by the predicted response-act and ensure semantic richness of the responses via BERTScore in our reward computation. We evaluate READER on HOPE, a benchmark counseling conversation dataset, and observe that it outperforms several baselines across METEOR, ROUGE, and BERTScore. We also furnish extensive qualitative and quantitative analyses, including error analysis and human evaluation.

Aseem Srivastava, Ishan Pandey, MdShad Akhtar, Tanmoy Chakraborty