Large Language Models (LLMs) like GPT-3 offer the potential to create more human-like and helpful chatbots. This paper presents a case study evaluating the feasibility of a GPT-3-based chatbot designed for improving mood and mental well-being. A randomized factorial experiment with 945 Mechanical Turk participants tested three prompt design dimensions: identity, intent, and behavior. Quantitative and qualitative analyses of conversations and user perceptions are presented, offering insights into prompt engineering for GPT-3 chatbots and providing a methodology for HCI designers and researchers to build upon.

Harsh Kumar, Ilya Musabirov, Jiakai Shi, Adele Lauzon, Kwan Kiu Choy, Ofek Gross, Dana Kulzhabayeva, Joseph Jay Williams