Sentiment Analysis in the Era of Large Language Models: A Reality Check

Sentiment analysis (SA) has been a long-standing research area in natural language pro-cessing. With the recent advent of large lan-guage models (LLMs), there is great potential for their employment on SA problems. How-ever, the extent to which current LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capa-bilities of LLMs in performing various senti-ment analysis tasks, from conventional senti-ment classification to aspect-based sentiment analysis and multifaceted analysis of subjec-tive texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory per-formance in simpler tasks, they lag behind in more complex tasks requiring a deeper under-standing of specific sentiment phenomena or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their poten-tial when annotation resources are limited. We also highlight the limitations of current evalua-tion practices in assessing LLMs' SA abilities and propose a novel benchmark, SENTIEVAL, for a more comprehensive and realistic evalua-tion. Data and code are available at https://github.com/DAMO-NLP-SG/LLM-Sentiment.