Performance and biases of Large Language Models in public opinion simulation

Large Language Models (LLMs) like ChatGPT offer a promising new approach to simulating public opinion, potentially aiding public policy development. However, concerns remain about their global applicability and potential biases across demographics and themes. This research uses data from the World Values Survey to assess ChatGPT's performance in diverse contexts, revealing significant performance disparities across countries and demographic groups (gender, ethnicity, age, education, and social class). Thematic biases in political and environmental simulations were also uncovered, highlighting the need for improved LLM representativeness and bias mitigation for equitable integration into public opinion research.

Yao Qu, Jue Wang