Encouraging cooperation within human groups remains a significant challenge. This study investigates the application of deep reinforcement learning to design a 'social planner' AI that can optimize cooperation within a network of interacting individuals. The spread of pro-social and anti-social behaviors through social networks is well-documented; cooperation is contagious, but so is selfishness. Previous research has explored assortative mixing—where cooperators connect preferentially with other cooperators and defectors with other defectors—as a mechanism to mitigate the negative impact of antisocial behavior. This strategy works by protecting cooperators and potentially punishing defectors through isolation from beneficial cooperative relationships. This paper builds upon this research by leveraging the power of deep reinforcement learning to create a social planner that can dynamically adjust network connections to maximize cooperation. The use of machine learning in social contexts is increasingly prevalent, making this approach particularly relevant for understanding and improving human interaction in networked environments. The core research question is: Can a deep reinforcement learning agent, trained in a simulated environment, learn to structure a human network to foster and enhance cooperation?

Existing literature highlights the contagious nature of both cooperation and defection in social networks. Studies have shown that social contact can spread pro-social behavior, creating cascades of cooperation within groups. Conversely, antisocial behavior can also spread contagiously, leading to a propagation of selfishness. Assortative mixing has emerged as a key strategy to encourage cooperation, with studies demonstrating that allowing individuals to form and break links based on shared strategies can promote clustering and cooperation. Methods such as embedding cooperative 'bots' within networks have also shown promise in fostering homophilic clusters. These approaches suggest that separating cooperators from defectors can prevent the spread of defection and incentivize prosocial behavior. Research on hunter-gatherer tribes further supports the idea that assortative mixing may have played a role in the evolution of human cooperation. This study expands on previous work by exploring the potential of deep reinforcement learning to discover novel strategies for scaffolding cooperation, offering a more adaptive and dynamic approach compared to rule-based systems.

The researchers developed a network cooperation game where players (either simulated bots or human participants) are positioned on the nodes of a graph, and edges represent connections between them. Players accumulate capital by cooperating with or defecting from their neighbors. Cooperation involves a cost to the cooperator but provides benefits to their neighbors. A social planner observes player choices and network structure and makes recommendations to add or remove connections. Players can accept or reject these recommendations. The researchers used deep reinforcement learning to train a Graph Neural Network (GraphNet) to act as the social planner. The GraphNet was trained in simulation using a reinforcement learning algorithm (advantage actor-critic) to optimize for high levels of group capital and recommendation quality. Simulated human behavior was modeled using logistic functions, informed by parameters fitted to data from baseline conditions in later experiments with real human participants. The trained GraphNet social planner was then evaluated in experiments with 16-player human groups (N=768 participants across 48 groups) playing the game for real monetary rewards. The GraphNet planner's performance was compared against several baseline strategies: a static network (no rewiring), random recommendations, and a cooperative clustering strategy that separates cooperators from defectors. Generalized linear mixed models were used to analyze individual cooperation decisions, while group-level linear models assessed overall group outcomes. Additional follow-up studies investigated the specific strategy learned by the GraphNet planner, comparing it to a simpler rule-based 'encouragement planner' and additional control conditions to test the impact of network density on cooperation.

The GraphNet social planner significantly outperformed all baseline conditions in promoting cooperation. Groups under the GraphNet planner ended the game with an average cooperation rate of 77.7%, significantly higher than the static network (42.8%), random recommendations (57%), and cooperative clustering (61.2%). The GraphNet planner achieved this by employing a conciliatory approach to defectors, unlike the cooperative clustering method which aimed to isolate defectors. Instead of isolation, the GraphNet encouraged defectors to connect with cooperators, initially facilitating interaction, and later prioritizing the protection of cooperators by removing cooperator-defector links. This strategy created networks with a core-periphery structure where cooperators formed a densely connected core, and defectors were located in smaller, highly cooperative neighborhoods. Analysis of the GraphNet's recommendations showed a conditional approach, taking into account player cooperation choices and the round number. Importantly, despite defectors receiving higher average payoffs than cooperators, cooperation thrived due to factors beyond simple utility maximization, possibly including social norms and preferences for fairness. Follow-up studies confirmed the efficacy of this conciliatory approach, showing that an 'encouragement planner' based on the GraphNet's strategy also significantly improved cooperation. Control conditions manipulating network density confirmed that high density alone was not sufficient to drive the improved cooperation rates observed with the GraphNet and encouragement planners. The GraphNet planner's strategy does not create assortative mixing between cooperators and defectors, but rather a core-periphery structure.

The findings demonstrate that a deep reinforcement learning agent can effectively learn to scaffold cooperation in human groups, exceeding the performance of established strategies focused on assortative mixing. The unexpected finding that defectors received higher average payoffs underscores the complexity of human behavior in cooperative settings, highlighting factors beyond simple utility maximization. The success of the GraphNet planner suggests that encouraging interaction between cooperators and defectors, while gradually protecting the core of cooperators, is a more effective strategy than strict separation. The core-periphery structure emerges as a key characteristic of successful networks. This work contributes to a broader understanding of how to design and engineer social structures to promote pro-social behavior. The conciliatory approach learned by the AI represents a novel approach with significant implications for managing collective action problems.

This study presents a novel approach to fostering cooperation in human groups using deep reinforcement learning. A social planner AI, trained via simulation, effectively enhanced cooperation rates by employing a conciliatory approach towards defectors, forming a core-periphery network structure. This method outperformed established strategies focused on isolating defectors. Future research could explore scaling this approach to larger networks, improving interpretability of the AI's decision-making process, and investigating the interplay between economic and social factors in shaping cooperative behavior.

While the study demonstrates the effectiveness of the AI social planner in a controlled experimental setting, the generalizability to real-world scenarios requires further investigation. The specific design of the game and the incentives used might influence participant behavior. The simulated human models used in training were based on specific parameters fitted to data from baseline human groups; these parameters might not capture the full complexity of human decision-making. Finally, the relatively small size of the networks studied limits the extent to which findings can be extrapolated to larger, more complex networks.