The ability to learn from feedback and adapt behavior is crucial for achieving goals. While previous research has examined age-related changes in learning, most studies used simplified tasks where outcomes solely depend on individual actions. Real-world scenarios often involve external, uncontrollable factors influencing outcomes. Understanding how individuals across different ages infer the causality between their actions and outcomes, especially in the presence of external factors, is important. Prior work shows adults use inferences about environmental controllability to guide learning. They update value estimates less when outcomes are attributed to external causes. This study examines whether this pattern is present across development, investigating whether children and adolescents use their understanding of external causal agents to inform their reinforcement learning, similar to adults. Infants show an early capacity to infer hidden causes, linking them to both deterministic and probabilistic events. Throughout childhood and adolescence, individuals encounter increasingly complex environments with external causal agents that affect the controllability of outcomes. A prior study in adults demonstrated that beliefs about external, asymmetrically influencing valenced outcomes led to asymmetries in learning. Adults learned more from positive outcomes when an adversarial agent caused negative ones and vice-versa. This suggests that rational use of causal beliefs guides value updating, which may underpin age-related learning differences. To test the hypothesis that the use of causal attributions to guide learning increases with age, the researchers used a modified version of a reinforcement learning task. Participants mined for gold, choosing between two mines in three territories with different hidden agents (benevolent millionaire, mean robber, and sneaky sheriff) influencing outcomes.

Existing research highlights age-related changes in reinforcement learning. Studies show developmental differences in the ability to learn from feedback, but often use simplified task designs ignoring external causal factors. Adult studies indicate that individuals incorporate their beliefs about the controllability of the environment when adjusting behavior to attain goals. This involves inferring the causal relationship between actions and outcomes, discounting uncontrollable outcomes when assigning credit to actions. The capacity to infer external causes emerges early in development. Toddlers can infer hidden causes and link them to deterministic and probabilistic events. Adolescents, compared to younger and older individuals, exhibit unique learning patterns concerning causal relationships, potentially due to developmental shifts in understanding and applying causal structure knowledge. Some evidence suggests children and adolescents rely on simpler action-outcome learning, not fully integrating complex environmental reward structures. Although they understand complex environments, their ability to use this understanding to guide performance might change during childhood and adolescence.

Ninety participants (ages 7-25; 30 children, 30 adolescents, 30 adults) participated. The task involved a modified reinforcement learning paradigm. Participants chose between two mines ('digging for gold') in three different territories, each with a hidden agent (millionaire, robber, sheriff) influencing outcomes. The agents intervened on 30% of trials, either adding gold (millionaire), replacing gold with rocks (robber), or randomly influencing outcomes (sheriff). Participants indicated whether they believed the hidden agent caused the outcome after each trial. Three simple reinforcement learning models (one, two, and three learning rate models) and four Bayesian reinforcement learning models (empirical Bayesian, empirical Bayesian by territory, adaptive Bayesian, and noisy Bayesian) were fit to the data. The Bayesian models incorporated participants' beliefs about agent intervention when updating value estimates. Model recovery analysis was conducted by simulating data from the models and determining the recoverability of each model. Logistic mixed-effects models were used to analyze the causal attribution data (trial-wise attributions to the hidden agent) and learning data (trial-wise optimal choices). Age was treated as a continuous variable.

Participants' beliefs about agent intervention aligned with the true probabilities, indicating understanding of the task structure. Younger participants attributed positive outcomes to external causes more often than older participants. Older participants learned to select the better mine faster across territories than younger participants. Younger participants demonstrated better learning in the 'millionaire' territory (where the agent only provided positive outcomes). Computational modeling revealed that adults were best fit by the empirical Bayesian model, meaning that their learning was significantly influenced by their beliefs about agent intervention. Adolescents were best fit by the adaptive Bayesian model, showing some flexibility in how they used causal information to guide learning. Children were best fit by the one learning rate model, suggesting they primarily relied on experienced outcomes without incorporating causal beliefs. Model recovery analyses showed that all three best-fitting models (one learning rate, adaptive Bayesian, empirical Bayesian) were recoverable. Simulations showed the one learning rate model mirroring children's better learning in the 'millionaire' condition, whereas Bayesian models reflected the similar learning trajectories across territories observed in adolescents and adults.

The study demonstrates age-related differences in using causal attributions to guide reinforcement learning. While all age groups understood the causal structure, older participants integrated this knowledge into their learning more effectively. Children primarily used simple action-outcome associations, whereas adolescents and adults incorporated beliefs about external causes. Adolescents displayed greater flexibility in learning than adults, potentially reflecting increased belief in self-control and a tendency to flexibly update beliefs about the environment. This flexibility might be advantageous during adolescence, when individuals face new choices in varied environments. The shift towards more complex learning strategies during adolescence may relate to developmental changes in brain regions supporting model-based learning. Older participants generally outperformed younger ones, potentially due to developmental changes in neural systems, although younger participants learned better when negative outcomes were more informative. The study's limitations include the heterogeneity of model fits within each age group, particularly among younger participants. Future research needs to investigate whether younger participants use causal information differently, perhaps incorporating prior beliefs about external causes.

This research highlights the developmental progression of integrating causal inference into reinforcement learning. While younger children demonstrate an understanding of causal relationships, older adolescents and adults effectively utilize this knowledge to guide their decisions. This developmental shift likely reflects maturation of brain regions associated with model-based learning. Further research should explore nuances in how children process causal information and investigate the specific neural mechanisms underlying this developmental trajectory.

The study showed some heterogeneity in the best-fitting models within age groups, especially among younger participants. While the group-level analysis indicated a preference for the one-learning rate model for children, individual variation might indicate more complex learning processes. Further, the task involved invisible and ambiguous agent interventions; future research using observable interventions could clarify the role of confidence in causal inferences in guiding learning. The study also focused on a specific type of reinforcement learning task; the findings may not generalize to other contexts.