Neural systems have evolved not only to solve environmental challenges through internal representations but also to communicate these representations to conspecifics under social constraints. This research aims to understand the structure of these internal representations and how they are optimized for effective information transmission between individuals. Previous research has emphasized self-experience and common circuitry priors in task representation, but shared neural representations are crucial for communication and the development of cognition. The authors hypothesize that social communication facilitates task-efficient representations, enabling generalization of experiences among cooperative agents. This builds upon prior work focusing on the conditions for artificial language evolution and multi-agent systems with communication policies, but extends this by examining the nature of shared representations. This study utilizes a teacher-student framework, leveraging reinforcement learning (RL) to generate empirical task abstractions that vary among agents. This RL framework recapitulates critical features of language, including interchangeability, total feedback, and productivity. Unlike prior research, this study focuses on how hidden representations are shared and the effect of the lower-dimensional language space structure. The researchers analyze how agents internally abstract real-world variables, how these abstractions translate into a shared language, and how these elements interact. A non-discrete language model is used to compare the continuous nature of brain processes and real-world phenomena, allowing individualized abstractions to emerge organically rather than being pre-defined. By analyzing the language embedding structure, insights into information content and its relation to neural representations are gained.

Existing research explores task representations in biological and artificial agents, focusing on self-experience and common circuitry priors. Shared neural representations underpin similar behaviors among conspecifics, with common convergent abstractions essential for inter-individual communication. Social pressure suggests neural circuits may have evolved to optimize internal representations for communication efficacy. Early studies focused on the conditions for artificial language evolution and its similarity to human communication. The advent of deep learning has spurred research combining multi-agent systems with communication policies, encompassing multi-agent games with message exchange, translation tasks, and low-level policy formation. However, these studies largely concentrate on performance consequences rather than the nature of shared representations. The current work builds upon a teacher-student framework to develop a communication protocol for cooperative task-solving, employing reinforcement learning to produce empirical task abstractions. In contrast to previous work, this research emphasizes understanding how hidden representations are shared and the impact of the structure of the low-dimensional language space.

To study language emergence between agents, a teacher-student framework was implemented using deep neural networks. The teacher network is trained using reinforcement learning (RL), outputting a state-action value function (Q-matrix), while the student solves the same task using observations and a message from the teacher. A sparse autoencoder (SAE) acts as a communication process, compressing the teacher's high-dimensional Q-matrix into a low-dimensional message. The SAE consists of an encoder and decoder, promoting sparsity in the lower-dimensional representations (mimicking brain sparsity). The L1-norm of the message vector is added to the autoencoder loss to promote sparsity. Three communication protocols were employed: (1) sequential training of language and student networks (no feedback); (2) language and student connected with feedback from student performance; and (3) a 'closing-the-loop' architecture, where the student generates messages after task completion. The navigational tasks are performed in a grid-world maze, where the teacher learns a single task and passes a message to the student. The chosen task is relatively simple, allowing focus on message structure and generalization analysis. The agents don't use a predetermined vocabulary; language emerges naturally from the task and the low-dimensional encoding. The study analyzes the structure of the low-dimensional representations and the performance of the student in interpreting messages from the embedding space. The teacher agents were trained to solve individual maze tasks, and the SAE embeds their Q-matrices into a low-dimensional space. Topographic similarity analysis compared distances in the message space to task labels and teacher Q-matrices to assess compositionality. Information-theoretic analysis using Shannon entropy measured the information-carrying capacity of messages, considering discretization of the continuous embedding space. The student's performance and generalization capabilities were evaluated by providing messages from teachers mastering mazes with zero or one wall state, training the student on patterned subsets of goal locations, and testing on unknown goals. A 'closing-the-loop' experiment involved the student generating messages based on learned information and assessing performance with self-generated messages. Analysis of variance (ANOVA) examined the structure of message spaces using variance within and between groups, and statistical tests (t-tests) compared student performance across different conditions. Specific details regarding network architectures, training parameters, reward structures, and statistical analysis are provided in the Methods section.

The study revealed several key findings: 1. **Structure of the lower-dimensional message space:** Without student feedback, the latent space prioritized wall positions, followed by goal locations, reflecting a hierarchical structure. With feedback, this structure shifted, indicating a more task-relevant representation (Tables 1 and 2). 2. **Topographic similarity:** Languages trained with feedback demonstrated greater topographic similarity (compositionality) between message and meaning spaces, indicated by positive slopes in linear regressions (Figure 3). 3. **Entropy analysis:** Entropy decreased moving through the communication framework (teacher, message, student), with feedback retaining more information, highlighting the importance of bi-directionality (Figure 3). Removing the reconstruction loss from autoencoder training significantly decreased entropy, emphasizing the pressure for language simplicity (Figure 3). 4. **Student performance and generalization:** Students trained with feedback outperformed misinformed students and random walkers on trained goals (Figure 4). However, generalization to unknown goals was challenging unless the training covered a substantial portion of the task space (checkerboard pattern) (Figure 4). Students trained with feedback also generalized better to unknown scenarios compared to those trained without feedback (Supplementary Figures S3 and S4). 5. **Closing the loop:** When students generated their own messages, information content degraded. Variability concentrated on goal location and initial action, leading to lower task solve rates. While performance decreased on unknown goals, informed students still outperformed misinformed ones (Figure 5), indicating that some task-relevant information remained in the degraded messages.

The findings highlight the interplay between environmental experiences and internal abstractions in shaping effective communication. The study demonstrated that embedding task solutions into a low-dimensional space creates effective abstractions, enabling agents to flexibly learn goals and states. The introduction of student feedback significantly improved the structure of the message space, making it more relevant for task completion and generalisation. This mirrors the concept of 'total feedback' in human communication, where speakers adapt their message based on environmental factors and the presence of others. The 'closing-the-loop' experiment provides valuable insights into how information degrades during communication, similar to the children's game 'telephone.' Even in degraded form, the messages still retained essential information, enabling better performance than uninformed approaches. The results suggest that the balance between teaching and learning is critical in multi-task and multi-agent systems, ensuring a relevant and generalizable message space. The study also points towards analogies with natural languages, where utility drives language evolution. Dimensionality and sparsity constraints in the model reflect biological limitations, resulting in hierarchical, task-relevant structures. Future work should examine the effects of channel size on representation, integrate discrete communication protocols, and investigate sequential composition to further understand the emergence and evolution of language.

This research introduces a multidisciplinary framework for studying language emergence using reinforcement learning agents and encoding networks. The findings highlight the importance of bi-directional feedback in shaping task-relevant representations and the challenges of generalization in communication. The model offers valuable insights into the interplay between individual and collective behavior and the emergence of social communication. Future work could explore sequential composition, different channel structures, and more model-agnostic approaches to enhance our understanding of emergent communication and its relationship to biological systems.

The study employed a simplified grid-world environment. The generalizability of the findings to more complex environments and tasks remains to be explored. The chosen communication mechanism using a sparse autoencoder may not fully capture all aspects of natural language processing. Furthermore, the language used lacked predefined syntax or grammar; future research should incorporate these elements. Finally, while the study analyzed compositionality, it did not specifically investigate sequential composition.