Building machines that learn and think with people

This Perspective addresses the question of how to build AI systems that act as effective partners in human thought, not merely tools. The authors situate the work in the context of recent advances in foundation models (especially large language models) that enable natural language interaction but often lack robust, explicit human-like reasoning about the world and other minds. They argue that effective thought partnership requires systems that: understand human goals, beliefs, and resource limitations; act in ways humans can understand; and share accurate, grounded models of the world. The purpose is to chart a human-centered, cognitively grounded engineering path for such systems, emphasizing the importance of collaborative cognition and the sociotechnical ecosystems that support it.

The paper synthesizes decades of work from behavioral and computational cognitive science relevant to collaborative cognition. It reviews limitations of scaling foundation models with human demonstrations and web-scale traces, noting their strengths in mimicking behavior but weaknesses in simulating cognition. It surveys motifs including probabilistic models of cognition, structured knowledge representations, hierarchical and program-based theories of mind, resource-rationality, goal-directed planning and search, Bayesian theory of mind, rational speech acts, and meta-learning. It discusses probabilistic programming languages as infrastructure for expressing and performing inference on generative world models, and touches on design and regulatory considerations for the ecosystems around AI thought partners (for example, EU AI Act requirements).

As a Perspective, the work proposes an engineering framework rather than conducting empirical experiments. The methodology centers on explicitly instantiating three desiderata—You understand me, I understand you, We understand the world—using computational cognitive science and Bayesian formalisms. The approach models humans as cooperative agents with structured beliefs, goals, knowledge, and bounded resources, and models human–machine communication via pragmatic reasoning (rational speech acts) and theory of mind. To scale and maintain these models, the authors advocate probabilistic programming for modular, transparent generative modeling and programmable inference, integrating LLMs as priors or evaluators where appropriate. They emphasize building sociotechnical infrastructure around thought partners to manage when and how humans engage with them, including routing, deferral, and affordances that mitigate over-reliance and support correct interpretation.

Key contributions are conceptual and design-oriented: (1) Articulation of three desiderata for human-centered thought partners—You understand me (accurate modeling of human goals, beliefs, and resource limits), I understand you (legible, predictable, and communicative behavior), and We understand the world (shared, grounded, uncertainty-aware models of reality). (2) A Bayesian thought partner toolkit (probabilistic generative modeling, structured representations, hierarchical modeling, program-based theory learning, resource-rational planning and inference, theory of mind, rational speech acts, and meta-learning). (3) Identification of modes of collaborative thought (planning, learning, deliberation, sense-making, creation/ideation) and exemplar domains (programming, embodied assistance, storytelling, medicine). (4) Case studies that demonstrate feasibility: WatChat for explaining program behavior via Bayesian mental-model debugging; CLIPS for cooperative instruction-following and goal assistance in embodied tasks; inverse inverse planning for shaping audience inferences in storytelling; and early probabilistic programming applications in medical generative modeling and reliability-aware routing. (5) Guidance on infrastructure, evaluation (interactive, process-sensitive assessments; games as testbeds), and risk management (reliance and access, anthropomorphization, misalignment and dual-use).

The proposed framework directly addresses the challenge of building AI systems that can be thought partners by aligning system design with how humans reason and collaborate. Explicit probabilistic modeling enables systems to infer human goals and misconceptions, communicate legibly, and reason over grounded world models, fostering shared understanding and synergy. Combining Bayesian methods with LLMs leverages strengths of both structured inference and fluent language, while probabilistic programming offers scalable, transparent, and editable models. The discussion extends to sociotechnical infrastructure—routing, deferral, and affordances—that shapes appropriate reliance and effective collaboration. The authors highlight the importance of interactive evaluation and consider expansion beyond dyads to multi-agent ecosystems, where conventions and group coordination can be modeled in Bayesian terms. The significance lies in moving beyond scale-first approaches toward cognitively informed design that is uncertainty-aware, human-compatible, and amenable to principled evaluation and risk mitigation.

The paper advocates a human-centered, cognitively grounded path to building AI systems that truly learn and think with people. It argues for engineering explicit models of humans and the world, guided by Bayesian motifs and implemented with probabilistic programming, to realize thought partners that are understandable, trustworthy, and synergistic. Through case studies and ecosystem-level considerations, it outlines how such partners can support collaborative cognition across domains. The authors call for continued interdisciplinary collaboration among behavioral scientists, AI practitioners, and domain experts, and point to future work on non-dyadic settings, interactive evaluation methodologies, and infrastructure that promotes appropriate reliance and equitable access.

The work is a conceptual Perspective rather than an empirical study; it does not provide quantitative performance evaluations or formal benchmarks demonstrating superiority over current systems. Some cognitive science theories referenced are debated within the field, and models of human cognition may be incomplete or brittle in real-world deployment. The proposed approach depends on accurate, robust, and safe modeling of humans, which carries dual-use risks (for example, manipulation or surveillance) and potential misalignment if models are incorrect. Practical challenges include scaling structured probabilistic models, integrating them with LLMs, ensuring transparency and legibility, and designing sociotechnical infrastructure to manage over- and under-reliance, access inequities, and regulatory compliance.