Artificial intelligence (AI) has achieved remarkable progress in various domains, including video games, board games, protein folding, and language modeling. However, a fundamental gap remains: state-of-the-art AI systems struggle with 'common sense' knowledge crucial for prediction, inference, and action in everyday scenarios. This research focuses on intuitive physics—the understanding of macroscopic object properties and interactions—a core component of embodied intelligence and conceptual knowledge. Even young children demonstrate a superior grasp of intuitive physics compared to existing AI systems. To bridge this gap, the study draws heavily on developmental psychology, a field that extensively examines how children acquire intuitive physics knowledge. The research hypothesizes that object-centric representations and processing, key insights from developmental psychology, are crucial for AI systems to develop richer intuitive physics understanding. This contrasts with typical AI approaches that rely on metrics like video or state prediction, binary outcome prediction, question-answering performance, or reinforcement learning rewards, which don't explicitly operationalize or probe a specific set of intuitive physics concepts.

Developmental psychology research on intuitive physics acquisition highlights two key principles. First, intuitive physics relies on discrete concepts (object permanence, solidity, continuity, etc.) that can be individually assessed. This contrasts with existing AI approaches that lack this focused, concept-specific evaluation. Second, possessing a physical concept involves forming expectations about future events. The violation-of-expectation (VoE) paradigm measures this by presenting infants with possible and impossible events. Longer gaze duration at impossible events indicates surprise and thus, concept understanding. This paradigm has provided substantial evidence of infants' early acquisition of various physical concepts within their first year of life. Prior work introduced a machine-learning video dataset to test model learning of specific physical concepts. This current work expands upon that by introducing a significantly richer video corpus, the Physical Concepts dataset, targeting five central concepts from developmental psychology: continuity, object persistence, solidity, unchangeableness, and directional inertia. Each concept has matched possible and impossible video probes, carefully controlled to isolate the concept being tested.

The Physical Concepts dataset includes two corpora: a training set of procedurally generated videos depicting diverse physical events (objects rolling, colliding, occluding, stacking, etc.), and a test set of VoE probes. The training videos feature composable interactions, random object properties (shapes, colors, masses), and a randomly drifting camera to introduce complexity and visual diversity. A deep-learning model, PLATO (Physics Learning through Auto-encoding and Tracking Objects), is designed based on developmental psychology principles. PLATO comprises a perception module and a dynamics predictor. The perception module, a ComponentVAE, uses object segmentation masks to generate object codes representing each object's visual features. The dynamics predictor, an InteractionLSTM, is a recurrent neural network with object-specific memory and an interaction network that computes relationships between objects. PLATO learns to predict the next frame's object codes given past object codes and their interactions. To assess the importance of object-centric representation, two control models, flat equal parameters (FEP) and flat equal capacity (FEC), are developed. These models lack object-level representations, instead using a single vector embedding for the entire scene. Both PLATO and the baseline models undergo a two-phase training process: first, training the perception module to reconstruct images; second, training the dynamics predictor to predict object codes in sequences. The VoE paradigm is used to evaluate all models. Surprise is measured as the sum of squared prediction errors across frames. Each model is tested on 5,000 probe tuples for each concept. A probe is correctly classified if the impossible event generates higher surprise than the possible event. Relative surprise and accuracy scores are used to quantify performance. The splicing method for generating probes ensures that possible and impossible probes are matched in terms of individual frames and pairs of frames; the only difference is the order of frames, producing aphysical events in the impossible probes. Furthermore, all adjacent frames are physically possible, ensuring the model's surprise isn't based only on temporally local inconsistencies.

PLATO demonstrated strong VoE effects across all five concept categories, significantly outperforming the object-agnostic baseline models. Even the FEC model, with significantly more parameters than PLATO, exhibited only above-chance accuracy on two of five concepts. This highlights the crucial role of object-centric representation in learning intuitive physics. PLATO achieved robust VoE effects with surprisingly limited training data, equivalent to just 28 hours of visual experience (50,000 training examples), suggesting high data efficiency. Furthermore, PLATO generalized well to unseen objects and dynamics from the independent ADEPT dataset, demonstrating robust VoE effects without retraining. The statistical significance of PLATO's superior performance over baseline models is established through one-tailed single-sample t-tests on relative surprise and accuracy, consistently showing p-values below 0.05 for all five concepts in the primary experiments and the three tested concepts in the generalization experiment. Frame-by-frame analysis revealed that relative surprise increased significantly at the onset of physically impossible events in the PLATO model, and the trajectory of this increase was specific to the physical phenomena involved. This consistency across multiple random seeds and concepts provides strong support for the effectiveness of the object-centric approach in learning intuitive physics.

The results strongly support the hypothesis that object-centric representations are critical for acquiring intuitive physics concepts. The findings align with developmental psychology's emphasis on the role of object individuation, tracking, and relational processing in infant physics understanding. The significant performance advantage of PLATO, even when compared to a model with substantially higher parameter count, showcases the importance of architectural design inspired by developmental principles. The model's ability to achieve strong VoE effects with a relatively small amount of training data indicates potential improvements in data efficiency for deep learning models, addressing a common criticism of the field. The successful generalization to the ADEPT dataset highlights the robustness and transferability of the learned representations. This contrasts with prior work, which either achieved success with a small number of concepts, or relied on hand-engineered physics engines rather than true learning. While PLATO uses ground truth object segmentations and tracking, ongoing research in unsupervised object discovery and tracking aims to remove this privileged information and create a truly ‘learning-from-scratch’ system. Nevertheless, control experiments show the importance of not only object representation but also dynamic prediction in learning intuitive physics. The object-level representation may act as a regularizer, biasing the learning system toward compositional representations that generalize better to new situations.

This study demonstrates that a deep learning model, PLATO, inspired by developmental psychology, can effectively learn intuitive physics concepts from visual data. Object-centric representation is shown to be crucial for this learning. The model's data efficiency and successful generalization suggest potential avenues for enhancing deep learning approaches. Future work should explore removing the dependence on ground truth object segmentation and tracking, investigating the developmental trajectory of concept acquisition using similar models, and applying similar relational architectures to other core domains in developmental psychology.

The study uses simplified simulated environments and objects, limiting the ecological validity of the findings. The use of ground truth object segmentation and tracking represents a limitation; future work should address learning these features from raw video data. While the five concepts are well-established in developmental psychology, they do not represent the full spectrum of intuitive physics knowledge.