SLEAP: A deep learning system for multi-animal pose tracking

Studies of social behavior and naturalistic settings require accurate markerless pose estimation for multiple interacting animals. SLEAP (Social LEAP Estimates Animal Poses) is a general-purpose machine learning system for multi-animal pose tracking that provides an end-to-end workflow for labeling, training, inference, and proofreading via a GUI and APIs. It offers standardized data formats, reproducible configuration, 30+ network backbones, both bottom-up and top-down multi-instance strategies, and two identity-tracking approaches. Across seven datasets (flies, bees, mice, gerbils), SLEAP attains high accuracy and very high speed (>800 FPS with <3.5 ms latency at 1,024×1,024 resolution), enabling real-time applications. The system’s real-time capability is demonstrated by closed-loop optogenetic control based on online detection of social interactions.