SLEAP: A deep learning system for multi-animal pose tracking

The study of brain-generated behavior necessitates advanced tools for quantifying natural animal behavior. While deep learning has enabled markerless pose estimation in individual animals, multi-animal tracking presents challenges. This paper introduces SLEAP (Social LEAP Estimates Animal Poses), a machine learning system for multi-animal pose tracking. SLEAP offers versatile workflows for data labeling, model training, and inference, featuring a user-friendly interface, standardized data model, reproducible configuration, various model architectures, and approaches to part grouping and identity tracking. Evaluated across diverse datasets (flies, bees, mice, gerbils), SLEAP demonstrates high accuracy and speed (over 800 frames per second), enabling real-time applications, as demonstrated by controlling an animal's behavior based on real-time social interaction tracking. SLEAP is open-source and available at https://sleap.ai.

Talmo D. Pereira, Nathaniel Tabris, Arie Matsliah, David M. Turner, Junyu Li, Shruthi Ravindranath, Eleni S. Papadoyannis, Edna Normand, David S. Deutsch, Z. Yan Wang, Grace C. McKenzie-Smith, Catalin C. Mitelut, Marielisa Diez Castro, John D’Uva, Mikhail Kislin, Dan H. Sanes, Sarah D. Kocher, Samuel S.-H. Wang, Annegret L. Falkner, Joshua W. Shaevitz, Mala Murthy