Understanding the neural mechanisms underlying natural animal behavior is a fundamental goal in neuroscience. Current research often focuses on simplified, overtrained behaviors due to their ease of study. However, this approach neglects the complexity of natural behavior and its intricate transition dynamics. Comprehensive behavioral tracking requires accurate identification and quantification, including kinematics and transitions at meaningful timescales. Traditional top-down methods, often involving laborious human rating or supervised machine learning, suffer from observer bias, inter-rater variability, and limited temporal resolution. Unsupervised learning algorithms, like MotionMapper, offer an alternative by using non-linear dimensionality reduction to identify stereotyped behaviors. However, these methods have limitations, particularly in vertebrate studies due to their reliance on specific movement patterns and uniform backgrounds. MoSeq, while an advancement, lacks the ability to capture both action and kinematics with sufficient temporal resolution and generalizability across different settings. This paper introduces B-SOID, a new platform that addresses these limitations by extracting the spatiotemporal patterns of identified body poses to create a robust and generalizable system for behavior identification and classification.

Existing methods for analyzing animal behavior fall into two main categories: top-down and bottom-up approaches. Top-down methods rely on pre-defined behavioral categories and often involve manual annotation or supervised machine learning, which are prone to biases and inter-rater variability. Bottom-up methods use unsupervised learning to discover behavioral patterns from raw data, such as MotionMapper, which uses non-linear dimensionality reduction to map behavioral space. However, MotionMapper has limitations, particularly regarding temporal resolution and generalizability across different experimental setups and organisms. MoSeq represents another approach but still faces challenges in providing comprehensive kinematic analysis and high temporal resolution. Recent advances in computer vision have enabled automated tracking of body parts, but translating this information into meaningful behavioral categories remains a significant hurdle.

B-SOID uses pose estimation software (e.g., DeepLabCut) to identify body part locations from video. It then extracts spatiotemporal relationships between these positions (speed, angular change, distance) and uses UMAP, a non-linear dimensionality reduction technique, to embed these high-dimensional measurements into a lower-dimensional space. HDBSCAN, a hierarchical clustering method, identifies dense clusters in this space, representing distinct behaviors. However, instead of relying solely on clustering, B-SOID trains a random forest classifier on the high-dimensional pose relationships to predict behavioral categories. This approach significantly speeds up processing and enables generalization across different datasets, animals, cameras, and experimental setups. To address the issue of low temporal resolution, B-SOID incorporates a frameshift paradigm. The algorithm downsamples the video to 10 fps to improve signal-to-noise, then repeatedly analyzes the data with shifted time windows to achieve high temporal resolution without sacrificing signal quality. B-SOID provides a user-friendly graphical user interface (GUI) for data processing, model adjustment, and performance quantification. The experiments involved mice in open field arenas, rats in a reach-to-grasp task, and humans performing kinesiological movements. Electrophysiological recordings were also conducted in mice, and lesion studies were done to assess the impact of specific neuronal pathways on behavior. Comparison of B-SOID's performance was done against MotionMapper, using motion energy image mean squared error (MSE) to quantify the quality of behavioral groupings.

B-SOID accurately identified and classified behaviors across various species and experimental setups, achieving high accuracy and generalizability. The frameshift paradigm significantly improved the temporal resolution of behavioral segmentation, enabling millisecond-level precision. This improved resolution allowed for a more precise alignment of neural activity with behavioral events, revealing distinct neural signatures associated with different behaviors. B-SOID also demonstrated robust performance across different camera angles (top-down vs. bottom-up), indicating adaptability to diverse experimental conditions. Comparison with MotionMapper showed that B-SOID produced more distinct behavioral clusters and achieved a much faster processing speed. The lesion study revealed subtle kinematic changes in grooming behaviors following cell-type specific lesions of the basal ganglia indirect pathway, highlighting B-SOID's sensitivity in detecting fine motor differences. These kinematic changes were not detectable with existing methods.

B-SOID addresses significant limitations of current behavioral analysis methods. Its unsupervised nature eliminates user bias, while its high temporal resolution and generalizability make it applicable across a wide range of experiments. The integration of a machine learning classifier significantly improves processing speed and robustness, making it a practical tool for researchers. The ability to detect subtle kinematic changes in grooming, itching, and locomotion provides a powerful new approach for studying motor control and neurological disorders. The improved temporal resolution allows for more precise correlation of neural activity with behavior, paving the way for deeper insights into the neurobiological mechanisms underlying natural behaviors.

B-SOID provides a powerful, open-source, and user-friendly platform for unsupervised behavioral analysis. Its high accuracy, speed, generalizability, and high temporal resolution make it a valuable tool for researchers studying animal behavior across various species and experimental conditions. Future work could explore the integration of multimodal data (e.g., acoustics, environmental stimuli) and real-time behavioral feedback systems.

While B-SOID addresses many limitations of existing methods, some limitations remain. The accuracy of the behavioral classifications depends on the quality of the pose estimation, which can be affected by factors such as occlusion or poor video quality. The frameshift paradigm, while improving temporal resolution, might introduce some complexity in interpretation. The generalizability of the algorithm across vastly different species or drastically different movement patterns has not yet been fully tested.