Virtual reality-empowered deep-learning analysis of brain cells

Quantifying protein expression and neuronal activity across the whole brain at single-cell resolution is essential for understanding physiology and disease. While tissue clearing combined with fluorescent imaging enables whole-organ analysis, unbiased and consistent automated detection and atlas registration of cells in large three-dimensional datasets remains difficult due to varying image acquisition conditions, uneven signal-to-noise ratios and low target abundance. Conventional threshold-based pipelines require extensive manual tuning and often miss subtle signals, while deep-learning solutions can perform better but typically demand advanced coding expertise. The authors developed DELIVR, a virtual reality-aided deep-learning pipeline to segment c-Fos+ cells from cleared mouse brains and map them to the Allen Mouse Brain Atlas. They leveraged VR to accelerate and improve annotation of 3D light-sheet microscopy data for training, packaged the method in a Docker container with a Fiji front end to improve accessibility, and demonstrated adaptability to other cell types and application to dissect brain activity differences in cancer models.

Prior work established tissue clearing and light-sheet imaging as powerful tools for whole-organ protein expression mapping, and c-Fos immunolabeling as a proxy for brain-wide neuronal activity. Traditional automated analysis methods such as ClearMap and ClearMap2 rely on thresholding and filtering, which can underperform in datasets with variable signal-to-noise and heterogeneity. Other approaches include random forest-based tools like Ilastik and various deep-learning methods for whole-brain cell detection. However, many existing pipelines require coding skills and do not provide end-to-end, easily deployable solutions with atlas registration and visualization. VR-based annotation has been proposed to improve efficiency and accuracy for volumetric data compared to 2D slice tools (for example, ITK-SNAP), motivating its use to generate high-quality ground truth for training deep models in this context.

Sample preparation and imaging: Whole-brain c-Fos immunolabeling used a modified SHANEL protocol with sequential dehydration, delipidation, rehydration, antigen retrieval steps, blocking, primary (Cell Signaling 2250, 1:1,000) and secondary antibody labeling (Alexa Fluor 647 goat anti-rabbit, 1:500), extensive washes, and clearing via 3DISCO (THF series, DCM, BABB). Microglia in CX3CR1GFP/+ mice were labeled with Atto647N-conjugated anti-GFP nanobooster. c-Fos brains were imaged by LSFM (UltraMicroscope II; 4× objective; 640/40 nm excitation, 690/50 nm emission; 3×3 tiling, 15–20% overlap; 1.625 µm xy, 6 µm z steps; 16-bit; bilateral illumination). Microglia brains were imaged on Ultramicroscope Blaze (×12 objective, nominal pixel size 0.54 µm). Tiles were stitched in Fiji.

VR annotation: Annotators used Arivis VisionVR and syGlass with an Oculus Rift S headset and controllers to annotate 3D volumes; 2D slice-by-slice annotation was performed with ITK-SNAP for comparison. For c-Fos training/testing, 48 patches of size 100^3 voxels (5,889 cells) from a c-Fos-labeled brain were VR-annotated and expert-vetted. For microglia, 161 VR-annotated 100^3 voxel patches (3,798 somata) were prepared.

Pre-processing and ventricle masking: The pipeline downsamples image stacks to 25 µm isotropic, segments ventricles with an Ilastik 3D pixel classifier, upsamples masks, and applies masking to the raw image stack to suppress false detections in ventricles.

Deep learning: Multiple 3D architectures were evaluated (3D BasicUNet, UNETR, SegResNet, MONAI DynUNet) using MONAI and PyTorch. DELIVR was trained with a 3D BasicUNet using Mish activation and the Ranger optimizer; binary cross-entropy loss; 500 epochs; initial LR 1e-3; batch size 4; trained on NVIDIA RTX8000. Split: 39 training and 9 test patches for c-Fos. Microglia model used similar settings, trained for 500 epochs on an NVIDIA A100 (129 train, 32 test patches).

Inference and post-processing: A customized sliding-window inference produces binary segmentations. Connected component analysis identifies individual cells, computes center coordinates and volumes, and filters by size. Cells are mapped to Allen Brain Atlas CCF3 regions after atlas registration.

Atlas registration and visualization: Brains (25 µm isotropic) were saved as .v3draw and registered to CCF3 (50 µm) with mBrainAligner using LSFM settings. Cell center points were transformed to atlas space. Custom scripts generated per-cell and per-region tables (counts, region IDs, Atlas color codes). Visualization used BrainRender for atlas-space plots and color-coded TIFF overlays in image space; cortical flat-map visualizations were generated from Allen code with adaptations.

Software packaging: The end-to-end inference pipeline (mBrainAligner, Ilastik, TeraStitcher, MONAI, PyTorch, SciPy stack, cc3d) was packaged in a Docker container (CUDA 11.7.2 runtime) and exposed via a Fiji (v1.52p) Java plugin that compiles config files and runs the container. A separate training Docker integrates with Fiji to fine-tune or train new models (config_train.json, progress reporting) for other cell types/datasets.

Cancer model application and statistics: Adult male BALB/c mice received subcutaneous injections of PBS (control), NC26 (weight-stable cancer) or C26 (cachexia-inducing) colon cancer cells. Body weight, tumor weight, tissue weights and brain weights were measured; brains underwent c-Fos labeling, clearing, LSFM, and DELIVR analysis. Statistics used GraphPad Prism: normality by Shapiro–Wilk; two-group comparisons by t-test or Mann–Whitney U; three groups by one-way ANOVA with Sidak or Kruskal–Wallis with Dunn’s. For region-wise c-Fos density, two-sided t-tests with Benjamini–Hochberg correction at FWER = 0.1 were applied.

VR vs 2D annotation: For a 100^3 voxel sub-volume (83 c-Fos+ cells), VR annotation was significantly faster than 2D slice-based annotation (P = 0.0005, two-sided Mann–Whitney U; n=7 for 2D; n=12 for VR with n=6 Arivis and n=6 syGlass) and improved instance Dice (F1) from 0.7383 to 0.8032 (P = 0.0445, two-sided unpaired t-test).

Model selection: Among tested architectures, 3D BasicUNet yielded the best instance-level F1 and was used in DELIVR.

Performance vs non-deep-learning baselines: On the c-Fos test set, DELIVR achieved F1 = 0.7918, instance sensitivity = 0.8470, instance precision = 0.7434, volumetric Dice = 0.6739. Relative to ClearMap2 (second best), these represent +89.03% (F1), +181.64% (sensitivity), +7.74% (precision) and +581.39% (volumetric Dice) improvements. In absolute detections, DELIVR identified 1,611 true positives versus 19 (ClearMap), 572 (ClearMap2), and 1,228 (optimized ClearMap), without over-segmentation.

Adaptation to microglia: A microglia somata model trained on 161 VR-annotated patches (3,798 cells) achieved F1 (Dice) = 0.92 and enabled mapping of 12.2 million microglia across one hemisphere, color-coded by CCF3 regions.

Cancer application: In vivo NC26 (weight-stable) and C26 (cachectic) tumor models were established (n(PBS)=12, n(NC26)=8, n(C26)=12). C26 mice lost significant body weight (P < 0.0001), not explained by tumor mass differences; they showed reduced gastrocnemius muscle and adipose tissue weights and a modest brain weight decrease (P = 0.0479). DELIVR-derived c-Fos density maps revealed increased brain activity in NC26 vs PBS controls, an increase absent in C26 mice. Nineteen brain areas showed significantly increased c-Fos in NC26 vs PBS after multiple-testing correction (Padj < 0.1; two-sided t-tests, BH correction, FWER 0.1). Seven areas differed between NC26 and C26, with no significant differences between PBS and C26 after correction. Hyperactivation in NC26 concentrated in cortical plate (notably somatosensory snout regions and motor areas) and retrosplenial/entorhinal regions, consistent with a foraging-related activation pattern.

DELIVR combines VR-enabled annotation with a 3D deep-learning model and automated atlas registration to deliver accurate, scalable whole-brain cell detection without requiring coding expertise. VR substantially accelerates and improves the quality of ground-truth labeling for volumetric data compared to 2D slice-based methods, enhancing downstream model performance. Against threshold and filter-based pipelines (ClearMap/ClearMap2) and alternative ML approaches, DELIVR’s 3D BasicUNet achieved markedly superior instance detection and volumetric overlap, especially in challenging datasets with variable signal-to-noise ratios. The accessible Docker + Fiji packaging places end-to-end processing, visualization, and statistics within reach of non-expert users, and a training Docker enables adaptation to new markers and cell types, as demonstrated for microglia. Applying DELIVR to cancer models uncovered a distinctive neuronal hyperactivation phenotype in weight-stable NC26 tumor-bearing mice—particularly in somatosensory and motor cortices and retrosplenial cortex—absent in cachectic C26 mice. These findings align with regions implicated in sensorimotor control and foraging and suggest neurophysiological alterations associated with cancer that may relate to weight maintenance. Thus, DELIVR both advances the methodological state of the art for 3D cell detection and provides biological insight into cancer-related brain activity patterns.

The study introduces DELIVR, a VR-empowered deep-learning pipeline that accurately detects and atlas-maps cells in whole-brain cleared LSFM datasets via a user-friendly Docker/Fiji interface. VR-based annotation accelerates training data creation and enhances quality, enabling DELIVR to outperform state-of-the-art threshold-based and alternative ML methods for c-Fos+ cell detection. The framework generalizes to other cell types, as shown for microglia, and yields comprehensive, atlas-registered visualizations and region-level quantifications. Biologically, DELIVR revealed a cancer-specific hyperactivation signature in weight-stable NC26 mice localized to cortical and retrosplenial regions. Future work may integrate active learning within VR to further reduce annotation burden, broaden cell-type generalizability, and explore mechanistic links between altered brain activity and systemic phenotypes in cancer.

Training data for the c-Fos model were derived from a limited number of annotated patches from a single brain, which may constrain generalizability across diverse imaging conditions and protocols; users may need dataset-specific fine-tuning. The pipeline’s accuracy depends on the quality of VR annotations and atlas registration. While DELIVR is packaged for accessibility, it requires VR hardware for annotation and substantial compute resources for training and inference on large whole-brain datasets. In the biological application, although region-wise differences in c-Fos density were robust after multiple-testing correction, interpretation is limited by the observational design and by unresolved questions such as the cellular basis of reduced brain weight in cachectic mice. Very large raw imaging data volumes also limit data sharing and may affect reproducibility without access to full datasets.