Self-supervised machine learning for live cell imagery segmentation

The study addresses the challenge of robust, reproducible, and accessible cell segmentation in time-lapse live-cell microscopy. Traditional supervised learning approaches, particularly deep neural networks, depend on large curated label sets and extensive pre-processing, which introduces labor, subjectivity, and potential bias, and often generalize poorly to diverse cell types, imaging modalities, and experimental setups. The authors propose leveraging a ubiquitous property of live-cell time-lapse data—cellular motion—to remove the need for manual labels and parameter tuning. They hypothesize that optical flow-derived motion between consecutive frames can self-label pixels as cell or background, enabling a self-training pipeline that adapts to the user’s own data across modalities and conditions, thereby improving accessibility, reproducibility, and performance relative to generalist supervised models.

The paper reviews recent efforts in supervised ML for bioimage analysis, highlighting the dominance of deep learning methods (e.g., U-Net architectures) that require extensive labeled datasets such as COCO, EVICAN, CellPose, and LIVEcell. Despite these efforts, supervised models often fail to generalize beyond training distributions due to the diversity of cell types, imaging modalities, and experimental conditions, leading to frequent retraining. The authors also discuss biases introduced during human annotation and pre-processing, transparency issues due to opaque model weights and overfitting risks, and community concerns around reproducibility and best practices in ML for life sciences. Self-supervised learning is posited as an alternative that can exploit inherent data structure for labeling, with motion in time-lapse imagery as a robust signal independent of modality or cell type.

Core idea: Use optical flow (OF), specifically multi-resolution Farneback Displacement (FD), between consecutive frames (t−1, t) to self-label pixels for training a cell/background classifier, then apply the trained classifier to segment all pixels at time t. The pipeline runs recursively for each frame pair, continuously adapting to phenotype and background changes. Steps: 1) Dynamic self-labeling with FD: - Compute FD vector field between frames t−1 and t. - Use FD magnitude to auto-label pixels: highest displacements → 'cell'; lowest displacements → 'background'; intermediate remain unlabeled. - A liberal low FD threshold is used for background labeling; for cell labeling a high FD threshold selects high-confidence cell pixels. - For very low-contrast data, iteratively reduce the FD threshold for 'cell' until the entropy feature distribution of cell pixels is separable from that of background. 2) Static feature extraction for supervised classification: - From the single image at time t, compute static feature vectors (on local neighborhoods) for the self-labeled pixels: image gradient and entropy (modular to allow more features). 3) Classifier training and application: - Train a Naïve Bayes classifier on the self-labeled pixels using the static features. - Apply the classifier pixel-wise to the entire image at time t to produce a binary semantic segmentation (cell vs background). - Repeat the self-label, train, and segment cycle for each subsequent frame pair to adapt over time. Automations and post-processing: - Size filtering: A fast standalone FD pass (without self-tuning/model building) estimates average cell size and excludes much smaller objects (debris) automatically, leveraging that debris typically lacks motion. - Hole filling: Because FD detects widespread intracellular motion, a fixed morphological blur (circular structuring element, radius 5 pixels) robustly fills holes (e.g., unlabeled nuclei in fluorescence or lamellipodia in phase contrast). Cell area was invariant across a range of radii tested. - Optional FD visualization: FD vectors can be plotted to quantify intracellular flow dynamics. Evaluation: - Data: Multiple datasets spanning modalities (phase contrast, transmitted light, DIC, IRM, fluorescence), magnifications (10×–100×), and cell types (Hs27 fibroblasts, Dictyostelium, MDA-MB-231, A549). Ground truths were manual segmentations per dataset. - Comparator: CellPose generalist supervised model (trained on ~70,000 manually annotated objects) with automatic or tuned size parameter as needed. - Metric: Pixel-wise F1-score using TP, FP, FN relative to ground truth. Computational aspects: - Implemented and validated on laptops; ~7 s to process a pair of 1216×1920 8-bit images. - Classifier selection: Naïve Bayes chosen over alternatives (random forest, SVM, kNN) for robustness and favorable bias-variance properties under feature independence assumption. Additional experimental details: - Cell culture and imaging conditions provided (media, substrates, microscopes, time increments), with modality-specific parameters listed in Supplementary materials. - Reproducibility: The SSL method is inherently blinded and reproducible since it does not depend on curated datasets or user-tuned parameters.

- Across six validation datasets, the SSL approach achieved robust segmentation with F1-scores ~0.67–0.964 and outperformed the CellPose generalist supervised model in four datasets; in two high-magnification single-cell datasets, performance was statistically equivalent. Per-dataset results (F1, number of objects #O; CellPose trained on ~70k labels, SSL on 0 labels): 1) 10× phase / Hs27 fibroblasts (multi-cell): - CellPose: F1 = 0.480 (#O = 23) - SSL: F1 = 0.670 (#O = 23) 2) 10× transmitted light / Dictyostelium (multi-cell): - CellPose: F1 = 0.681 (#O = 63) - SSL: F1 = 0.711 (#O = 63) 3) 10× phase / MDA-MB-231 (multi-cell): - CellPose: F1 = 0.201 (#O = 7) - SSL: F1 = 0.759 (#O = 7) 4) 20× DIC / MDA-MB-231 (multi-cell): - CellPose: F1 = 0.682 (#O = 32) - SSL: F1 = 0.886 (#O = 32) 5) 40× IRM / Hs27 (single-cell): - CellPose: F1 = 0.959 (#O = 1) - SSL: F1 = 0.964 (#O = 1) 6) 100× fluorescence (GFP-actin) / A549 (single-cell): - CellPose: F1 = 0.963 (#O = 1) - SSL: F1 = 0.957 (#O = 1) - SSL provided complete automation: no manual labels (#L = 0), no parameter tuning, automated size filtering and hole filling. - Generality: Worked across five imaging modalities (phase, transmitted light, DIC, IRM, fluorescence), multiple cell types, magnifications, and phenotypes (rounded vs spread cells). - Practicality: Ran efficiently on standard laptops (~7 s per 1216×1920 image pair). - Additional utility: OF vectors quantified intracellular actin flow in fluorescence data.

The findings support that leveraging motion via optical flow enables a self-supervised, fully automated segmentation pipeline that generalizes across diverse live-cell imaging conditions without curated labels or user-set parameters. By retraining on each frame pair, the model adapts to temporal changes in cell phenotype and background illumination, mitigating the brittleness seen in supervised models when applied out of distribution. This automation reduces bias from human annotation and pre-processing, improving reproducibility and accessibility for typical biology labs lacking extensive ML expertise or computational resources. Comparisons with CellPose, a strong supervised generalist model, demonstrate that large curated libraries do not guarantee robust transfer to new datasets, especially at lower magnifications and multi-cell scenes. The SSL approach sidesteps label bias and training data mismatch by learning directly from the data to be analyzed, achieving competitive or superior segmentation quality while also automating common preprocessing steps like size filtering and hole filling. The modular feature framework (currently entropy and gradient) allows future incorporation of additional features. Overall, the results align with calls for reproducible ML in life sciences by reducing dependencies on opaque training pipelines and extensive hyperparameter tuning.

This work introduces a fully automated self-supervised segmentation algorithm for live-cell time-lapse imagery that uses Farneback optical flow to self-label training data, then trains a Naïve Bayes classifier on static features to produce semantic segmentations. It generalizes across cell types and imaging modalities without curated labels or user parameters, outperforms a leading supervised generalist model (CellPose) in most tested multi-cell datasets, and matches performance in high-magnification single-cell cases, all while running efficiently on standard hardware. The approach reduces labor, biases, and variability inherent in supervised pre-processing, enhancing accessibility and reproducibility in bioimage analysis. Future work will extend the modular framework with additional features and add instance segmentation via de-clumping (e.g., watershed) to separate touching cells, as well as robust drift compensation modules if needed.

- Applicable only to live-cell time-lapse imagery where motion provides supervision; not suitable for static images. - Requires stable acquisition conditions; lateral stage drift or focus drift can confound the assumption of a stationary background. While modern microscopes are typically stable and software alignment (e.g., ImageJ) can mitigate drift, instability can degrade performance. - Currently performs semantic segmentation only; touching cells are not separated. De-clumping/instance segmentation (e.g., watershed) is proposed as future work. - Relies on adequate motion signals; extremely low-contrast or low-dynamics scenarios may require iterative FD threshold adjustment for sufficient training pixels.