Learning noise-induced transitions by multi-scaling reservoir computing

Noise-induced transitions occur in many multistable systems (e.g., circuits, genetic switches, protein conformations, and chemical reactions). In many real settings only time series are available and the underlying equations are unknown. Learning and predicting these transitions requires distinguishing fast (noise/relaxation within wells) and slow (rare transitions between wells) time scales, which standard methods struggle to separate. Prior machine learning approaches (e.g., SINDy variants, physics-informed neural networks, Koopman-based methods, recurrent networks, FORCE learning) are effective for denoising and learning deterministic dynamics but have not captured noise-driven transitions, even in simple bistable systems with white noise. This work asks whether one can learn and predict noise-induced transitions directly from data, without prior knowledge of governing equations, by exploiting multiscale structure.

The paper reviews data-driven dynamical systems methods: SINDy for identifying nonlinear dynamics and parameterizing noisy probability distributions; physics-informed neural networks for PDE discovery; Koopman-based deep learning for linear embeddings. These methods often require large datasets, complex training, and are oriented toward deterministic dynamics or denoising. Empirically, SINDy (2016 and 2021 versions), RNNs (e.g., LSTM), and filtered-data SINDy fail to predict stochastic transitions in bistable systems with white noise; FORCE learning variants can sometimes capture transitions but at higher computational cost and with limited performance on experimental data. Thus, there is a gap for methods that explicitly learn noise-induced transitions from noisy time series.

The authors propose multi-scaling reservoir computing (RC) to learn noise-induced transitions in a model-free way by separating slow deterministic dynamics from fast noise. Core idea: exploit the leak rate hyperparameter alpha (time-scale parameter) to tune the RC to capture slow dynamics, then separate residuals as noise. Training phase: given time series u_t containing both scales, use an echo state network with state r_{t+1} = (1 − alpha) r_t + alpha tanh(A r_t + W_in u_t), and output u_{t+1} ≈ W_out r_{t+1}. Only W_out is trained via ridge regression minimizing sum_t ||u_t − W_out r_t||^2 + beta ||W_out||^2, yielding closed-form W_out = (U R^T)(R R^T + beta I)^{-1}. Choosing a smaller alpha targets slower time scales (continuous-time form shows 1/alpha scaling). Hyperparameter search: two strategies are used. (1) Stable state convergence heuristic: segment training data between large jumps to infer stable states (segment means). Adjust alpha first, then other hyperparameters (N, K_in, average degree D, spectral radius rho, regularization beta) so that trajectories from multiple initial conditions under the trained RC converge to the inferred stable states. If no convergence, adjust in the opposite direction; move to next hyperparameter when improvements stall. (2) Power spectral density (PSD) matching: choose hyperparameters to maximize the match between PSDs of predicted and training time series, indicating accurate capture of deterministic components. Noise separation: with trained slow-scale RC, compute residuals eta_{t+1} = u_{t+1} − û_{t+1} as the fast-scale noise; optionally rescale noise magnitude if convergence speed mismatch skews residual intensity. Prediction (white noise): perform rolling prediction starting from u_s. At each step, compute û_{s+1} from RC and add a noise sample mu_{s+1} drawn from the separated noise distribution to form the next input u_{s+1} = û_{s+1} + mu_{s+1}. Repeat to generate stochastic trajectories and evaluate statistics (transition counts and transition-time distributions). Prediction (colored noise): use a second RC to learn the time evolution of the separated noise series (since colored noise has memory). Train the first RC on deterministic slow-scale dynamics; train a second RC with its own hyperparameters (often smaller alpha to smooth and capture the colored noise trend) on the extracted noise. In prediction, roll out the deterministic RC and simultaneously generate predicted noise from the noise RC, additively driving transitions. Systems studied include 1D and 2D gradient and non-gradient SDEs (including tilted potentials, tristable systems), and an experimental protein folding dataset. Hyperparameters used in examples are summarized in the paper’s Table 1. Code is implemented in PyTorch and released publicly.

- Conventional approaches underperform: SINDy-2021 separates noise but fails to find stable states or predict stochastic transitions and is computationally costly; SINDy-2016 fails on noisy data even after filtering; RNNs do not accurately predict transitions; FORCE can capture some white-noise cases but is computationally heavier and performs poorly on experimental data. - White-noise bistable gradient system (1D): Using multi-scaling RC with alpha tuned to slow scale, the method accurately reproduces transition statistics. Example results: over 100 replicates and 10000 δt prediction windows, the average number of transitions is close (True: 41; Predicted: 37). The average transition time distribution matches well (True mean 2.34; Predicted mean 2.59). Rolling predictions show qualitatively similar dynamics to test data. A small scaling (e.g., 1.1) of sampled noise can compensate for slight convergence-speed mismatches and improve accuracy. - Colored-noise bistable system (Lorenz-63-driven): With a second RC trained on the separated noise, the method predicts the timing of a specific stochastic transition accurately without prior knowledge of deterministic equations (unlike earlier approaches). Across 50 prediction runs with the same hyperparameters, the average predicted trajectory closely matches the ground truth with near-zero mean absolute error around the transition interval. - 2D bistable non-gradient system (with rotational dynamics due to non-detailed balance): The approach reconstructs slow dynamics and predicts transition statistics (counts and transition-time histograms) that match simulated ground truth across 100 replicates, despite rotational behavior complicating the dynamics. - Protein folding experimental data (talin): The method learns upward (native→unfolded) and downward (unfolded→native) transition-time distributions and reproduces asymmetric dynamics using a single hyperparameter set. Accurate prediction is achievable with reduced training data; approximately 7500 time steps suffice, while 6000 steps yield noticeably larger errors. Compared with SINDy-2021 and FORCE, the proposed method achieves higher accuracy on this dataset. - Robustness and awareness of structure: The method can recognize asymmetry in bistable potentials (possibly requiring two hyperparameter sets), handle rotational (non-gradient) dynamics, and perform well across mixed frequency content (white and colored noise). PSD matching is a useful diagnostic for hyperparameter selection.

The study addresses the challenge of learning noise-induced transitions directly from noisy time series by separating time scales via the reservoir leak parameter alpha. By aligning alpha with slow dynamics, the RC captures deterministic drift between stable states, while residuals provide a data-driven noise model. For white noise (memoryless), sampling from the learned residual distribution enables accurate statistics of transition times and counts. For colored noise (with memory), a second RC models the noise evolution, enabling prediction of specific transition timing. The approach succeeds where denoising-focused or purely deterministic model discovery methods typically fail, because it treats noise as functional rather than a nuisance. The method generalizes across gradient and non-gradient systems, handles tilted/asymmetric landscapes, and works on experimental protein folding data with limited samples. Hyperparameter selection, especially alpha, is pivotal; PSD matching and convergence-to-stable-states heuristics offer practical guidance. Overall, the framework demonstrates that multiscale RC can capture stochastic transition mechanisms and statistics in complex systems from data alone.

The paper introduces a model-free, multi-scaling reservoir computing framework that learns noise-induced transitions by tuning the reservoir time scale to capture slow dynamics and separating fast fluctuations as noise. It delivers accurate transition statistics under white noise and precise transition timing under colored noise across a variety of synthetic SDE systems and experimental protein folding data, often with modest training lengths. The method is computationally efficient (simple linear regression for W_out) and robust across parameter ranges. Future work may incorporate automated hyperparameter optimization (e.g., Bayesian optimization, simulated annealing), extend to systems with hidden nodes/links, other noise types (e.g., learned via conditional GANs), and applications to dynamical phase transitions in many-body systems.

- Hyperparameter selection lacks a universal procedure and currently relies on trial-and-error guided by convergence heuristics and PSD matching; suboptimal choices can degrade performance. - For asymmetric (tilted) potentials, two separate hyperparameter sets may be needed to capture differing time scales, adding complexity. - The separated residuals may misestimate noise intensity if deterministic convergence speed is mismatched, sometimes requiring manual scaling of noise magnitude. - Capturing a broader set of protein folding transitions or exploration of higher-energy barriers may require longer training datasets and increased computational cost. - Colored-noise prediction requires a second RC and careful tuning (typically smaller alpha), adding model components and hyperparameters. - As with RC generally, very large reservoirs risk overfitting without adequate regularization and may increase computation time.