Neural general circulation models for weather and climate

Accurate weather and climate prediction relies heavily on general circulation models (GCMs). These physics-based simulators integrate numerical solvers for large-scale atmospheric dynamics with parameterizations for smaller-scale processes like cloud formation. While traditional GCM development has focused on improving numerical methods and physical models, parameterization remains a challenge, leading to persistent errors and biases. The difficulty in reducing uncertainty in long-term climate projections and estimating extreme weather event probabilities poses significant challenges for climate mitigation and adaptation strategies. Recently, machine learning has emerged as an alternative approach to weather forecasting. Machine-learning models, trained on extensive reanalysis data, have demonstrated impressive skill in deterministic short-term weather prediction, often exceeding the performance of traditional models at a fraction of the computational cost. However, limitations exist. Current machine learning models primarily focus on deterministic prediction, lacking the ability to produce calibrated uncertainty estimates essential for practical weather forecasting. They also struggle to accurately represent derived variables like geostrophic wind and lack the long-term stability needed for climate simulations. Hybrid models, combining GCMs with machine learning, offer a potentially powerful approach. These models aim to leverage the interpretability and established success of traditional atmospheric models while integrating the power of machine learning to improve parameterizations. However, existing hybrid models often suffer from instability and climate drift due to the lack of coupling between the machine-learning components and the governing equations during training. This paper addresses these challenges by introducing a new approach.

The authors review the progress and limitations of traditional GCMs, highlighting the challenges of parameterizing subgrid-scale processes and the resulting biases and uncertainties in predictions. They then discuss the recent successes of machine learning in weather forecasting, particularly the improved accuracy and computational efficiency demonstrated by models like GraphCast and Pangu. However, these machine learning models are limited by their focus on deterministic predictions, their inability to provide calibrated uncertainty estimates, and their inability to perform long-term climate simulations. The authors also discuss existing hybrid models that combine GCMs and machine learning, highlighting their limitations due to offline training of machine-learning components and the resulting instability and climate drift. This motivates the development of a fully differentiable hybrid model.

NeuralGCM, the model presented in this paper, is a fully differentiable hybrid GCM. Its core components are: 1. **Differentiable Dynamical Core:** This core solves the discretized governing equations of atmospheric dynamics (large-scale fluid motion and thermodynamics). The differentiability is crucial for the end-to-end training approach. 2. **Learned Physics Module:** This module uses a neural network to parameterize unresolved physical processes such as cloud formation, radiation, precipitation, and subgrid-scale dynamics. It predicts the effects of these processes on the simulated fields. The key innovation lies in the end-to-end training approach. The model is trained by advancing it multiple time steps and then using stochastic gradient descent to minimize the discrepancy between model predictions and reanalysis data. The rollout length is gradually increased during training, which proves crucial for accuracy and stability. Both deterministic and stochastic versions of NeuralGCM are trained using distinct protocols. NeuralGCM is trained and evaluated at various horizontal resolutions (2.8°, 1.4°, and 0.7°). For weather forecasting, its performance is compared against leading conventional models (ECMWF-HRES and ECMWF-ENS) and other machine-learning models (GraphCast and Pangu). For climate simulations, comparisons are made with a global cloud-resolving model and AMIP runs. The evaluation metrics include RMSE, RMSB, CRPS, and spread-skill ratio, assessing both accuracy and physical consistency.

NeuralGCM demonstrates significant improvements across various timescales: **Medium-Range Weather Forecasting (1-15 days):** * **Deterministic Forecasts:** For short lead times (1-3 days), NeuralGCM's deterministic forecasts (especially at 0.7° resolution) rival or surpass those of GraphCast and achieve comparable accuracy to ECMWF-HRES, although ECMWF-HRES remains slightly better in some metrics. At longer lead times, RMSE increases due to chaotic divergence, highlighting the need for ensemble forecasting. * **Ensemble Forecasts:** NeuralGCM's stochastic ensemble model (NeuralGCM-ENS) significantly outperforms ECMWF-ENS in terms of CRPS across various variables and lead times, demonstrating superior skill in capturing uncertainty. The spread-skill ratio is close to one, indicating calibrated forecasts. * **Physical Consistency:** Unlike some pure machine learning models, NeuralGCM produces less blurry forecasts, suggesting better representation of physical processes. It more accurately represents geostrophic wind balance and shows a realistic spatial distribution of precipitation minus evaporation. * **Generalization:** NeuralGCM shows better generalization to unseen data than GraphCast, suggesting enhanced robustness. **Climate Simulations (Decades):** * **Seasonal Cycle:** NeuralGCM accurately simulates the seasonal cycle of various atmospheric variables, including global mean temperature, precipitable water, and kinetic energy. It also captures essential atmospheric dynamics such as the Hadley circulation, zonal-mean zonal wind, and monsoon circulation. * **Emergent Phenomena:** NeuralGCM successfully simulates emergent phenomena like tropical cyclones, matching both the number and trajectories observed in ERA5 data, outperforming X-SHIELD, a global cloud-resolving model. It exhibits low bias in precipitable water compared to X-SHIELD and climatology. * **Long-Term Trends:** In AMIP-like simulations, NeuralGCM accurately captures global warming trends over 40 years, showing lower bias than CMIP6 AMIP runs. It also shows improved representation of the vertical structure of tropical warming trends. * **Computational Efficiency:** NeuralGCM achieves comparable skill to state-of-the-art models at significantly coarser resolutions, resulting in substantial computational savings (3 to 5 orders of magnitude).

NeuralGCM's success addresses the limitations of both traditional GCMs and pure machine-learning models. The combination of a differentiable dynamical core and a learned physics module allows for end-to-end training, resolving the instability and climate drift issues observed in previous hybrid models. The results support the hypothesis that learning to predict short-term weather effectively tunes parameterizations for climate simulations. The model's computational efficiency opens possibilities for tasks previously impractical with conventional GCMs, like large ensemble forecasting. NeuralGCM’s ability to capture key aspects of atmospheric dynamics, including seasonal cycles and emergent phenomena, demonstrates the potential of this hybrid approach. While NeuralGCM demonstrates excellent performance, limitations remain. Its ability to extrapolate to substantially different future climates is limited, a challenge shared by many current climate models. Further development is necessary to improve generalization to unprecedented climates and to ensure adherence to physical constraints, especially under extreme conditions.

NeuralGCM presents a significant advancement in weather and climate modeling. Its superior performance in medium-range weather forecasting and its ability to accurately simulate climate characteristics, including emergent phenomena, demonstrate the power of combining differentiable physical models with machine learning. Future research should focus on enhancing its ability to generalize to unseen climates, integrating it with other Earth system components, and addressing numerical instabilities. The differentiable hybrid modeling approach has broader implications for simulation across various scientific disciplines.

While NeuralGCM shows promising results, limitations remain. The model's generalization to substantially different future climates is limited, meaning its accuracy may decrease significantly when predicting scenarios far removed from its training data. Furthermore, while computationally efficient compared to high-resolution models, NeuralGCM still requires substantial computational resources for extended climate simulations. Additionally, the current version does not directly distinguish between precipitation and evaporation, which could be improved in future iterations. Finally, integrating the model with other components of the Earth system (ocean, land, chemical composition) presents a challenge for future research.