Causal networks for climate model evaluation and constrained projections

The study addresses how to more objectively evaluate climate models and constrain their projections by moving beyond traditional correlation-based metrics that can mask compensating errors. The authors propose that causal discovery techniques applied to atmospheric fields can extract directed, time-lagged interdependencies (teleconnections) that reflect underlying physical processes. They hypothesize that climate models whose causal interaction networks more closely resemble those inferred from reanalyses will better simulate precipitation patterns and that these causal fingerprints may help constrain future precipitation changes. This is important because substantial uncertainties persist across models, especially for precipitation, and standard metrics may allow models to be right for the wrong reasons.

The paper builds on process-oriented model evaluation and emergent constraints literature, noting persistent inter-model differences tied to parameterizations (e.g., clouds, convection, aerosols) and limitations of aggregate metrics. Prior work has used correlations, spectral properties, and storm-track measures for evaluation, as well as causal/graphical models in climate (e.g., PC-based networks and studies of information flow). The authors reference advances in causal discovery for time series (PCMCI) and prior findings that storm tracks and ENSO-related teleconnections shape network connectivity. They also cite multi-model dependence and weighting approaches and previous findings that historical performance in simple metrics does not necessarily constrain future projections.

Causal Model Evaluation (CME) framework: 1) Data and dimensionality reduction: Daily sea level pressure (SLP) anomalies are used as a standard metric of large-scale atmospheric dynamics. PCA followed by Varimax rotation (PCA-Varimax) is applied to NCEP-NCAR reanalysis (1948–2017) to extract spatially localized components representing regional climate modes. For each meteorological season (DJF, MAM, JJA, SON) 50 components (nodes) are selected based on spatial separability and frequency characteristics. The same loadings are applied to CMIP5 model data to extract consistent component time series. Time series are averaged to 3-day means to capture short-term interactions while limiting dimensionality. 2) Causal network reconstruction: Directed, lagged interdependencies among component time series are inferred using PCMCI, a causal discovery method tailored for high-dimensional, autocorrelated time series. Maximum time delay T_max=10 (≈30 days). Linear partial-correlation tests (ParCorr) are used within PC conditioning and MCI testing. The pruning parameter pc-α is selected via AIC over a grid. Links are declared at a strict significance level α=1e-4, with robustness checks over broader α producing similar conclusions. The method removes spurious links from common drivers and indirect paths, yielding directed networks with inferred lags and strengths. 3) Network comparison metric: Networks are compared via a modified asymmetric F1-score that accounts for binary link existence relative to a reference, allows a ±2 time-lag tolerance (≈6 days) and requires matching sign and direction; this balances precision and recall and handles imbalanced link sets. Seasonal F1-scores are averaged to an annual score. 4) Experimental design: - Pre-industrial (piControl) evaluation: For 20 CMIP5 models, 210 years of DJF SLP at 3-day resolution are analyzed. Each model is split into three 70-year segments (treated as ensemble members) to assess robustness. All pairwise comparisons among ensemble members and models are conducted. Two artificial baselines are constructed: Random (5% random coupling at lag-1 plus autocorrelation 0.7) and Independent (only autocorrelation 0.7; no cross-coupling). - Historical evaluation versus observations: For 20 CMIP5 models with ≥3 ensemble members, historical simulations covering ~1948–2017 are analyzed for each season, with reanalysis networks from NCEP-NCAR as the reference (ERA-Interim used for robustness). - Precipitation skill and projections: Model precipitation skill is quantified via Taylor S-scores comparing CMIP5 precipitation to CRU TS v4.02 land precipitation (1948–2017), globally and for regions (e.g., North America, East Asia, Africa, South Asia). Storm-track skill metrics (e.g., pattern correlation of 2–6-day bandpass-filtered SLP standard deviation) are also computed for comparison. Future precipitation change is computed as land mean precipitation rate difference between 2050–2100 under RCP8.5 and 1860–1910. Nonlinear relationships between F1/S-scores and projected changes are modeled with Gaussian Process regression (RBF + white noise kernel; hyperparameters via log-marginal likelihood). 5) Data processing: All SLP and precipitation data are bilinearly interpolated to 2.5°×2.5°, linearly detrended, and anomalies computed by removing long-term daily means. Historical and RCP8.5 series are merged at 2006-01-01 where needed. Stationarity over analyzed lags is assumed for causal inference. Robustness analyses explore link-density sensitivity (hundreds to over a thousand links), alternate reanalyses, and seasonal dependence.

• Model-identifiable causal fingerprints: In piControl DJF analyses, ensemble members of the same model have significantly higher F1-scores against each other (dark red diagonal), indicating stable, model-specific causal fingerprints across multi-decadal segments. - Detection of shared development backgrounds: CME recovers expected model interdependencies. Clear subgroups include HadGEM2-ES/HadGEM2-CC/ACCESS1-0/ACCESS1-3 (HadGEM family), IPSL variants, MPI variants, and GFDL models; two of three MIROC models (MIROC-ESM, MIROC-ESM-CHEM) cluster, while MIROC5 is less similar than the multi-model average. - Physical coherence versus artificial baselines: Random and Independent surrogate systems yield low cross-model F1-scores, confirming that climate model networks share physically meaningful structure absent in artificial systems. - Historical comparisons to reanalyses: Using NCEP-NCAR as reference, models form a continuum of agreement; causal fingerprints differ significantly among models (Kruskal–Wallis p < 9×10^-10; one-way ANOVA p < 5×10^-30). Rankings are similar when using ERA-Interim as the reference. - Link timescales and dynamics: Networks are dominated by interactions with lags ≤1 week, consistent with storm-track dynamics, yet causal network scores correlate better with precipitation skill than simple storm-track pattern metrics, indicating added value from capturing broader causal structures (e.g., ENSO teleconnections and extratropical effects). - Relationship to precipitation skill: Across models, higher F1 (closer to reanalysis fingerprints) associates with smaller land precipitation biases (higher S-scores). Reported linear correlations (R) between F1 and S-score: Global land R=0.70 (p≈5.2×10^-4); North America R=0.74 (p≈1.7×10^-3); East Asia R=0.61 (p≈4.7×10^-3); Africa R≈0.50 (p≈0.026); South Asia R≈0.49 (p≈0.028). Results are robust when excluding models from closely related subgroups. Some regions (e.g., Australia, Indonesia) show weak or insignificant correlations. - Emergent constraint for future precipitation change: No meaningful relationship is found between historical precipitation S-scores and projected global land precipitation change. In contrast, an approximately parabolic relationship emerges between historical F1-scores and projected 2050–2100 minus 1860–1910 global land precipitation rate change under RCP8.5, supported by GP regression (log-marginal likelihood 44.15) and direct parabolic fits. This constrains likely changes toward intermediate values around 0.0–0.1 mm/day, disfavoring extreme negative or positive changes. Simpler dynamical metrics (e.g., SLP S-scores, storm-track metrics) do not yield such emergent relationships.

The findings support the hypothesis that data-driven causal fingerprints of atmospheric dynamics capture physically meaningful coupling pathways that are relevant for hydrological outcomes. CME identifies model families and individual model characteristics, demonstrating both interdependence and distinctiveness across the ensemble. The stronger association of causal network similarity with precipitation skill, compared to simpler storm-track metrics, suggests that causal networks encode additional processes (e.g., remote teleconnections such as ENSO and their extratropical impacts) beyond short-term variability. Crucially, the emergent parabolic link between historical F1-scores and future global land precipitation change offers a pathway to constrain projections using process-informed metrics that are difficult to tune directly. This addresses the challenge that traditional performance metrics often fail to predict future skill due to compensating errors. The approach can inform targeted model development by revealing causal pathways where models diverge from observations, and it can complement existing evaluation frameworks to reduce projection uncertainty, provided underlying physical processes remain relevant under climate change.

The study introduces causal model evaluation (CME) based on PCMCI-inferred causal networks of SLP variability as a robust, process-oriented evaluation tool. CME: (1) extracts stable, model-specific fingerprints; (2) detects expected development-related interdependencies; (3) links causal fingerprint fidelity to improved precipitation simulation across key populated regions; and (4) reveals an emergent, approximately parabolic relationship that constrains projections of global land precipitation changes toward intermediate values (≈0.0–0.1 mm/day) under strong forcing. These contributions demonstrate the potential of causal discovery to improve model evaluation and to inform constrained projections. Future work should test the approach across CMIP6 ensembles, additional scenarios, variables beyond precipitation, alternative causal and dimensionality reduction methods, and more granular spatial scales, as well as investigate causal pathway interpretations to guide model development.

• Generalizability of emergent relationships: Constraints are derived within the CMIP5 ensemble and RCP8.5 scenario; applicability to other ensembles (e.g., CMIP6) and forcings must be verified. - Metric scope: The causal metric emphasizes dynamical interactions; processes predominantly thermodynamically driven (e.g., soil moisture feedbacks) may not be fully captured, potentially limiting relevance for certain future changes. - Model structural errors: Processes underrepresented or missing in current models could become important in the future, challenging the assumption that present-day causal structures persist. - Node selection and coverage: PCA-Varimax component selection excluded some regions with questionable reanalysis reliability (e.g., parts of Asia, Africa, Middle East), reducing network coverage there, though global metrics appeared insensitive. - Causal assumptions and stationarity: PCMCI relies on assumptions (Causal Markov, Faithfulness, stationarity over considered lags). Very long time-scale nonstationarities cannot be fully addressed with available samples. - Finite-sample and hyperparameter choices: Results depend on significance thresholds, lag tolerances, and component counts, although sensitivity analyses indicate robustness across wide ranges. - Precipitation evaluation data limitations: Observational uncertainties in CRU precipitation and regional data sparsity may affect S-scores, especially in data-poor regions.