Forecasting trends in food security with real time data

The global food crisis, exacerbated by conflict, climate change, and economic shocks, necessitates effective early warning systems. Humanitarian agencies like the WFP require timely and accurate forecasts to target assistance effectively. Machine learning (ML) offers a powerful tool for developing such forecasting systems by analyzing diverse data streams. This study leverages the WFP's Real-Time Monitoring (RTM) system, which provides daily updates on key food security indicators, to develop a 60-day food consumption forecast at the sub-national level. This frequent update is crucial for dynamic resource allocation in humanitarian emergencies, unlike other systems with less frequent data releases. The study focuses on predicting 'insufficient food consumption' using data from four countries: Mali, Nigeria, Syria, and Yemen. The choice of a 60-day forecast horizon balances the need for timely intervention with logistical constraints of data distribution and resource planning.

Existing research on food security modeling spans decades, with recent work exploring the application of ML techniques. The FAO has investigated long-term country-level forecasts, while the World Bank has focused on modeling Integrated Food Security Classification (IPC) phases using FEWS-NET data. Other studies have employed ML algorithms for household classification based on caloric intake, forecasting food security indicators like the Food Consumption Score (FCS) and Household Dietary Diversity Score (HDDS), and incorporating text-based features from news articles to enhance predictive capabilities. While previous studies have shown promise, many suffer from high computational costs, limited transferability, or reliance on less frequent data updates. This study aims to address these limitations by utilizing the WFP's high-frequency RTM data.

The study employed five forecasting methodologies: ARIMA, CNN, LSTM, XGBoost, and RC. The primary target variable was the aggregated percentage of households with insufficient food consumption at the sub-national level, derived from the WFP's RTM system's Food Consumption Score (FCS). Secondary variables included climate data (rainfall, NDVI), conflict data (battle fatalities, civilian fatalities), economic data (inflation, currency exchange), and external data (crop calendars, Ramadan). For enhanced transparency and explainability, an ensemble method was used for dynamic feature selection. The core methodology focused on Reservoir Computing (RC), a simplified recurrent neural network known for its efficient training and robustness. An ensemble of 100 RC models was used to reduce the effects of random weight initialization. The RC model iteratively predicts the target and secondary variables, incorporating known future values of exogenous variables (e.g., Ramadan). A walk-forward optimization procedure was used to evaluate the model's performance across different time periods, simulating real-world retraining scenarios. The performance of all models was evaluated using the Root Mean Squared Error (RMSE). A classification task, categorizing future behavior as 'Deteriorating,' 'No Change,' or 'Improving,' was also performed.

The Reservoir Computing (RC) model consistently outperformed ARIMA, CNN, LSTM, and XGBoost in terms of RMSE across different aggregation methods (forecasting time step, target variable variation, and country). The RC model's error increased more slowly than other algorithms over the 60-day forecast horizon, particularly excelling when the target variable underwent substantial changes. In the classification task, RC demonstrated superior performance in accurately predicting 'Deterioration' events, although all models showed a conservative bias towards predicting 'No Change'. The RC model’s training time was comparable to other methods, typically less than 1 minute per hyperparameter combination. Analysis of feature selection indicated that the most frequent feature groupings selected were those including the target variable (FCS), suggesting the importance of autoregressive components. However, climate and economic data also played significant roles in certain instances. The results highlight the suitability of the RC framework for constructing early warning systems due to its performance, resource efficiency, and ease of use compared to more complex deep learning methods. Comparisons with previous studies show that RC achieves comparable or better RMSE values.

The findings demonstrate the effectiveness of the proposed methodology, based on the RC algorithm and the WFP's RTM data, for predicting insufficient food consumption 60 days into the future. The superior performance of RC over other tested algorithms, especially in predicting severe deteriorations, highlights its suitability for early warning systems in humanitarian contexts. The model's relatively low computational requirements and ease of use make it practical for implementation in data-constrained environments. The dynamic feature selection adds to the model's transparency and adaptability to varying conditions across different regions and time periods. The study's limitations, such as the focus on four specific countries, and the relatively limited frequency of severe deterioration events in the data, suggest areas for future research.

This research presents a novel forecasting methodology based on Reservoir Computing, effectively predicting food insecurity trends using high-frequency data. The RC model's superior performance and efficiency make it a promising tool for developing robust early warning systems. Future work should focus on extending the forecasting horizon, improving the detection of severe deterioration events, optimizing the model for different data granularities, and expanding the geographical scope of the study.

The study's findings might not be generalizable to all contexts due to the focus on four specific countries with varying characteristics and data availability. The relatively limited number of severe deterioration events in the dataset could have affected the model's performance in those situations. Further research with a more comprehensive dataset and across a wider range of countries is needed for enhanced generalizability.