Al perceives like a local: predicting citizen deprivation perception using satellite imagery

Urban inequality is a significant social problem, particularly in Low-and Medium-Income Countries (LMICs), where rapid urbanization has led to the emergence of deprived areas, often called slums. These areas exhibit high levels of physical deprivation, intertwined with energy poverty and environmental risks. While Earth Observation (EO) methods and Artificial Intelligence (AI) have shown promise in capturing physical aspects of urban deprivation, their ability to predict how locals perceive deprivation remains largely unexplored. This research aims to bridge this gap by developing a method that integrates EO data, citizen science, and AI to predict citizens' perception of deprivation. The study addresses three key research questions: (i) Can satellite imagery reliably capture perceived physical deprivation by citizens? (ii) Can AI, using satellite imagery, predict citizens' deprivation perception? (iii) What physical environment features most influence citizens' perception of deprivation? The study leverages the extensive spatial coverage, temporal frequency, and high resolution of EO data, coupled with advancements in AI, particularly machine learning (ML) and deep learning (DL) algorithms. However, challenges remain, including the variability of deprivation across cities, the lack of reliable reference data, and the underrepresentation of deprived areas in training sets. Citizen science provides a crucial element, offering a cost-effective and participatory means of gathering data on perceived deprivation. Previous citizen science assessments have largely relied on street-level photographs, which are not globally available, especially in LMICs. This research innovatively integrates EO data into citizen science assessments, using satellite imagery as a more readily available and comprehensive data source.

Existing literature highlights the potential of AI and Earth Observation (EO) in mapping physical aspects of urban deprivation, such as building footprints and land cover. Studies have used various techniques, including machine learning and deep learning algorithms, to analyze satellite imagery and identify areas with high levels of deprivation. However, there's a gap in research focusing on how these physical characteristics relate to citizens' actual perceptions of deprivation. The use of citizen science in urban studies has also been explored, primarily using street-level imagery and focusing on high-income countries. This study aims to bridge this gap by combining satellite imagery, AI, and citizen science to predict citizen perception of deprivation, especially in data-scarce environments like LMICs.

The study was conducted in Nairobi, Kenya, focusing on seven slums covering approximately 20 square kilometers. A grid-based approach divided the area into 1998 1-hectare (100m x 100m) grids. WorldView-3 satellite imagery, land-cover classifications, OpenStreetMap (OSM) data for roads and rivers, and Google Open Buildings data were used. A crucial aspect was the use of citizen science: a mobile website was developed for pairwise comparisons of satellite image subsets (100m x 100m) by slum residents. Participants were asked, “Which is the best place to live?”, enabling the creation of a deprivation perception score, where higher scores indicated less deprivation. Over one million votes were recorded, and a TrueSkill Bayesian rating system was used to derive a deprivation score (0-1) for each grid cell. The data was then used to train and validate both deep learning (DL) and conventional machine learning (ML) models to predict the deprivation scores. Two CNN architectures (VGG and DenseNet-121) were used for DL, with experiments conducted using RGB and RGNir image bands. Transfer learning was employed using DenseNet-121 pretrained on ImageNet. For ML, three algorithms (SVM, RF, XGBoost) were tested, using standardized and log-transformed features derived from the geospatial datasets. Feature selection was performed using Lasso regularization and Pearson correlation analysis. The performance of the models was evaluated using R-squared and Root Mean Squared Error (RMSE). Finally, the feature importance of the best-performing ML model was analyzed to identify influential urban characteristics.

The study yielded three key findings: 1. **Satellite imagery effectively captures perceived urban deprivation:** The citizen science-based deprivation scores, derived from satellite imagery, provided a robust measure of perceived deprivation, overcoming the limitations of incomplete street-level imagery coverage. High levels of individual consistency in the pairwise comparisons were observed, though some variation in group agreement was noted, possibly due to subtle differences between image pairs and contextual factors. 2. **AI accurately predicts citizen-perceived deprivation:** Both deep learning and conventional machine learning models accurately predicted the citizen-derived deprivation scores. Deep learning models, particularly the pre-trained DenseNet-121 model (R² = 0.903), significantly outperformed conventional machine learning models, demonstrating AI's ability to replicate citizen perception of urban deprivation. The pre-trained DenseNet-121 model also exhibited significantly lower variance compared to models trained from scratch, indicating higher robustness and stability. 3. **Specific urban features strongly influence deprivation perception:** Machine learning analyses revealed the most influential factors driving deprivation perceptions. Road density emerged as the most significant feature, followed by building proportion and ground surface. The presence of rivers, waste piles, and roof colors were also found to be important predictors, corroborating observations from slum dwellers. The findings indicate the importance of considering infrastructural aspects (roads), environmental factors (rivers, waste), and building characteristics (size, roof color) when assessing deprivation.

This study successfully demonstrated that AI, trained on citizen-perceived deprivation scores derived from satellite imagery, can accurately predict deprivation levels in slums. The findings highlight the potential of integrating citizen science with advanced AI and EO technologies for urban planning and policymaking. The high accuracy of the pre-trained DenseNet-121 model suggests the potential for developing scalable and transferable models that could be applied to other cities and regions, even with limited local training data. The interpretability of the conventional machine learning models further aids in identifying key features of the urban environment that drive deprivation perception, which can guide targeted interventions. However, the study's reliance on a non-random sample of citizen scientists limits the generalizability of the findings. Future work should focus on addressing this limitation and expanding the geographic scope of the study.

This research makes significant contributions by demonstrating the feasibility of using satellite imagery, citizen science, and AI to assess and predict citizen perceptions of urban deprivation. The high accuracy of the AI models, combined with the interpretability of the ML models, provides a powerful tool for understanding and addressing urban inequalities. Future research should focus on expanding the study's geographical scope, incorporating larger and more representative citizen science datasets, and exploring the use of explainable AI methods to enhance model transparency and usability for local stakeholders. The development of a globally applicable tool for mapping and predicting citizen-perceived deprivation could significantly improve urban planning and the implementation of sustainable development goals.

The study's limitations include the use of a non-random sample of citizen scientists from only seven slums in Nairobi. While the diverse socio-economic profiles of the participants were considered, the lack of random sampling limits the generalizability of findings to the entire population of Nairobi’s slums. Another limitation is the reliance on a single time point for satellite imagery, which may not fully capture the dynamic nature of slums. The interpretability of deep learning models remains a challenge, limiting our ability to fully understand their predictions. Finally, data augmentation techniques were employed to compensate for limited training data; however, it is possible that overfitting to the specific augmentation strategy occurred.