Assessing urban livability in Shanghai through an open source data-driven approach

Rapid urbanization, particularly in developing countries, has brought about significant economic growth but also negative environmental and socio-economic consequences. These include pollution, ecosystem degradation, and limited access to essential services. The impact of urbanization on quality of life remains a subject of debate, with some arguing it decreases with city size while others highlight potential improvements due to economic prosperity and community well-being. Urban livability has emerged as a key concept in urban policy and practice, encompassing various dimensions beyond service availability, such as demographic transitions, socioeconomic changes, cultural factors, and environmental shifts. Existing urban livability assessments often rely on limited or aggregated data, hindering a nuanced understanding of spatially varied living conditions. This study addresses this gap by utilizing readily available open-source data, including spatially explicit demographic, socio-economic, and POI data, which is increasingly available globally but often difficult to integrate meaningfully. China, with its rapid urbanization and evolving research environment, presents an ideal context for developing and implementing data-driven approaches for assessing urban livability. While previous studies have explored this, they often lack the spatially disaggregated detail offered by open-source data. This research focuses on Shanghai, a city with significant internal variability in socioeconomic conditions and service access, leveraging open-source data on residential building clusters (RBCs), population distribution, POIs, and the transportation network to address the challenges of data integration and low spatial resolution inherent in many open data approaches. The study employs advanced data processing and dimensional reduction techniques to extract meaningful insights from the large datasets, ensuring robustness and interpretability, and ultimately providing a more comprehensive understanding of urban livability patterns within Shanghai and the factors influencing them.

Numerous studies have explored urban livability using diverse conceptual frameworks and methodologies. City rankings employing various indicators capture different aspects at the city scale. Some studies create composite urban livability indices for city comparisons or distribution pattern analysis, exploring relationships between these indices and urban activities like transport choice or urban form. However, these conventional approaches often rely on limited indicator sets or highly aggregated data, inadequately utilizing the wealth of spatially explicit data now available. While POI data-driven approaches are increasingly used in tourism and urban planning, their application in comprehensive urban livability assessments remains limited. Challenges in integrating diverse open-source datasets due to differences in spatial resolution, coverage, quality, or accessibility are significant barriers. Data overload and difficulty in generalization often hinder effective utilization of large open-source datasets for livability assessments. The existing literature on data-driven urban livability studies, particularly in the context of rapidly urbanizing areas like China, is still limited, necessitating the development of robust frameworks for synthesizing and interpreting readily available open source datasets, while avoiding the pitfalls of potentially incomplete or inaccurate official statistics.

This study employs a data-driven approach to assess urban livability in Shanghai using open-source data. The research focuses on residential building clusters (RBCs) as the primary unit of analysis. Data was collected from various sources including Lianjia (for RBC characteristics such as price, age, and location), Worldpop (for population density at 100m x 100m resolution), and Baidu Maps API (for POIs). The methodology involves several key steps: 1. **Data Collection and Preprocessing:** Data on RBCs, population distribution, POIs (categorized into five major domains: education, medical service, recreation, transportation, and living services), and the transportation network (bus lines and metro entrances) were gathered and preprocessed to ensure data quality and consistency. 2. **Normalized Factor Scores:** A normalized factor score (S) was calculated for each RBC, combining housing price, building age, population density, access to transport, and POI diversity (using logarithmic transformation to handle zero values and ensure positivity). Population density was calculated as the ratio of population to the number of buildings within an RBC. Access to transport was calculated using a cumulative opportunity method (CUM), considering the proximity of RBCs to bus stops and metro stations within 1km and 2km radii. POI diversity was calculated using an entropy-based method within the same radii. Min-max normalization was applied to standardize the values for price, age, density and accessibility to a range of 0-1. 3. **Livability Score Calculation:** Livability scores were computed for each RBC based on a weighted average of the normalized factors and a balance index of POIs (using the Gini-Simpson Index) reflecting both the diversity and potential concentration preferences of residents. Five separate livability scores were generated reflecting the five POI categories, enabling a detailed analysis of the strengths and weaknesses of the different spatial areas. 4. **Spatial Analysis:** Standard Deviational Ellipse (SDE) and inverse distance-weighted (IDW) interpolation methods were used to analyze the spatial distribution and trends of livability scores, providing a visual representation of areas with high and low livability within Shanghai, and highlighting the trends across the city. The analysis considers both 1km and 2km radii around each RBC to account for different walking distances and accessibility levels. The study focuses on eight central districts of Shanghai known for economic vibrancy, diverse amenities, and population density, allowing for a concentrated analysis of POI data and simplified computation.

The study revealed uneven spatial distributions of urban livability in Shanghai. Analysis of residential building clusters (RBCs) and their surrounding areas within 1km and 2km radii showed: * **Uneven Livability Distribution:** With the exception of recreation, livability scores generally followed an east-west distribution pattern in the city's downtown area. This uneven distribution highlights significant discrepancies in access to amenities and services. * **High Livability in Central Areas:** The central and northwestern areas of Shanghai (e.g., Hongkou and Yangpu districts) generally exhibited higher livability scores across multiple dimensions compared to other areas, particularly for living services and transportation, which are essential for citizens' needs. The highest livability scores for living services were observed in the downtown core, reflecting greater ease of access to diverse amenities. * **Recreation Distribution:** Recreation-related livability scores were concentrated in the western part of the city, aligning with the distribution of historical sites and green spaces. * **Medical and Educational Services:** Medical and educational facilities showed a relatively dispersed distribution, primarily in the western part of the city. This uneven distribution affects housing prices and population density of RBCs in different areas. * **Transportation Access:** Transportation-related livability scores covered a vast area, reflecting the city's dense transportation network and extensive commuting demands. * **POI Data Analysis:** The analysis of Points of Interest (POIs) revealed a wide range of distribution patterns and usage frequencies. Named entities were the most frequent type of POI (14.88%), likely due to the incorporation of geographical entities. Shops (13.42%) and daily life services (10.57%) were also significant. The spatial analysis using the Standard Deviational Ellipse (SDE) method visually illustrated the distribution patterns of different livability dimensions, highlighting the spatial variability and potential trade-offs in urban livability across various aspects.

The findings highlight significant spatial inequalities in urban livability within Shanghai. The uneven distribution of essential services like medical and educational facilities warrants attention, particularly in areas with lower densities or limited access. Expansion of these services to underserved areas in the north, east, and south is crucial. This requires coordinated action, which has precedents in China. Enhancing the diversity and availability of living services and recreation opportunities through government investment and economic incentives are also necessary. The unbalanced distribution of recreation opportunities points to the need for additional green spaces, although this is often challenging in rapidly urbanizing cities. While efforts have been made to increase green spaces in Shanghai, ensuring equitable access remains a key challenge. The study's results, while not directly comparable to other spatially explicit studies due to the unique methodology, demonstrate similar trends of urban livability differences between city areas. This confirms findings in other cities such as Singapore, Melbourne, Vancouver, and Wuhan. The study complements previous research on Shanghai, highlighting the interconnectedness of urban livability challenges, emphasizing the transport sector, the built environment, and green spaces. The areas of Shanghai with higher urban livability show some similarity to areas of greater urban vitality and street livability, indicating that this study provides potentially useful insights into existing and prior research.

This study demonstrates the potential of open-source data-driven approaches for assessing sub-city level urban livability patterns, offering a valuable tool for urban planners and policymakers. By leveraging readily available data, this method can circumvent limitations posed by the lack of or low-quality official spatial data. The framework's applicability to other cities in China and beyond is promising, although adjustments for local contexts are necessary. Future research should focus on integrating additional data sources (e.g., environmental indicators like air quality), refining the livability scoring system, and expanding the geographical scope to encompass a wider range of urban contexts. Further exploration of the relationships between specific livability dimensions and factors like socioeconomic status would enrich the understanding of urban livability inequalities.

The study has some limitations that should be acknowledged when interpreting the results. Firstly, integrating various datasets with different spatial resolutions and coverage introduced uncertainties, especially in combining RBC-level data with 100m resolution population data. Secondly, some crucial aspects of urban livability (e.g., air quality, detailed recreation facilities) could not be fully incorporated due to data limitations. Thirdly, the RBC data may not be completely comprehensive, potentially affecting the sample's reliability. Fourthly, a composite livability score across all dimensions was not feasible due to variations in the denominator of each livability score calculation. Finally, while the methodology is transferable, the analysis is specific to the Shanghai context and cannot be directly generalized without adjustments for each city's unique characteristics.