Street View Video for Urban Sensing: Potential for Measuring Urban Vitality

Urban vitality is a key quality of the livability and attractiveness of a city, with street-level social dynamics recognized as a central representation. However, existing approaches to capturing such dynamics, primarily through pedestrian volume, rely on static imagery, location-based proxies, or custom sensors. Such methods are typically discontinuous, limited in spatio-temporal resolution, costly, and overlook the role of sound in shaping environments. To address these gaps, this study explores the feasibility of using Street View Video (SVV), a novel, promising, yet under-utilized data source, for capturing street-level dynamics. Leveraging widely available crowdsourced city tour videos as a representative form of SVV, we develop an automated and scalable framework for high-precision geo-localization of SVV by extracting textual and depth information. Further, we propose a visual-auditory pipeline that integrates continuous pedestrian activity and sound environments as complementary dimensions that reveal street-level social dynamics. Using Melbourne as a case study, we identified 892 high-quality and geolocatable videos totaling over 541 hours and achieved an accuracy of 77% for geo-localization. We generated spatio-temporal maps that not only reproduced well-studied pedestrian volume patterns, but also previously neglected aspects, including human actions (walking, standing, sitting), sound intensity, and sound types (speech, music, vehicle). Correlation and quadrant analyses further explored the relationships among these vitality dimensions, revealing distinctive street characteristics and highlighting the limitations of conventional road classifications. Overall, these findings demonstrate the potential of SVV as a multimodal and reproducible source of urban vitality indicators, providing a proof-of-concept for future urban sensing applications.