Cultural heritage tourism is a significant and rapidly growing sector, severely impacted by the COVID-19 pandemic. The pandemic's economic losses significantly affected the tourism industry globally, impacting both tourist behavior and the perceptions of stakeholders. Health concerns and travel inconveniences caused hesitation, shifting the industry's focus from tourists as 'ambassadors' to managing the virus's transmission. This study investigates the impact of these changes on the visitor experience at UK heritage sites, recognizing geographical factors influence the pandemic's effects. While developed economies suffered losses, the impact on developing economies, heavily reliant on tourism, was more devastating. In the UK, heritage site closures, especially during peak seasons, caused significant financial difficulties. The study uses the COVID-19 pandemic as a case study to demonstrate a novel methodology for analyzing large-scale social media data to assess the impact of disruptive events on heritage tourism. The study aims to analyze the impact of COVID-19 on visitor numbers and experiences at UK heritage sites and to demonstrate the effectiveness of advanced machine learning techniques in interpreting visitor sentiments from social media.

Existing research highlights the variable impacts of COVID-19 on heritage tourism based on geographical factors and site characteristics. Studies in countries like India, Australia, and Malaysia documented significant negative impacts, while the impacts varied considerably across countries depending on their level of economic development. Research suggests urban tourism faced severe consequences due to economic uncertainty and travel restrictions, while rural tourism, mainly relying on domestic tourists, was less affected. Outdoor sites generally proved more resilient than indoor sites. In the UK, heritage organizations faced high risks to their long-term viability. Previous studies have successfully employed social media data to analyze the impact of COVID-19 on various societal aspects. Ginzarly and Srour (2022) examined cultural heritage content on Instagram, while Ridhwan and Hargreaves (2021) analyzed Twitter data in Singapore to assess public sentiment and emotion. Sanders et al. (2021) investigated attitudes towards face coverings on Twitter, and Lyu et al. (2021) examined vaccine-related content. This study builds upon this existing research by applying state-of-the-art weakly supervised and zero-shot learning language models to analyze a large dataset of Google Maps reviews, providing a more nuanced understanding of visitor perceptions towards COVID-19 measures in UK heritage sites.

This study utilizes data collected from Google Maps, encompassing over 1.4 million reviews for 775 UK heritage sites from February 2006 to April 2022. Inclusion criteria required over 100 reviews per site and a primary association with cultural heritage and tourism. Data sources included heritage organizations (English Heritage, National Trust, etc.), lists of most-visited attractions, and other relevant sites. The dataset includes textual comments and imagery data. Sites were classified as indoor/outdoor using a convolutional neural network (CNN) model (places365) analyzing the 'outdoorness' of collected photographs, and urban/rural classifications were determined using the Code-Point dataset based on population density. COVID-19 related comments were identified using keyword searches (keywords associated with COVID-19 such as 'covid', 'coronavirus', 'social distance', etc.) and a fine-tuned BERTweet-large model implementing natural language inference, aiming to minimize false positives. The number of online reviews served as a proxy for visitor involvement. The study compared the actual number of reviews with an expected number (estimated using ARIMA) to assess recovery levels. The relationship between review volume and official visitor data from DCMS-sponsored sites was also investigated. Correlation analysis explored the relationship between the reduction in online reviews and inbound visitor numbers from various sources and purposes. Sentiment analysis was conducted at both document and word levels. At the document level, sentiment was categorized (positive/negative) using user ratings; the four sub-topics of face covering, social distancing, restrictions and sanitization were used to analyze sentiment in COVID-19 related comments. Keyword searches and a weakly supervised text classification model (BFV) were utilized to identify these subtopics. A sentiment language model (DistilBERT) with Integrated Gradient analysis was used for word-level sentiment analysis. This enabled identifying words associated with positive or negative sentiments.

The study analyzed 15,300 COVID-19 related comments from 689 sites. The number of COVID-19 related comments decreased from the end of 2021, indicating diminishing visitor focus on the pandemic. However, the difference between actual and expected review numbers remained significant, particularly in indoor urban sites, suggesting a slower recovery of visitor numbers. Correlation analysis revealed significant positive correlations between the reduction in monthly comments and inbound visitors from Europe (holiday purpose) in urban sites, but not rural sites. Sentiment analysis showed that sanitization was positively associated with sentiment in indoor sites but not outdoor sites. Social distancing was positively associated with sentiment in outdoor sites but not indoors. Restrictions and closures were consistently negatively associated with visitor sentiment. Word-level sentiment analysis revealed 'closed' and 'restrictions' as strongly negative words, while 'COVID' was surprisingly associated with positive sentiment, possibly due to the positive association with the end of restrictions. A pseudo-qualitative analysis of negative comments on face coverings and social distancing indicated that lack of enforcement and non-compliance by other visitors were the primary sources of negative sentiment. These results suggest general acceptance of measures but emphasis on enforcement and maintaining access to sites.

The findings address the research question by demonstrating the significant impact of COVID-19 on UK heritage tourism, particularly for urban, indoor sites heavily reliant on international visitors. The slower recovery of these sites highlights the need for support policies and financial planning. Effective enforcement of measures is crucial for positive visitor experiences. The methodology provides a valuable alternative to traditional visitor surveys, offering cost-effectiveness and scalability. The results emphasize the importance of maintaining access to sites while ensuring safety and the value of using online reviews for gathering comprehensive and cost-effective visitor feedback.

This study successfully measured the impact of COVID-19 on UK heritage tourism using social media data and advanced machine learning. Key findings emphasize the slower recovery of urban, indoor sites and the importance of effective COVID-19 measure enforcement. The use of online reviews provides a valuable, cost-effective methodology for assessing visitor perceptions of heritage sites during disruptive events. Future research could explore the long-term effects of the pandemic, examine the impact on specific types of heritage sites, and investigate other disruptive events.

This study's limitations include language ambiguity in natural language processing, uneven topic coverage in user reviews, and the passive nature of data collection. Specific limitations related to the dataset include potential biases from the Google Maps platform (e.g., overrepresentation of younger or tech-savvy users) and the exclusion of non-English reviews. The findings should be interpreted with caution, considering these limitations.