Harmonizing government responses to the COVID-19 pandemic

Government responses to the COVID-19 pandemic, including lockdowns and travel bans, profoundly impacted societies worldwide. Access to comprehensive and high-quality PHSM data is crucial for understanding these responses and their effects on various sectors, including the economy, environment, and society, as well as their impact on disease spread. While numerous research groups have documented COVID-19 PHSM, their individual efforts provide incomplete and fragmented data, often due to funding limitations. This paper addresses this issue by harmonizing data from eight of the largest PHSM tracking projects – ACAPS, COVID AMP, CIHI, CoronaNet, HIT-COVID, OxCGRT, WHO EURO, and WHO CDC – into the CoronaNet taxonomy. The goal is to create a more comprehensive dataset for researchers studying the pandemic's impact. The paper details the methodology used for harmonization, including the challenges faced and strategies employed, providing a transparent and detailed account of the data harmonization process.

The introduction cites the existing literature on COVID-19 PHSM, highlighting the limitations of existing data due to funding constraints and variations in data collection methods. This implicitly reviews the literature by demonstrating the need for the current study, which focuses on creating a more complete and consistent dataset by harmonizing existing ones.

The authors employed a five-step semi-manual harmonization methodology. Step 1 involved creating taxonomy maps between the CoronaNet taxonomy and each external dataset, making these maps publicly available. Step 2 performed basic data cleaning and subsetting. Step 3 employed automated deduplication algorithms to identify and remove duplicates within and across datasets, prioritizing datasets based on data quality assessments. Step 4 involved piloting the manual harmonization process on a subset of data for validation and refinement of methods. Step 5 involved manually assessing overlap between CoronaNet and external datasets, and recoding non-overlapping data into the CoronaNet taxonomy. The manual harmonization process utilized Google Sheets, providing a centralized platform for multiple researchers to contribute. Detailed steps are provided for each stage, including the criteria for identifying duplicates, prioritizing datasets, and handling missing or incomplete data. The methodology emphasizes manual verification to ensure accuracy, acknowledging the limitations of automated methods when dealing with inconsistent and incomplete data.

The paper presents various tables and figures that illustrate the challenges and results of the harmonization process. Key findings include: * **Dataset Selection:** The authors justify their choice of datasets based on geographical and temporal coverage, data volume, taxonomy similarity to CoronaNet, and collaborative capacity. * **Challenges of Harmonization:** The authors detail several challenges, including different taxonomic approaches, dirty data (inaccurate or incomplete coding), and inconsistently preserved raw sources. They discuss the difficulties of mapping conceptually similar policies with different terminology or structural implementation. The problems posed by dirty data are analyzed, particularly concerning miscoded policies, missing information, and the impact on data reliability. The challenges posed by the lack of consistently preserved raw sources for validation are analyzed. Tables are provided illustrating the extent of the problem. * **Data Quality Assessment:** The study provides quantitative assessments of data quality across datasets based on description length, missing data, and data completeness. The high inter-coder reliability (around 80%) of the CoronaNet dataset for its policy type variable is highlighted in contrast with external datasets. * **Harmonization Process and Results:** The five-step harmonization process is detailed. Tables present the status of the external data at each step, showing the reduction in the number of policies after cleaning and deduplication, and the progress of manual harmonization. Figures visualize the differences in policy coverage across time and regions. Results reveal that a large proportion of the external data (approximately 83%) did not overlap with the CoronaNet dataset, indicating substantial gains in data completeness. They show that around 45% of non-overlapping data was successfully harmonized into the CoronaNet taxonomy and dataset. * **Value of Harmonization:** The paper argues that the harmonized dataset provides a more complete and consistently coded dataset than any single existing dataset. It demonstrates improved coverage across time, space, and administrative levels. The paper notes that some information may be lost from harmonization due to the CoronaNet taxonomy not capturing all measures from the external datasets.

The harmonization effort significantly expands the coverage and consistency of COVID-19 PHSM data. The challenges encountered underscore the complexity of data harmonization, especially with data collected under emergency conditions. The results show substantial gains in data completeness, improving the ability to conduct more robust and generalizable analyses. While the harmonized dataset is not exhaustive, it represents a significant improvement over existing resources and provides a valuable resource for future research. The authors highlight the inherent limitations, including gaps in subnational data and potential for residual errors, emphasizing the ongoing nature of their work. They also emphasize that data production itself involves the framing and creation of reality, highlighting the importance of transparency and methodological rigor in the harmonization process.

This paper presents a substantial effort to harmonize COVID-19 PHSM data from eight major datasets. The resulting harmonized dataset significantly improves data completeness and consistency. Despite limitations, this resource greatly enhances the understanding of government responses to the pandemic and serves as a valuable model for future data harmonization projects in the social sciences. The authors call for further research and funding to address remaining gaps, especially regarding subnational data and challenges posed by low state capacity. They suggest making their approach more available and accessible to the research community in the future.

The harmonized dataset is still incomplete, particularly for subnational policy making in many countries and for low-state capacity governments where data collection is particularly challenging. The study acknowledges the inherent limitations of data harmonization and the potential for residual errors despite rigorous methodology. The study is limited in its temporal scope, focusing on policies implemented before September 21, 2021.