Design, content validity, and inter-observer reliability of the 'Digitization of Cultural Heritage, Identities, and Education' (DICHIE) instrument

The digitization of cultural heritage is increasingly important for preserving, conserving, restoring, researching, disseminating, and promoting both tangible and intangible cultural assets. The European Commission, through initiatives like Horizon 2020 and Europeana, actively supports member states in this endeavor. Shared production and retrieval of heritage information, along with digital documentation, are key research areas, offering solutions like augmented and virtual reality services to improve physical access. A 2019 declaration by 26 European countries emphasized the synergy between digital technologies and cultural heritage, focusing on 3D digitization, intersectoral cooperation, and public participation. Interactive multimodal technologies enhance understanding, conservation, and educational promotion, fostering bidirectional commitment and co-creation of heritage content. While digital technologies offer significant potential for heritage education, a systematic evaluation of heritage education programs revealed low levels of educational quality and a lack of design specificity when advanced technologies are included. ICTs support access, comprehension, and dissemination of cultural heritage, aiding in the analysis of cultural identities, understanding of historical change, and promotion of participatory public culture. Crowdsourcing initiatives further contribute to a more open, connected, and smart cultural heritage. However, activities in crowdsourcing remain heterogeneous, particularly compared to fields like natural sciences. Existing empirical evidence highlights the administrative efficacy of managing digital cultural heritage, its impact on collective cultural groups and identities, and its contribution to heritage research. Nevertheless, there's a scarcity of validated instruments for collecting precise data on the needs and challenges faced by professionals in digital heritage conservation, management, and dissemination. This study aims to address this gap by validating the DICHIE instrument for use in focus groups in Castile and Leon, Spain.

The introduction extensively reviews the existing literature on the digitization of cultural heritage, highlighting the roles of the European Commission, crowdsourcing initiatives, and the use of ICTs in heritage education and preservation. It cites numerous studies demonstrating the potential and challenges of digital approaches to cultural heritage, underscoring the need for validated instruments to assess the needs and challenges faced by professionals in the field. The review emphasizes the importance of public participation and inter-professional collaboration in the successful digitization and utilization of cultural heritage resources. Studies on gamification, participatory cultural behavior, and the challenges related to budget, sustainability and intellectual property are discussed as a basis for the DICHIE instrument.

The study employs a quantitative-descriptive methodology, focusing on instrumental psychometric studies of reliability and content validity. The DICHIE instrument, comprising 15 key questions across 5 dimensions (Access and use; Potentialities; Participative cultural behavior; Memory and collective identities; Obstacles and limitations), was developed based on related investigations (Table 1). Twenty experts in heritage conservation, education, and cultural management participated in the validation process using intentional sampling (Table 2). The experts evaluated the clarity, internal coherence, bias, language, and precision of the questions in two rounds. In the first round, each professional group independently evaluated the questions. Based on the results, questions that didn't meet the evaluation criteria were revised. In the second round, the revised questions were re-evaluated. Kendall's concordance coefficient (W), Fleiss's Kappa, and Krippendorff's α were used to assess inter-judge agreement and inter-observer reliability. Aiken's V coefficient evaluated the content relevance of each question. Bangdiwala's weighted coefficients of concordance (BWN) assessed global theoretical adjustment and intra-group agreement. Kruskal-Wallis H test examined inter-professional differences, with Epsilon-squared coefficient calculating effect size and Games-Howell post-hoc test identifying specific inter-group differences. The study adhered to ethical standards, obtaining informed consent from all participants, ensuring data confidentiality and anonymity in accordance with data protection laws.

The validity analysis revealed high levels of inter-judge agreement across both evaluation rounds, supported by strong concordance indices (W = 0.693 and 0.706, p = 0.000). Satisfactory general scores (M ≥ 3) were obtained across all specific criteria, except for questions 5 (in the first round) relating to clarity and coherence. After rewording, question 5 achieved optimal values (Table 4). Aiken's V coefficient confirmed these improvements. Fleiss' Kappa coefficient and Krippendorff's α indicated excellent inter-judge agreement (κ = 0.712 and 0.725, α = 0.859 and 0.878, p = 0.000, respectively) (Tables 5 and 6). Bangdiwala weighted coefficients of concordance showed excellent intra-professional agreement (BWN ≥ 0.913), reflecting the adequacy of the questions regarding clarity, coherence, bias, language, and precision (Table 7, Fig. 1). Statistically significant inter-group differences were found in the application of specific criteria to certain questions (Table 8). Moderate effect sizes confirmed these differences. Cultural educators and managers generally assigned lower scores than heritage conservationists (Table 8). These differences highlight potential discussion points for focus groups.

The DICHIE instrument demonstrates high validity and reliability, addressing a gap in validated qualitative data-collection instruments for assessing the needs and challenges in digital cultural heritage. The instrument's comprehensive approach evaluates management, conservation, and educational aspects of digital heritage, contributing to the existing body of knowledge on heritage education. The study's findings highlight the importance of inter-professional collaboration and public participation in the digitization and utilization of cultural heritage. The identified inter-group differences offer valuable insights for guiding focus group discussions and collaborative efforts among professionals in heritage conservation, education, and management.

The DICHIE instrument provides a robust tool for gathering qualitative data on the needs and challenges of digitizing cultural heritage. Its strong psychometric properties ensure reliable and valid data collection. Future research could explore the instrument's application in other cultural contexts and investigate its effectiveness in facilitating collaborative initiatives among professionals and the public.

The study's sample is limited to experts in Castile and Leon, Spain, potentially limiting the generalizability of findings. The focus on expert opinions does not capture the perspectives of the broader public. The instrument's effectiveness in diverse cultural settings needs further investigation. Future research could also explore potential cultural biases in question interpretation.