The proliferation of social media platforms has created a valuable, yet challenging, resource for emergency information management (EIM). User-generated content (UGC) on platforms like Twitter and Facebook provides real-time information crucial for effective emergency response, including details on damage severity, rescue needs, and missing persons. However, the sheer volume and heterogeneous quality of this UGC make extracting truly useful information a significant hurdle, particularly relying solely on machine learning techniques. Pure machine learning approaches often lack the contextual understanding and expert knowledge necessary for accurate and reliable EI extraction. This study addresses this limitation by proposing a novel approach: a machine learning and rule-based integration method (MRIM). This method combines the strengths of machine learning algorithms, which can process vast amounts of data efficiently, with the incorporation of expert knowledge through rule-based classification. The study investigates the following key questions: 1) Can integrating machine learning and rule-based classification improve EI extraction from social media? 2) What characteristics of UGC influence the performance of the integrated method? 3) How can social media UGCs be better utilized in EIM? The research focuses on the specific context of natural disasters, utilizing microblog data from the devastating July 2021 Zhengzhou rainstorm in China as a case study. The objective is to demonstrate the feasibility and effectiveness of MRIM in extracting useful EI and to offer practical recommendations for emergency management agencies (EMAs).

Existing research highlights the significant role of social media in disseminating and obtaining emergency information. The public utilizes social media to share real-time updates, express emotions, and seek assistance, while EMAs leverage these platforms for official announcements and information gathering. Studies show the effectiveness of social media during various emergencies, from earthquakes to hurricanes, showcasing its value in resource allocation and improving relief efficiency. However, the challenge lies in the quality and authenticity of UGCs. The heterogeneous nature of user contributions, coupled with the lack of information verification on most platforms, necessitates efficient methods for extracting reliable and useful EI. While machine learning techniques have been employed for text mining in EIM, they often suffer from limitations in accuracy and reliability due to their sole reliance on algorithms without integrating expert knowledge. Rule-based classification methods, on the other hand, leverage expert experience to establish clear rules for information classification, but they struggle with the scale and complexity of social media data. This gap in the literature motivates the development of an integrated approach that combines machine learning and rule-based methods to address the limitations of each individual approach.

This study employs a novel machine learning and rule-based integration method (MRIM) to classify microblogs from Sina Weibo, China's largest microblogging platform, related to the July 2021 Zhengzhou rainstorm. The methodology involves three primary stages: 1) Data Collection and Preprocessing: Microblogs under the hashtag "#Henan rainstorm mutual assistance #" were collected using a Python crawler. Data cleaning and preprocessing steps included removing duplicates, handling missing values, and standardizing data formats. 2) Feature Extraction and Classification: The Support Vector Machine (SVM) algorithm was chosen for its effectiveness in short text classification. TF-IDF (Term Frequency-Inverse Document Frequency) was used for feature extraction, focusing on unigrams. A rule-based classification system was developed based on expert knowledge, encompassing content, communication, and user characteristics. Six judgment dimensions were identified and weighted using the Delphi method with three experts to score the usefulness of each microblog. 3) Classification Result Integration: The results from the SVM and rule-based classifications were integrated using a conservative strategy: a microblog is classified as useful only if either method identifies it as useful. This approach prioritizes identifying all useful information, minimizing the risk of overlooking crucial EI. Comparative experiments were conducted against pure machine learning (SVM alone) and rule-based methods (rule-based alone) to assess the performance of MRIM. The performance was evaluated using precision, recall, and F-measure. Furthermore, the influence of various microblog characteristics, including word count, presence of exact address and contact information, and user attention (likes, comments, shares), on the classification results were analyzed to understand factors affecting the effectiveness of MRIM.

The MRIM significantly outperformed both the pure SVM and pure rule-based methods in extracting useful emergency information. The F-measure for MRIM was 0.831, compared to 0.714 for SVM alone and 0.750 for the rule-based method alone. The improved recall rate (0.777 for MRIM) indicates a greater ability to detect useful EI without sacrificing accuracy. Experiments comparing different integration strategies demonstrated the superiority of the conservative approach adopted by MRIM over weighted scoring methods. The analysis of microblog characteristics revealed several key findings: 1) The number of words in a microblog is positively correlated with classification performance, with longer microblogs providing more information for the algorithms to process. 2) The presence of exact addresses and contact information significantly improves classification accuracy. However, inconsistencies or informal writing styles in address information reduced the effectiveness of both the SVM and MRIM. 3) User attention, measured by the sum of likes, comments, and shares, showed a complex relationship with classification performance. While higher attention generally correlated with better performance, very high attention microblogs (often from professional media) were less likely to contain directly useful EI for emergency rescue and were often misclassified. The misclassification analysis identified two main reasons for errors: 1) Microblogs that simply copied content from professional media were often misclassified as useful, despite lacking direct rescue value. 2) Missing feature data, especially incomplete addresses, hindered the SVM’s performance and affected MRIM's overall results. Microblogs with informal language and limited word counts also contributed to misclassification.

The findings strongly support the hypothesis that integrating machine learning and rule-based methods enhances the extraction of useful emergency information from social media. The MRIM's superior performance demonstrates the value of combining the efficiency of machine learning with the contextual understanding provided by expert-defined rules. The impact of microblog characteristics highlights the importance of considering data quality and the context of information sources. The conservative integration strategy employed by MRIM proved effective in minimizing false negatives, prioritizing the identification of all potentially useful information in time-critical emergency situations. The limitations of keyword-based approaches, particularly with incomplete or informally written data, are underscored by the findings. This study makes a significant contribution to the field of EIM by providing a more robust and effective method for extracting actionable information from social media during emergencies.

This study successfully demonstrates the effectiveness of MRIM, a novel method integrating machine learning and rule-based classification, for extracting useful emergency information from social media. MRIM significantly outperforms traditional methods, highlighting the value of combining algorithmic efficiency with expert knowledge. The analysis of microblog characteristics provides valuable insights into data quality and source reliability. Future research could explore the application of MRIM to other types of emergencies, adapt the method to different social media platforms, and refine the rule-based system by incorporating a more extensive expert panel and a more comprehensive set of rules.

The study's limitations include the focus on a single type of emergency (rainstorm) and a specific social media platform (Sina Weibo). The rules developed in this study might not be directly transferable to other emergencies or platforms. Additionally, the use of a relatively small number of experts in the Delphi method may limit the generalizability of the rule-based component. Future research should address these limitations by expanding the scope of the study to encompass a wider range of emergencies and social media platforms.