The widespread adoption of online interactive learning environments has revolutionized education, offering flexibility and enhanced communication. However, these environments can also lead to inefficient learning, negative emotions, and high assessment failure rates. This is due to several factors: (1) the inability of the environment to meet learner demands, resulting in overwhelming information; (2) a lack of clear temporal sequences and correlations between learning contents; and (3) learners' cognitive limitations in processing massive information. This study aims to address these challenges by developing an interpretable early warning system that provides timely recommendations and interventions to improve learning outcomes. The study focuses on analyzing large-scale learning behavior data to identify patterns and relationships that can inform effective early warning recommendations. The goal is to enhance learners' self-organization skills and create a more adaptive and interpretable learning experience.

Existing research on interpretable methods for learning analytics primarily focuses on interpretable models (Model Agnostic and Model Specific) and interpretable tools (LIME, SHAP, PDP, etc.). While Model Agnostic methods offer broader applicability, Model Specific tools are often more readily implemented. Interpretable recommendation mechanisms are also explored, aiming to personalize services and reduce information overload. However, current research lacks effective solutions for large-scale learning behavior data and the integration of temporal sequences in the learning process. This study bridges this gap by proposing a novel approach that integrates interpretable methods, tools, and recommendation mechanisms within a knowledge graph framework.

The proposed methodology involves several key steps: (1) constructing an interpretable knowledge graph of learning behavior, representing entities (learners, learning contents, concepts) and relationships; (2) designing a deep-neural network model (DNNA) based on this knowledge graph to process and interpret massive learning behavior data; (3) decomposing the deep-neural network's weight vectors into interpretable feature vectors; (4) visualizing the output features and interpreting the relationships between features and temporal sequences; (5) calculating recognition scores and confidence levels for temporal sequences; (6) evaluating the reliability of the vector decomposition; (7) constructing a knowledge graph of temporal sequences; (8) calculating the similarity of temporal sequences; (9) mining discriminant features; and (10) testing learner credibility using MSE and TF-IDF. The DNNA algorithm is compared against several other methods: Logistic Regression (LR), Factorization Machines (FM), DNNFM, DNNCross, AutoInt, and AFN. The evaluation metrics include AUC, RI, F1 score, and MTL-Gain. Data from an AI-enabled online learning platform (1.3PB) was used, addressing challenges of data sparsity and uncertainty in learning behavior by introducing more characteristics and analyzing knowledge graphs of learning content.

The DNNA algorithm outperformed other methods in terms of AUC (0.9035), RI, F1 score (0.8101), and MTL-Gain. The interpretable feature recommendation test showed that DNNA provided better results compared to other methods. Analysis of the negative samples using different sampling methods (R-method, FR-method, L-method, FS-method) highlighted the superior performance of DNNA's adaptive similarity calculation and recommendation. The study identified key temporal sequence intervals for both successful and unsuccessful learners across different learning content categories. For successful learners, the key temporal sequences show significance in the knowledge graph, while unsuccessful learners lack such significance. Three key knowledge graphs (I, II, III) were constructed, revealing critical learning paths and early warning areas for successful and unsuccessful learners. The analysis showed strong correlations between feature classes and the importance of concept classes (especially "key points of LC" and "difficulties of LC") in predicting learning success. For learners who passed, an "OR" relationship between key temporal sequences was observed, allowing for selective intervention; while for those who failed, an "AND" relationship indicated the need for continuous tracking and intervention across all relevant temporal sequences.

The findings address the research question by demonstrating the effectiveness of the DNNA-based approach in providing interpretable early warning recommendations for interactive learning. The superior performance of DNNA compared to other methods highlights its ability to effectively analyze complex learning behavior data and identify key patterns. The identification of critical temporal sequences and the distinction between successful and unsuccessful learners' knowledge graph significance offer valuable insights for educators to design targeted interventions and improve learning outcomes. The results emphasize the importance of considering learning content concept classes, learning behavior feature classes, and effective temporal sequences in designing an early warning system.

This study introduces a novel DNNA-based approach for generating interpretable early warning recommendations in interactive learning environments. The method effectively leverages learning behavior knowledge graphs and addresses challenges related to data sparsity and uncertainty. The superior performance demonstrated by DNNA in various experiments highlights its practical value. Future research could explore more sophisticated knowledge graph representations, integrate learner-specific characteristics, and develop more adaptive intervention strategies based on real-time learning behavior data.

The study relied on data from a specific AI-enabled online learning platform, limiting the generalizability of the findings to other learning environments. The interpretation of the knowledge graphs and temporal sequences may be subject to researcher bias, and further validation across different datasets is needed. The study focused on specific learning behavior features and concepts; including a broader range of features might provide more comprehensive insights.