High-quality explanations are crucial for effective learning, fostering understanding and generalization of concepts beyond rote memorization. However, many online learning platforms lack such explanations due to limited instructor time and resources. This often leads to learners gaming the system by repeatedly trying answers without grasping the underlying principles. While intelligent tutoring systems sometimes provide explanations, creating high-quality explanations is significantly more challenging than simply providing answers. This paper addresses the challenge of generating and improving explanations at scale by crowdsourcing the effort from learners. Learners are uniquely positioned to identify common misconceptions and knowledge gaps. AXIS addresses the unreliability of individual learner-generated explanations by using machine learning to identify and present the most helpful ones, thereby improving the learning experience.

AXIS builds upon prior research in learnersourcing and machine learning for educational applications. Learnersourcing systems, like Scribe and Crowdy, have successfully integrated learners' contributions into educational processes, enabling real-time captioning and interactive video summarization. In the realm of machine learning, reinforcement learning, particularly multi-armed bandit problems, has proven effective in educational contexts for optimizing teaching sequences and hint generation. AXIS leverages Thompson sampling, a Bayesian algorithm, to balance exploration (testing new explanations) and exploitation (using known effective explanations) for optimal explanation selection.

AXIS comprises two main components: a learnersourcing interface and an explanation selection policy. The learnersourcing interface prompts learners to generate, revise, and evaluate explanations while solving problems. It incorporates elements of self-explanation, a known effective learning strategy. The explanation selection policy, based on a multi-armed bandit algorithm (Thompson sampling), dynamically chooses which explanation to present to a new learner based on the collective ratings from previous learners. The system uses a filtering mechanism to ensure only high-quality explanations are added to the pool. The Beta distribution is employed to represent the system's beliefs about the effectiveness of each explanation, updated after each learner interaction. The policy balances exploration and exploitation by selecting explanations proportionally to their probability of being the best, given the available data and uncertainty. The system was implemented using Qualtrics for the user interface and Google Spreadsheets/Apps Script for the machine learning algorithm. A case study involved 150 Mechanical Turk participants solving four math problems. An independent randomized experiment with 524 participants assessed the quality and effectiveness of AXIS explanations compared to explanations written by an experienced instructor and those filtered out by AXIS. Participants solved problems, rated explanation helpfulness, and completed an assessment phase to measure learning gains.

The case study demonstrated AXIS's ability to collect and refine explanations over time. The randomized experiment revealed that AXIS-selected explanations were rated as significantly more helpful than discarded explanations. Specifically, learners who received AXIS explanations showed significantly improved learning outcomes compared to those who received no explanations or explanations that were filtered out by AXIS. Learners receiving AXIS explanations displayed a 12% increase in accuracy on subsequent problems compared to a 3% increase for the control group. Moreover, AXIS explanations led to a significant increase in learners' perceived ability to solve similar problems. Importantly, the learning gains achieved with AXIS-curated explanations were comparable to those obtained with explanations from an experienced instructor, indicating the efficacy of the learnersourcing approach. The results demonstrate that AXIS efficiently identifies helpful learnersourced explanations and improves learning compared to the standard practice of providing answers without explanations.

The findings support the hypothesis that crowdsourcing explanations from learners, coupled with machine learning for selection, can effectively generate high-quality explanations for online learning materials. The comparable performance of AXIS-generated explanations to those from an experienced instructor highlights the potential of this approach to address the scalability challenge in providing detailed explanations for large numbers of online problems. This approach significantly reduces the burden on instructors and offers a more sustainable solution for enhancing the learning experience at scale. The successful integration of self-explanation prompts within AXIS further underscores the importance of active learning strategies in maximizing learning outcomes.

AXIS demonstrates a successful approach to generating and improving explanations for online learning materials by leveraging learners' collective knowledge and machine learning. The results from a case study and a randomized experiment show that AXIS effectively produces high-quality explanations that enhance learning, rivaling those created by experienced instructors. Future work should explore personalizing explanations based on learner characteristics and investigating alternative reward signals beyond subjective ratings, such as performance on subsequent problems or learner persistence.

The current study primarily focused on math problems and used Mechanical Turk participants, which may limit the generalizability of the findings to other learning domains and student populations. AXIS currently presents a single best explanation to all learners, regardless of individual needs or preferences. Future research could incorporate learner profiling and contextual bandit algorithms for personalized explanation delivery. The reliance on subjective ratings as a reward signal may be affected by learners' metacognitive limitations. Further research could explore alternative reward signals, such as performance on subsequent problems or engagement measures.