Exploring the connection between deep learning and learning assessments: a cross-disciplinary engineering education perspective

Deep learning, characterized by a focus on understanding and meaning-making, is contrasted with surface learning, which emphasizes memorization. Research indicates that deep learning leads to better learning outcomes and knowledge retention. Employers increasingly seek engineers with deep learning capabilities, including systems thinking, innovation, and adaptability. While the impact of assessment methods on student learning is acknowledged, the specific relationship between assessment systems and deep learning factors in multidisciplinary engineering education remains under-researched. This study aims to bridge this gap by investigating the factors associated with deep learning and the relationships between different assessment systems and those factors. The study uses a conceptual model and a structured questionnaire to examine these relationships in a sample of Queensland University of Technology (QUT) engineering students. The results will inform the design of more effective assessment strategies that promote deep learning in engineering education.

The literature review examines the concept of deep learning, contrasting it with surface learning and highlighting its importance in engineering education. It explores factors influencing deep learning, focusing on the learning environment (student-teacher relationships, resource availability, assessment effectiveness, real-world applications) and course content/design (key concepts, learning opportunities, relevance to engineering applications). The review also discusses various assessment methods, including summative (e.g., final exams) and formative (e.g., in-class problem-solving) assessments, and their potential impact on deep learning. The authors review various studies supporting the importance of engagement, creating relevant learning experiences, and incorporating real-world applications to foster deep learning. The literature suggests a positive correlation between innovative learning environments and deep learning, the effectiveness of group learning, and the importance of aligning assessments with learning objectives.

A structured questionnaire was developed based on an extensive literature review, encompassing items related to deep learning factors (learning environment, course content/design) and assessment perceptions (summative and formative assessments). The questionnaire was distributed to 600 QUT multidisciplinary engineering students (2nd, 3rd, and final year), with 243 completed responses received (40% response rate). The data were analyzed using SPSS and SEM. Exploratory factor analysis (EFA) was performed to identify underlying factors within the data related to deep learning and assessment. Structural equation modeling (SEM) was then employed to test the hypothesized relationships between assessment types (summative and formative) and deep learning factors (learning environment and course content/design). The EFA assessed the suitability of the data for factor analysis using the Kaiser-Meyer-Olkin (KMO) measure and Bartlett's test of sphericity. Principal component analysis with varimax rotation, an eigenvalue of 1, and factor loadings greater than 0.65 were used. Internal consistency reliability was assessed using Cronbach's alpha. The SEM analysis used AMOS version 25 to test the hypothesized relationships. Goodness-of-fit indices were used to evaluate the model's fit to the data. The hypotheses testing is discussed based on the standardized regression weight (β) greater than 0.2 and p-value less than 0.05.

The EFA revealed two factors influencing deep learning: learning environment and course content/design. Two factors representing assessment methods were also identified: summative and formative assessments. SEM analysis revealed significant positive relationships between both summative and formative assessment and both learning environment and course content and design. Specifically, summative assessment showed a stronger influence on the learning environment (β=0.36, p<0.001), while formative assessment showed a slightly stronger effect on course content/design (β=0.28, p=0.002). All four hypothesized relationships were supported, indicating that both summative and formative assessments significantly impact deep learning through their influence on learning environment and course content/design. The findings support the argument that integrating both summative and formative assessment methods is crucial for fostering deep learning.

The findings support the importance of considering both learning environment and course content/design when aiming to promote deep learning. The study confirms previous research highlighting the impact of assessment on student learning, but extends this understanding by showing the distinct yet complementary roles of summative and formative assessments in influencing deep learning. The significant influence of summative assessment on the learning environment suggests students adapt their learning strategies based on the end-of-semester evaluation, motivating engagement with available resources and course materials. Similarly, the significant impact of formative assessment on course content/design indicates that in-class problem-solving assessments encourage active participation in group learning and a deeper engagement with course concepts. The study emphasizes the need for a balanced approach to assessment, integrating both formative and summative methods to maximize their positive impact on student learning and deep learning outcomes. The results suggest that assessment design is a key pedagogical tool for influencing student learning.

This study provides empirical support for the interconnectedness of assessment and deep learning in engineering education. The findings demonstrate that both summative and formative assessments are significant factors influencing deep learning through their effects on learning environment and course content/design. Engineering educators should strategically design assessments to foster deep learning by incorporating both summative and formative approaches. Future research could explore the impact of different assessment types on various learning outcomes and deep learning dimensions. Investigating diverse student populations and expanding the study to different educational contexts would further enhance the generalizability of these findings.

The study's limitations include the reliance on self-reported data through a questionnaire survey, which may be subject to response bias. The sample was limited to QUT engineering students, restricting the generalizability of the findings to other universities or disciplines. Common method bias (CMB), despite attempts to mitigate it through anonymity and confidentiality, remains a potential concern in survey-based research. Future research could address these limitations through experimental designs, larger and more diverse samples, and the use of statistical techniques to control for CMB.