Online vs in-person learning in higher education: effects on student achievement and recommendations for leadership

The COVID-19 pandemic in 2020 prompted a global, rapid shift from traditional in-person instruction to distance teaching. Despite widespread adoption, the comparative effects of online versus in-person instruction on student achievement remain insufficiently examined. Prior research often reports lower outcomes for online learners compared to traditional classrooms, but many studies focus on potential impacts or perceptions rather than concrete academic outcomes such as test scores. This study addresses that gap by comparing the effectiveness of distance learning versus in-person instruction in five required freshmen courses at King Saud University (KSU), Saudi Arabia. It analyzes final exam results of 8297 freshmen who took the courses in person in 2020 and 8425 freshmen who took the same courses online in 2021. The five courses are University Skills 101 (CI_101), Entrepreneurship 101 (ENT_101), Computer Skills 101 (CT_101), Computer Skills 102 (CT_102), and Fitness and Health 101 (FAJB_101). The analyses account for confounders associated with achievement, including gender, class size, and admission scores, and consider KSU’s preparatory-year tracks (health, nursing, science, business, and humanity). The purpose is to provide a nuanced, data-driven assessment of modality effects on student achievement in higher education.

The literature comparing academic achievement in online versus in-person higher education is mixed. Several large-scale and experimental studies find traditional in-person instruction yields slightly better academic outcomes (e.g., Fischer et al., 2020; Bettinger et al., 2017; Figlio et al., 2013; Kaupp, 2012). Conversely, some studies report better outcomes for online students, including during COVID-19 disruptions (e.g., Iglesias-Pradas et al., 2021; Gonzalez et al., 2020). Other research highlights heterogeneity driven by student characteristics (e.g., conscientiousness, prior proficiency), instructor factors, and classroom environment (Chesser et al., 2020; Cacault et al., 2021; Bergstrand and Savage, 2013). The theoretical framing draws on constructivism and subject-specific pedagogy, emphasizing that effective instructional design should align with disciplinary practices and leverage the strengths of each modality. Online learning can uphold constructivist principles through interactive activities, asynchronous engagement, and technology-enhanced authenticity, while in-person environments facilitate immediate feedback and social presence. These perspectives suggest that modality effects may vary by subject matter and instructional design, supporting a nuanced evaluation rather than a one-size-fits-all conclusion. The study also notes motivations for expanding e-learning, including crisis resilience, accessibility, flexibility, and cost-effectiveness, while acknowledging challenges such as technical barriers, distractions, and reduced face-to-face interaction.

Design and setting: A retrospective comparative study was conducted at King Saud University (KSU), examining final exam performance in five mandatory freshman-level courses across two academic years: 2020 (in-person) and 2021 (online due to COVID-19). Courses included CI_101 (University Skills), ENT_101 (Entrepreneurship), CT_101 (Computer Skills 101), CT_102 (Computer Skills 102), and FAJB_101 (Fitness and Health). The KSU preparatory year comprises five tracks (health, nursing, science, business, humanity), and course content, placement, and organization were consistent across the two years. Course structure: Course credits ranged from 1 to 3 credit hours. Semester length was 15 weeks. Learning activities varied by course: CT_101 and CT_102 had 50% lecture and 50% laboratory (45 total semester hours each); ENT_101, FAJB_101, and CI_101 were 100% lecture (ENT_101 and FAJB_101, 15 hours; CI_101, 45 hours). Participants and data: The dataset comprised final exam scores for 16,722 first-year students: 8297 in-person (2020) and 8425 online (2021). In 2020, 51.5% were female and 48.5% male; in 2021, 48.5% female and 51.5% male. Enrollment totals per course were similar across years, but average class sizes increased substantially in four courses from 2020 to 2021 (e.g., ENT_101 and CT_102 nearly doubled), while CT_101 saw a reduction. Some students took up to three different courses in a year, but no student took the same exam in both years; students failing in 2020 were required to complete courses in summer and were not included. Instruments and measurement: Final exams are centrally developed by departmental committees, and exam questions were consistent across 2020 and 2021. Reliability was strong across tests (Cronbach’s alpha: CI_101 0.78 in 2020/0.84 in 2021; ENT_101 0.83/0.75; CT_101 0.81/0.86; CT_102 0.77/0.78; FAJB_101 0.84/0.79). Validity: Content validity ensured via subject matter expert review; criterion validity established via correlations between admissions scores and final test scores (r ≈ 0.37–0.56); construct validity supported by expert review and iterative improvements to test content. Variables: Outcomes were final exam scores. Covariates included gender (male=0, female=1), class size (raw numbers), and admission scores (composite of high school grades and entrance exams). Year was coded 2020=0, 2021=1. Analysis: Descriptive statistics and independent-samples t-tests compared 2020 vs 2021 by course. Multiple linear regression models then estimated differences between years while controlling for gender, class size, and admission scores. Separate models were run by course. Adjusted R-squared values indicated moderate explanatory power across courses (approx. 0.23–0.43). Ethical procedures: Institutional approval was obtained from the KSU Research Ethics Committee (No. 4/4/255639). All data were anonymized.

Descriptive and t-test results: • CI_101: Mean 2020=42.13 (SD 6.56), 2021=41.95 (SD 6.88); t=1.37, p=0.171; small decline of 0.18 points, not statistically significant. • CT_101: Mean 2020=40.59 (SD 8.16), 2021=40.57 (SD 8.35); t=0.12, p=0.901; no meaningful difference. • CT_102: Mean 2020=39.20 (SD 7.96), 2021=35.62 (SD 9.03); t=15.18, p<0.001; decline of 3.58 points (ES≈0.422). • ENT_101: Mean 2020=40.88 (SD 4.79), 2021=39.87 (SD 6.34); t=7.92, p<0.001; decline of 1.01 points (ES≈0.180). • FAJB_101: Mean 2020=42.08 (SD 6.22), 2021=41.79 (SD 6.11); t=2.45, p=0.014; decline of 0.29 points (ES≈0.047). Regression-adjusted results (controlling for gender, class size, and admission scores): • CI_101: 2021 online cohort scored higher than 2020 in-person by about +0.89 points. • FAJB_101: 2021 online cohort scored higher by about +0.56 points. • CT_101: 2021 online cohort scored substantially higher by about +5.28 points. • ENT_101: 2021 online cohort scored lower by about −0.69 points. • CT_102: No statistically significant difference between years. Additional notes: Reliability of exams was strong (Cronbach’s alpha 0.77–0.86). Admissions scores positively predicted final exam performance; models showed moderate fit (adjusted R-squared roughly 0.23–0.43 across courses). Overall, after adjustment, three courses favored online delivery, one favored in-person, and one showed no difference, indicating course-specific modality effects.

The study aimed to determine whether online learning affected student achievement relative to in-person instruction for five required freshman courses at KSU. Simple mean comparisons via t-tests suggested declines in most courses from 2020 to 2021, with CT_101 unchanged. However, multivariate regressions that controlled for gender, class size, and admissions scores revealed a more nuanced pattern: online learning in 2021 was associated with higher scores in CI_101, FAJB_101, and particularly CT_101, while in-person students performed better in ENT_101, and CT_102 showed no difference. These findings suggest modality effects are course-dependent and influenced by contextual factors such as subject matter and instructional design. The notable +5.28 point advantage for online students in CT_101 may reflect benefits of self-paced learning supported by recorded instructional materials in a foundational, conceptual course where students enter with heterogeneous prior skills. Minimal adjusted differences in CI_101, FAJB_101, ENT_101, and CT_102 imply that at introductory levels, effective communication of core concepts may mitigate modality impacts, provided instructional alignment is maintained. Collectively, results argue against blanket conclusions about online learning’s inferiority or superiority, instead highlighting the importance of aligning pedagogy with subject-specific needs and leveraging modality strengths. For leadership, this supports strategic, data-driven decisions about when to deploy online versus in-person formats to optimize learning outcomes and resource allocation.

This study contributes empirical evidence from a large sample at KSU comparing online (2021) and in-person (2020) instruction across five required freshman courses. After controlling for key covariates, online delivery improved outcomes in three courses (CI_101, FAJB_101, CT_101), in-person delivery was better for one (ENT_101), and one showed no difference (CT_102). These course-specific effects underscore that modality effectiveness depends on the subject and instructional approach. Recommendations for leadership include adopting a measured, data-driven strategy rather than a one-size-fits-all policy: expand online delivery where it demonstrably benefits learning, maintain or emphasize in-person formats where outcomes are stronger, and consider hybrid or flexible models aligned with subject-specific pedagogy. Future research should investigate mechanisms behind course-level differences, including instructional design features, assessment types, student prior proficiency and engagement, and instructor practices, to refine modality choices and enhance student success.

The study was conducted at a well-resourced major public university, which may limit generalizability to other contexts (e.g., community colleges, private institutions). Detailed data on in-class teaching practices and variation in learning activities across courses were limited, constraining deeper interpretation of why certain courses favored one modality. Additionally, although some students took multiple different courses within a year, the design precluded within-student comparisons across modalities for the same course, and students who failed courses in 2020 completed them in summer and were excluded.