This study evaluated ChatGPT's role in enhancing English-as-a-foreign-language (EFL) writing assessments using generalizability (G-) theory and qualitative feedback analysis. The reliability of holistic scores assigned by ChatGPT versions 3.5 and 4, compared to college English teachers, and the relevance of their qualitative feedback were assessed. Analysis of 30 CET-4 essays revealed that ChatGPT 3.5 had lower reliability than teachers, while ChatGPT 4 showed higher reliability. Both ChatGPT versions provided more relevant feedback than teachers, focusing equally on language, content, and organization, unlike teachers who prioritized language. ChatGPT versions 3.5 and 4 are suggested as useful AI tools for enhancing EFL writing assessments.

Junfei Li, Jinyan Huang, Wenyan Wu, Patrick B. Whipple