Translation quality assessment practices of faculty members of colleges of languages and translation in Arab countries: an exploratory study

The paper situates translation as a multifaceted cognitive, social, crosslinguistic, and cross-cultural activity and argues that improving translation education requires robust Translation Quality Assessment (TQA) methods. Prior research highlights persistent challenges: lack of consensus on defining translation quality, variability in assessment procedures and criteria, limited objectivity, and the absence of generally accepted competence structures. Instructors’ actual TQA practices, especially in Arab countries, remain under-examined. This study documents current General Assessment (GA) and TQA practices among translation instructors in Arab colleges to illuminate assessment literacy, theoretical underpinnings, and instructional alignment. The research questions are: (1) What GA practices are adopted by faculty members of colleges of Languages and Translation in Arab countries? (2) What specialised TQA practices are adopted by these faculty members? (3) Is there a correlation between GA and TQA practices?

The literature underscores the centrality and difficulty of TQA. Quality in translation is contested, reflecting divergent theoretical positions and expectations. Assessment is often subjective, with ongoing efforts to increase objectivity via rubrics, error analysis, and structured criteria, though some argue subjectivity is inherent. Additional concerns include non-standardized terminology, varied procedures across theoretical approaches, and insufficiently defined competence frameworks. Empirical studies show mixed practices: e.g., Waddington (2001) found sizeable use of holistic (subjective) methods among Canadian/European teachers, with others using error analysis or combined approaches; Firoozkoohi et al. (2012) found inconsistency in criteria and low student awareness. Calls have been made for multidimensional and more objective approaches, while recognizing the complexities of completely eliminating subjectivity. These gaps justify investigating instructors’ concrete GA and TQA practices in the Arab higher-education context.

Design: A three-part online, self-explanatory survey was administered to faculty teaching translation at colleges of Languages and Translation in Arab countries. - Part 1 (Demographics): gender, job title, academic field, linguistic specialization, university, years of experience, and prior GA/TQA education and training. - Part 2 (GA practices): Modified Approaches to Classroom Assessment Inventory (ACAI; DeLuca et al., 2016), 25 Likert items (1–5) covering assessment purposes (assessment of/for/as learning), processes (design, scoring, communication), fairness (standardized/equitable/differentiated), and theory (reliability, validity, mixed). ACAI dimensions and priorities are summarized in Table 1 of the paper. - Part 3 (TQA Practices Inventory; TQAPI): A newly designed, comprehensive 31-item Likert instrument targeting translation-specific practices across themes: variation by course, rubric use, role of errors, focus of assessment (product vs subskills), fairness, methods (e.g., exams, assignments, portfolios, peer/self-assessment), and objectivity (including views on subjectivity and criteria-based evaluation). Full item list provided in Table 2. Participants: N=98 faculty members from 16 universities. Fields of specialization included translation/translation studies (36.7%), applied linguistics (25.5%), among others. Linguistic specializations predominantly English (58.2%) and Arabic (17.3%), with some multilingual specializations. Procedures: The survey was piloted with four translation faculty (three from Saudi universities and one from Kuwait), then distributed widely via email, social media, and personal networks. Study objectives and informed consent were provided. Data analysis: Reliability assessed via Cronbach’s alpha; internal consistency confirmed. Normality assessed via Kolmogorov–Smirnov. Descriptive statistics (frequencies, percentages, means, SDs) summarized GA and TQA practices. Inferential tests included independent samples t-tests (e.g., gender), one-way ANOVA (job title, major, linguistic specialization), LSD post hoc where relevant, chi-square tests, and Pearson correlation to examine the relationship between GA and TQA indices. Reliability results: ACAI (GA) alpha=0.910 (standardized=0.915); TQAPI alpha=0.876 (standardized=0.880), indicating high internal consistency.

- Education/training background: 45.9% had a university course in GA; 60.2% had professional development in GA. For TQA-specific training, 35.7% had a university course and 27.6% had a training course, indicating a relative gap in TQA-focused preparation. - GA practices: Overall endorsement was high; item means ranged approximately 3.80–4.54. Most endorsed included communication and formative practices (e.g., providing timely feedback; using varied formative assessment; aligning tasks with objectives; using varied summative types). Lower-end items involved fairness dimensions (standardized/equitable) and some aspects of assessment-of-learning. - TQA practices: Overall endorsement was high; item means ranged approximately 2.98–4.46. Most endorsed themes were: • Role of errors: classifying errors and explaining solutions (mean ≈ 4.46). • Assessment methods: translating full texts (into Arabic; into source language) and use of midterm/final exams (means ≈ 4.21–4.33 for full-text translation; ≈ 4.24 for exams). • Objectivity: TQA should be objective and based on specific, clear criteria (mean ≈ 4.26). Less endorsed items: • Assessment methods: fill-in-the-blank (mean ≈ 2.99), true/false and pop quizzes also relatively low. • Subjectivity: endorsement of comprehensive and subjective (non-objective) assessment was low (mean ≈ 3.06). • Rubrics: not seen as the only sufficient means (mean ≈ 3.29); standardized rubrics for all courses less favored (mean ≈ 3.07). Many favored instructor-selected rubrics tailored to course objectives (mean ≈ 4.21) and reported rubric awareness (mean ≈ 3.56). - Correlation between GA and TQA: Pearson r=0.555, p<0.01, indicating a significant positive association—faculty who endorse robust GA practices tend also to endorse robust TQA practices. - Group differences: • Gender: No significant difference in GA (t=0.257, p=0.797). Significant difference in TQA (t=2.77, p=0.040) in favor of male faculty. • Job title: No significant differences for GA (F=0.564, p=0.784) or TQA (F=0.108, p=0.998). • Major: No significant differences for GA (F=0.485, p=0.694) or TQA (F=0.445, p=0.721). • Linguistic specialization: No significant differences for GA (F=0.263, p=0.983) or TQA (F=0.758, p=0.656). - Reliability: High internal consistency for GA and TQA scales (alphas ≈ 0.91 and 0.88, respectively).

The findings address the research questions by documenting that translation instructors in Arab colleges broadly value and implement GA and TQA practices. GA emphasizes communication of assessment results, formative assessment, and alignment with learning objectives, whereas equitable and standardized fairness practices are less emphasized. For TQA, instructors prioritize error analysis, objective criteria, and full-text translation tasks, reflecting a drive toward more structured, criterion-referenced evaluation while recognizing the complexity of translation assessment. The significant positive correlation between GA and TQA suggests a coherent assessment literacy across general pedagogy and translation-specific practices. Rubrics are viewed as important and reliable but not exclusive tools; instructors favor flexibility to tailor rubrics to course content and objectives rather than employing a single standardized rubric across all courses. This aligns with the multidimensional and context-sensitive nature of translation tasks. The endorsement of objectivity indicates an aspiration to reduce subjectivity through clear criteria, though the literature recognizes that some subjectivity is unavoidable. Group comparisons indicate largely uniform practices across ranks, majors, and linguistic specializations, with a noted gender-related difference in TQA favoring male faculty. This homogeneity may reflect shared institutional cultures and unified assessment practices. Overall, the study underscores the need for enhanced TQA literacy and professional development, especially in designing effective rubrics, diversifying assessment methods, and operationalizing objective criteria without oversimplifying complex translation competencies.

This study provides the first large-scale empirical documentation of GA and TQA practices among translation faculty in Arab colleges of Languages and Translation. It shows high endorsement of both GA and TQA, a prioritization of error-based analysis, objective criteria, and full-text translation tasks, and a preference for course-tailored rubrics rather than one-size-fits-all models. A significant positive correlation between GA and TQA suggests aligned assessment orientations. The study contributes a new instrument (TQAPI) for examining TQA practices and highlights a training gap in TQA-specific preparation. Practical implications include developing targeted professional development for rubric construction, objective criteria design, and diversified assessment methods. Future research should further validate and refine the TQAPI, conduct comparative studies across contexts and disciplines, investigate the impact of specific TQA approaches on student outcomes, and explore objective–subjective balances with triangulated data (e.g., classroom observations, graded artifacts) beyond self-report.

- Sample size was relatively small and purposively focused on translation instructors, limiting generalizability. - Data are based on self-reported practices and beliefs, which may be subject to bias. - The TQAPI instrument, while promising, was piloted in this study and requires further development and validation.