Performance and perception: machine translation post-editing in Chinese-English news translation by novice translators

Globalization has exponentially increased the demand for translation, particularly in news translation, which requires capturing cultural nuances and subtleties. While speed is crucial in news translation, the advancement of machine translation (MT) offers a potential solution to accelerate this process. Although MT has shown promise in overcoming communication barriers, its performance in news translation, especially for novice translators, remains uncertain. This study aims to compare the performance and perceptions of translation learners using MT post-editing versus manual translation in Chinese-English news translation. The goal is to assess MT's applicability in translator training programs and contribute to developing curricula that optimize MT's use in news translation. The study addresses three research questions: 1) How well does Google Translate perform in Chinese-English news translation linguistically and culturally? 2) How do novice translators perform in post-editing compared to manual translation? 3) How do novice translators perceive using Google Translate in Chinese-English news translation?

News translation is a complex field within translation studies, often analyzed using product-oriented and process-oriented approaches. Product-oriented approaches compare translated news across different languages and cultures. Process-oriented approaches focus on the translation process itself. The challenges of news translation include linguistic styles, structures, and the need to convey meaning across languages and cultures. Machine translation (MT), while improving, still generates errors requiring post-editing to reach acceptable quality. Different studies have categorized MT errors, including lexical, syntactic, and punctuation issues. Post-editing can be classified as light or full, with full post-editing aiming for professional translation quality. The cognitive workload of post-editing involves temporal, technical, and cognitive efforts. Prior research indicates that experience level influences translator attitudes towards MT. Inexperienced translators may focus on lexical issues, while professionals prioritize coherence and style. The use of MT in news translation is increasingly explored, but the specific challenges for novice translators remain under-investigated.

This study used a mixed-methods approach combining qualitative and quantitative methods. Participants were 24 third-year Chinese-native-speaking translation learners with about six months of training. A pretest questionnaire collected demographic information. Two comparable excerpts from a 2019 report on China's climate change policies were selected as testing materials. Text 1 was for manual translation, and Text 2 for post-editing using Google Translate. Text complexity was analyzed using the Common Text Analysis Platform (CTAP). The experiment involved participants first manually translating Text 1, then post-editing Text 2. Screen recordings monitored the process. A post-test questionnaire assessed workload (using NASA-TLX) and self-assessed performance. Two raters evaluated translation quality using a system adapted from Daems et al. (2013), considering acceptability and adequacy. Data was analyzed using SPSS 17.0, with Wilcoxon signed-ranks test for comparisons due to the small sample size.

Analysis of Google Translate's output revealed frequent lexical and syntactic errors. Mistranslations were common, particularly with noun phrases and conjunctions. The MT struggled with semantic nuances (e.g., mistaking ‘basic state policy’ for ‘basic national policy’), sentence structure (omission or misunderstanding of conjunctions in complex sentences), and tense/modality (inappropriately using present tense instead of present perfect). Regarding quality, the mean score for manual translation (Text 1) was 84.20 (SD = 4.66) and 84.58 (SD = 2.39) for post-edited Text 2. No significant quality difference was found, although post-editing showed a slightly higher score. However, novice translators' error identification and correction rates were low: lexical error identification was 61.29%, but correction was only 19.35%; syntactic error identification was 38.71%, with correction at 29.03%; grammatical and style errors had even lower identification and correction rates. Self-assessment showed that learners rated machine translation quality as acceptable (70.83% rated it as average, 29.17% as above average), but they lacked confidence in their post-editing abilities (20.83% rated their performance as poor, 54.17% as average, 25% as good). 96% of participants preferred MT post-editing to manual translation. In manual translation, lexical issues were the biggest challenge (33%), followed by semantic expression, terminology, background information, and structural issues. In post-editing, participants focused more on structural cohesion (92%), compared to other aspects. Workload analysis showed significantly lower time and physical demands in post-editing than manual translation; mental demand and frustration were also marginally lower. Screen recordings supported these findings, showing approximately 30 minutes for manual translation and 20 minutes for post-editing.

The study's findings indicate that while MT struggles with semantic nuances, structural complexity, and tense/modality in Chinese-English news translation, its use in post-editing by novice translators can improve efficiency without significantly compromising quality. Novice translators face significant lexical and semantic challenges in manual translation. The lower error identification and correction rates highlight the need for specific training in MT post-editing. The preference for MT post-editing and its demonstrable reduction in workload suggest its value in translator training. However, MT still has limitations in cultural translation.

This study demonstrates that MT post-editing offers a viable approach for novice translators in Chinese-English news translation, enhancing efficiency while maintaining comparable quality. The low error correction rate emphasizes the need for specialized training in MT post-editing within translator education programs. Future research should involve larger, more diverse samples and explore the effectiveness of various post-editing training methods.

The study's findings should be interpreted cautiously due to the small sample size (24 participants), which limits the generalizability of the results. The selection of excerpts from a government report may not fully represent the diversity of news texts. The reliance on Google Translate may not reflect the capabilities of all MT systems. Future studies should address these limitations by employing larger and more diverse samples and exploring different MT systems.