Dr. LIANG Maocheng is a professor of Applied Linguistics at Beihang University. Professor Liang is currently the chairperson of the Corpus Linguistics Society of China. His research interests include Corpus Linguistics, Second Language Writing, Natural Language Processing, and Data Science. In recent years, his research focuses on Deep Learning, Distributional Semantics, and Automated Writing Evaluation. He is the author of four monographs and more than 50 journal articles, and the system architect for iWrite, an online tutoring and diagnostic system for EFL writing.
Automated Evaluation of Translation Scripts in a Large-Scale Translation Contest
Translation is a common performance task for foreign language learners. The evaluation of translation scripts, however, is not only labor-intensive and time-consuming, but also rather subjective and prone to low reliability. The study of the automated evaluation of translation is therefore of immediate practical significance.
This study draws on the state-of-the-art Doc2vec technology in NLP to construct a model for the automated evaluation of translation. The model was then used to generate a score for each of the 11,049 translation scripts collected from the Han Suyin Translation Contest. When the machine-generated scores were compared with human-generated scores, it was found that the Doc2vec-based model can produce scores with high reliability and validity. The model can efficiently identify features of good translation, and can therefore be reliably used as a second rater in large-scale translation tests.
Some of the limitations in the automated evaluation of translation are also discussed.
Key words: translation, automated evaluation, Doc2vec