On Microsoft Translator’s Performance in English-Persian speech-to-text Translation: Recognizing Translation Errors and Identifying the Source of Errors

Document Type : research article


Department of Foreign Languages, Faculty of Literature and Humanities, Shahid Bahonar University of Kerman, Kerman, Iran


Over the past decades, the language industry has benefited from computer-aided tools; although technology has not affected interpreting to the same extent as translation, some improvements have been made in that profession as well. The desire to avoid cognitive saturation has increased interest in computer-assisted tools and speech translation systems among interpreters. However, to better understand the function of these systems, it is necessary to study their performance, identify possible errors, and evaluate output data quality. Despite worldwide interest in detecting the role of technology in interpreting, the area of computer-assisted interpreting has been quite under-researched in Iran. Therefore, in an attempt to fill this gap, this study sought to evaluate Microsoft Translator speech translation performance. Since Microsoft Translator is a free and easily accessible translation tool. The researchers aimed to identify translation errors as well as recognizing the possible source of errors. To that end, corpora of political hearings addressed at United Nation sessions, their speech-to-text translations by Microsoft Translator, and a reference translation were created and analyzed. To find answers to the first research question, Microsoft Translator errors were detected and categorized based on the component responsible for generating the errors. The second question concerned the probable causes of errors. The findings showed that Internet access, time delay, manual function of the microphone, and speaking features could lead to translation errors. The findings of this research can be a starting point for future research in the field of computer-assisted interpreting.


Main Subjects

Al-Khanji, R., El-Shiyab, S., & Hussein, R. (2000). On the use of compensatory strategies in simultaneous interpretation. Meta, 45(3), 548-557.  Retrieved from https://doi.org/10.7202/001873ar. Accesed on.
Almahasees, Z. M. (2018). Assessment of Google and Microsoft Bing translation of journalistic texts. International Journal of Languages, Literature and Linguistics, 4(3), 231-235. Retrieved from https://doi.org/10.18178/IJLLL.2018.4.3.178. Accesed on.  
Biagini, G. (2016). Printed glossary and electronic glossary in simultaneous interpretation: A comparative study [Unpublished doctoral dissertation]. Universita degli studi di Trieste. Retrieved from http://dx.doi.org/10.1075/intp.15.1.04jia.  Accesed on.  
Costa, A., Ling, W., Luís, T., Correia, R., Coheur, L., (2015). A linguistically motivated taxonomy for Machine Translation error analysis. Machine Translation. 29.  127-161. Retrieved from https://doi.org/10.1007/s10590-015-9169-0. Accesed on.
Errattahi, R., El Hannani, A., & Ouahmane, H. (2018). Automatic speech recognition errors detection and correction: A review. Procedia Computer Science, 128, 32-37. Retrieved from https://doi.org/10.1016/j.procs.2018.03.005. Accesed on.
European Master's in Translation (2017). Competence Framework. Directorate General for Translation of the European Commission. Retrieved from https://commission.europa.eu/system/files/2018-02/emt_competence_fwk_2017_en_web.pdf. Accesed on.
Fantinuoli, C. (2017). Computer-assisted preparation in conference interpreting. Translation & Interpreting, 9(2), 24-37. Retrieved from https://doi.org/10.12807/ti.109202.2017.a02. Accesed on.
Fantinuoli, C. (2018). Interpreting and technology: The upcoming technological turn. In C. Fantinuoli (Ed.), Interpreting and technology (pp. 1–12). Language Science Press. Retrieved from https://doi.org/10.5281/zenodo.1493289.  Accesed on.
Frederking, R., Rudnicky, A., Hogan, C., & Lenzo, K. (2000). Interactive speech translation in the diplomat project. Machine Translation, 15(1-2), 27-42.‌ Retrieved from https://doi.org/10.1023/A:1011172330853. Accesed on.
Fujita, T., Neubig, G., Sakti, S., Toda, T., & Nakamura, S. (2013). Simple, lexicalized choice of translation timing for simultaneous speech translation. In INTERSPEECH (pp. 3487- 3491). Retrieved from https://doi.org/10.21437/INTERSPEECH.2013-615. Accesed on.
Hamon, O., Fügen, C., Mostefa, D., Arranz, V., Kolss, M., Waibel, A., & Choukri, K. (2009, March). End-to-end evaluation in simultaneous translation. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009) (pp. 345-353). Retrieved from https://doi.org/10.5555/1609067.1609105.  Accesed on.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174. Retrieved from https://doi.org/10.2307/2529310. Accesed on.
Leeson, L. (2005). Making the effort in simultaneous interpreting: Some considerations for signed language interpreters. In T. Janzen (Ed.), Topics in signed language interpreting: Theory and practice (pp. 51-65). John Benjamins. Retrieved from https://doi.org/10.1075/btl.63.07lee. Accesed on.
Mackintosh, J. (1983). Relay interpretation: An exploratory study [Unpublished master thesis]. University of London.
McCarthy, M., & Carter, R. (2001). Size isn't everything: Spoken English, corpus, and the classroom. Tesol Quarterly, 35(2), 337-340. Retrieved from https://doi.org/10.2307/3587654.  Accesed on.
Nakamura, S., Markov, K., Nakaiwa, H., Kikui, G. I., Kawai, H., Jitsuhiro, T., ... & Yamamoto, S. (2006). The ATR multilingual speech-to-speech translation system. IEEE Transactions on Audio, Speech, and Language Processing, 14(2), 365-376. Retrieved from https://doi.org/10.1109/TSA.2005.860774. Accesed on.
Prandi, B. (2020). The use of CAI tools in interpreter training: Where are we now and where do we go from here? inTRAlinea Special Issue: Technology in Interpreter Education and Practice. Retrieved from http://www.intralinea.org/specials/article/2512. Accesed on.
Roy, C., & Metzger, M. (2014). Researching signed language interpreting research through a sociolinguistic lens. The International Journal of Translation and Interpreting Research, 6(1), 158-176. https://doi.org/ti.106201.2014.a09
Samir, A., & Tabatabaee-Yazdi, M. (2020). Translation quality assessment rubric: A Rasch model-based validation. International Journal of Language Testing, 10(2), 101-128. Retrieved from https://www.ijlt.ir/article_118019.html. Accesed on .
Schultz, T., Black, A. W., Vogel, S., & Woszczyna, M. (2006). Flexible speech translation systems. IEEE Transactions on Audio, Speech, and Language Processing, 14(2), 403- 411. Retrieved from https://doi.org/10.1109/TSA.2005.860768.  Accesed on.
Seeber, K. G. (2007). Thinking outside the cube: Modeling language processing tasks in a multiple resource paradigm. In Eighth Annual Conference of the International Speech Communication Association (pp. 1382-1385). Retrieved from http://dx.doi.org/10.21437/Interspeech.2007-21.   Accesed on.
Seeber, K. G. (2011). Cognitive load in simultaneous interpreting: Existing theories – new models. Interpreting, 13(2). 176–204. Retrieved from https://doi.org/10.1075/intp.13.2.02see. Accesed on.
Seligman, M. (2000). Nine issues in speech translation. Machine Translation15, 149-186. Retrieved from http://dx.doi.org/10.1023/A:1011180928513. Accesed on.
Singh, P., & Singh, A. (2014). A text to speech (TTS) system with English to Punjabi conversion. arXiv preprint arXiv:1411.3561. Retrieved from https://doi.org/10.48550/arXiv.1411.3561.  Accesed on.
Wickens, C. D. (1984). Processing resources in attention. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 63–102). Academic Press.
Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical issues in ergonomics science, 3(2). 159–177. Retrieved from https://psycnet.apa.org/doi/10.1080/14639220210123806.  Accesed on.
محمدی، ع. م.(1401) . تحلیلی بر راهبردهای مترجم شفاهی همزمان ایرانی بر اساس نظریه معادل های ترجمه مطالعه گفتمان نماهای استنباطی و توالی. پژوهشهای زبانشناختی در زبانهای خارجی، 12 (1)، 132-152.
ولی‌پور،  ع. (1400) .داده‌های بنیادین استراتژیک زبان‌شناسی، کلید اکمال متقابل تکنولوژی و هوش مصنوعی در حیطه ترجمه‌ ماشینی. پژوهشهای زبانشناختی در زبانهای خارجی، 11 (3)، 541-551.