On Microsoft Translator’s Performance in English-Persian speech-to-text Translation: Recognizing Translation Errors and Identifying the Source of Errors

Document Type : research article

Authors

Department of Foreign Languages, Faculty of Literature and Humanities, Shahid Bahonar University of Kerman, Kerman, Iran

Abstract

Over the past decades, the language industry has benefited from computer-aided tools; although technology has not affected interpreting to the same extent as translation, some improvements have been made in that profession as well. The desire to avoid cognitive saturation has increased interest in computer-assisted tools and speech translation systems among interpreters. However, to better understand the function of these systems, it is necessary to study their performance, identify possible errors, and evaluate output data quality. Despite worldwide interest in detecting the role of technology in interpreting, the area of computer-assisted interpreting has been quite under-researched in Iran. Therefore, in an attempt to fill this gap, this study sought to evaluate Microsoft Translator speech translation performance. Since Microsoft Translator is a free and easily accessible translation tool. The researchers aimed to identify translation errors as well as recognizing the possible source of errors. To that end, corpora of political hearings addressed at United Nation sessions, their speech-to-text translations by Microsoft Translator, and a reference translation were created and analyzed. To find answers to the first research question, Microsoft Translator errors were detected and categorized based on the component responsible for generating the errors. The second question concerned the probable causes of errors. The findings showed that Internet access, time delay, manual function of the microphone, and speaking features could lead to translation errors. The findings of this research can be a starting point for future research in the field of computer-assisted interpreting.

Keywords

Main Subjects

References

Al-Khanji, R., El-Shiyab, S., & Hussein, R. (2000). On the use of compensatory strategies in simultaneous interpretation. Meta, 45(3), 548-557.  Retrieved from https://doi.org/10.7202/001873ar. Accesed on.
Almahasees, Z. M. (2018). Assessment of Google and Microsoft Bing translation of journalistic texts. International Journal of Languages, Literature and Linguistics, 4(3), 231-235. Retrieved from https://doi.org/10.18178/IJLLL.2018.4.3.178. Accesed on.  
Biagini, G. (2016). Printed glossary and electronic glossary in simultaneous interpretation: A comparative study [Unpublished doctoral dissertation]. Universita degli studi di Trieste. Retrieved from http://dx.doi.org/10.1075/intp.15.1.04jia.  Accesed on.  
 
Costa, A., Ling, W., Luís, T., Correia, R., Coheur, L., (2015). A linguistically motivated taxonomy for Machine Translation error analysis. Machine Translation. 29.  127-161. Retrieved from https://doi.org/10.1007/s10590-015-9169-0. Accesed on.
Errattahi, R., El Hannani, A., & Ouahmane, H. (2018). Automatic speech recognition errors detection and correction: A review. Procedia Computer Science, 128, 32-37. Retrieved from https://doi.org/10.1016/j.procs.2018.03.005. Accesed on.
European Master's in Translation (2017). Competence Framework. Directorate General for Translation of the European Commission. Retrieved from https://commission.europa.eu/system/files/2018-02/emt_competence_fwk_2017_en_web.pdf. Accesed on.
Fantinuoli, C. (2017). Computer-assisted preparation in conference interpreting. Translation & Interpreting, 9(2), 24-37. Retrieved from https://doi.org/10.12807/ti.109202.2017.a02. Accesed on.
Fantinuoli, C. (2018). Interpreting and technology: The upcoming technological turn. In C. Fantinuoli (Ed.), Interpreting and technology (pp. 1–12). Language Science Press. Retrieved from https://doi.org/10.5281/zenodo.1493289.  Accesed on.
Frederking, R., Rudnicky, A., Hogan, C., & Lenzo, K. (2000). Interactive speech translation in the diplomat project. Machine Translation, 15(1-2), 27-42.‌ Retrieved from https://doi.org/10.1023/A:1011172330853. Accesed on.
Fujita, T., Neubig, G., Sakti, S., Toda, T., & Nakamura, S. (2013). Simple, lexicalized choice of translation timing for simultaneous speech translation. In INTERSPEECH (pp. 3487- 3491). Retrieved from https://doi.org/10.21437/INTERSPEECH.2013-615. Accesed on.
Gile, D. (1995). Basic concepts and models for interpreter and translator training. John Benjamins.
Hale, S., & Napier, J. (2013). Research methods in interpreting: A practical resource. A&C Black.
Hamon, O., Fügen, C., Mostefa, D., Arranz, V., Kolss, M., Waibel, A., & Choukri, K. (2009, March). End-to-end evaluation in simultaneous translation. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009) (pp. 345-353). Retrieved from https://doi.org/10.5555/1609067.1609105.  Accesed on.
Kahneman, D. (1973). Attention and effort. NJ: Prentice-Hall.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174. Retrieved from https://doi.org/10.2307/2529310. Accesed on.
Leeson, L. (2005). Making the effort in simultaneous interpreting: Some considerations for signed language interpreters. In T. Janzen (Ed.), Topics in signed language interpreting: Theory and practice (pp. 51-65). John Benjamins. Retrieved from https://doi.org/10.1075/btl.63.07lee. Accesed on.
Mackintosh, J. (1983). Relay interpretation: An exploratory study [Unpublished master thesis]. University of London.
McCarthy, M., & Carter, R. (2001). Size isn't everything: Spoken English, corpus, and the classroom. Tesol Quarterly, 35(2), 337-340. Retrieved from https://doi.org/10.2307/3587654.  Accesed on.
Müller, M., Fünfer, S., Stüker, S., & Waibel, A. (2016, May). Evaluation of the KIT Lecture Translation System. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16) (pp. 1856-1861).
Nakamura, S., Markov, K., Nakaiwa, H., Kikui, G. I., Kawai, H., Jitsuhiro, T., ... & Yamamoto, S. (2006). The ATR multilingual speech-to-speech translation system. IEEE Transactions on Audio, Speech, and Language Processing, 14(2), 365-376. Retrieved from https://doi.org/10.1109/TSA.2005.860774. Accesed on.
Pio, S. (2003). The relation between ST delivery rate and quality in simultaneous interpretation. The Interpreters' Newsletter, 12, 69-100.
Prandi, B. (2015). The use of CAI tools in interpreters’ training: A pilot study. In Proceedings of the Translating and the Computer 37 Conference (pp. 48-57).
Prandi, B. (2018). An exploratory study on CAI tools in simultaneous interpreting: Theoretical framework and stimulus validation. In C. Fantinuoli (Ed.), Interpreting and technology (pp. 29-59). Language Science Press.
Prandi, B. (2020). The use of CAI tools in interpreter training: Where are we now and where do we go from here? inTRAlinea Special Issue: Technology in Interpreter Education and Practice. Retrieved from http://www.intralinea.org/specials/article/2512. Accesed on.
Reppen, R. (2010). Using corpora in the language classroom. Cambridge University Press.
Roy, C., & Metzger, M. (2014). Researching signed language interpreting research through a sociolinguistic lens. The International Journal of Translation and Interpreting Research, 6(1), 158-176. https://doi.org/ti.106201.2014.a09
Ruiz, N., & Federico, M. (2014). Assessing the impact of speech recognition errors on machine translation quality. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas: MT Researchers Track (pp. 261-274).
Russo, M., Bendazzoli, C., & Defrancq, B. (Eds.). (2018). Making way in corpus-based interpreting studies. Springer.
Samir, A., & Tabatabaee-Yazdi, M. (2020). Translation quality assessment rubric: A Rasch model-based validation. International Journal of Language Testing, 10(2), 101-128. Retrieved from https://www.ijlt.ir/article_118019.html. Accesed on .
Schultz, T., Black, A. W., Vogel, S., & Woszczyna, M. (2006). Flexible speech translation systems. IEEE Transactions on Audio, Speech, and Language Processing, 14(2), 403- 411. Retrieved from https://doi.org/10.1109/TSA.2005.860768.  Accesed on.
Seeber, K. G. (2007). Thinking outside the cube: Modeling language processing tasks in a multiple resource paradigm. In Eighth Annual Conference of the International Speech Communication Association (pp. 1382-1385). Retrieved from http://dx.doi.org/10.21437/Interspeech.2007-21.   Accesed on.
Seeber, K. G. (2011). Cognitive load in simultaneous interpreting: Existing theories – new models. Interpreting, 13(2). 176–204. Retrieved from https://doi.org/10.1075/intp.13.2.02see. Accesed on.
Seligman, M. (2000). Nine issues in speech translation. Machine Translation, 15, 149-186. Retrieved from http://dx.doi.org/10.1023/A:1011180928513. Accesed on.
Singh, P., & Singh, A. (2014). A text to speech (TTS) system with English to Punjabi conversion. arXiv preprint arXiv:1411.3561. Retrieved from https://doi.org/10.48550/arXiv.1411.3561.  Accesed on.
Sridhar, V. K. R., Chen, J., Bangalore, S., Ljolje, A., & Chengalvarayan, R. (2013). Segmentation strategies for streaming speech translation. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 230-238).
Straniero Sergio, F., & Falbo, C. (2012). Breaking ground in corpus-based interpreting studies. Peter Lang.
Tognini-Bonelli, E. (2001). Corpus linguistics at work. Corpus Linguistics at Work, 1-236.
Tripepi Winteringham, S. (2010). The usefulness of ICTs in interpreting practice. The Interpreters' Newsletter, 15, 87–99.
Waibel, A., Badran, A., Black, A. W., Frederking, R., Gates, D., Lavie, A., ... & Zhang, J. (2003). Speechalator: two-way speech-to-speech translation in your hand. In Companion Volume of the Proceedings of HLT-NAACL 2003-Demonstrations (pp. 29-30).
Wickens, C. D. (1984). Processing resources in attention. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 63–102). Academic Press.
Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical issues in ergonomics science, 3(2). 159–177. Retrieved from https://psycnet.apa.org/doi/10.1080/14639220210123806.  Accesed on.
Williams, J., & Chesterman, A. (2002). A beginner's guide to doing research in translation studies. Jerome Publishing.
محمدی، ع. م.(1401) . تحلیلی بر راهبردهای مترجم شفاهی همزمان ایرانی بر اساس نظریه معادل های ترجمه مطالعه گفتمان نماهای استنباطی و توالی. پژوهشهای زبانشناختی در زبانهای خارجی، 12 (1)، 132-152.
https://doi.org/10.22059/jflr.2021.329419.880.
ولی‌پور،  ع. (1400) .داده‌های بنیادین استراتژیک زبان‌شناسی، کلید اکمال متقابل تکنولوژی و هوش مصنوعی در حیطه ترجمه‌ ماشینی. پژوهشهای زبانشناختی در زبانهای خارجی، 11 (3)، 541-551.
https://doi.org/10.22059/jflr.2021.331469.900.
Journal of Foreign Language Research
Volume 13, Issue 3
November 2023
Pages 439-456

Files

Share

How to cite

Statistics