Developing and Validating EFL Learners’ Perception of Machine Translation Questionnaire

Mirzaeian, Vahidreza

doi:10.22059/jflr.2020.307537.741

Developing and Validating EFL Learners’ Perception of Machine Translation Questionnaire

Document Type : research article

Author

vahidreza mirzaeian

faculty member

10.22059/jflr.2020.307537.741

Abstract

With huge improvements in the quality of Machine Translation (MT) in recent years, more interest has also been directed to its application in language learning; however, there is no standard questionnaire to assess English as a Foreign Language (EFL) learners’ perception of this technology. Therefore, this paper reports an attempt to describe the processes undertaken to validate a questionnaire developed for the first time to measure EFL students’ perception of Machine Translation. The validation processes employed were face and content validity, factor analysis informed construct validity, reliability via internal consistency using test-retest reliability as well as Cronbach’s Alpha correlation efficient. The exploratory factor analysis yielded four factors, namely, prior familiarity, degree of use, fear of detection and the importance of training among such students. The items on the questionnaire yielded factor loading ≥0.5. Reliability checking indicated that Machine Translation questionnaire was reliable. Cronbach’s Alpha obtained was 0.94 for the whole questionnaire and between 0.80-0.91 for the four subscales. Test-retest statistic examination displayed stability of the responses at two time points eight weeks apart. The final questionnaire consisted of 29 items and the psychometric analysis indicated that it was both reliable and valid. Therefore, this questioanaire can be used to gain more understanding of Machine Translation.

Keywords

References

Adel, S., Egtesadi, A., Sadeghi, F. (2019). Construction and Validation of a Critical Pedagogy Questionnaire to Assess ELT Teachers: A Mixed-Method Study. Issues in Language Teaching, 8(1), 333-375. doi: 10.22054/ilt.2020.46289.421

Alhaisoni, E., & Alhaysony, M. (2017). An Investigation of Saudi EFL University Students’ Attitudes towards the Use of Google Translate. International Journal of English Language Education, 5(1), 72-82.

Anderson, D. (1995). Machine Translation as a Tool in Second Language Learning. CALICO, 13(1), 68-97. doi:http://www.jstor.org/stable/24147744

Ashrafi Asgarabad A, Khanjani N, Samareh Fekri M, GhotbiRavandi M R. Designing and Validating The Persian Version of a Questionnaire for Estimating Exposure to Occupational Dust among Stone Carving and Stone Cutting Workers. Health Develop J. 2015; 4 (4): 277-286. URL: http://jhad.kmu.ac.ir/article-1-120-fa.html

Bahri, H., & Mahadi, T. (2016). Google Translate as a Supplementary Tool for Learning Malay: A Case Study at Universiti Sains Malaysia Advances in Language and Literary Studies, 7(3), 161-167. doi:10.7575/aiac.alls.v.7n.3p.161

Beanland, C., Scheneideer, Z., Lobiondo-Wood, G., & Haber, J. (1999). Nursing Research: methods, critical appraisal and utilisation. Sydney: Mosby.

Beikmohammadi, Maryam, Alavi, Seyyed-Mohammad, Kaivanpanah, Shiva (2020). Learning-oriented Assessment of Reading: A Mixed Methods Study of Iranian EFL University Instructors’ Perceptions and Practices. Journal of Foreign Language Research, 10 (2), 316-329. DOI: 10.22059/jflr.2019.264150.541

Bryman, A., & Cramer, D. (1999). Quantitative Data Analysis with SPSS Release 8 for Windows. A Guide for Social Scientists. London: Routledge.

Bryman, A., & Cramer, D. (2005). Quantitative Data Analysis with SPSS12 and 13. A Guide for Social Scientists: East Sussex Routledge.

Case, M. (2015). Machine Translation and the Disruption of Foreign Language Learning Activities. eLearning Papers 45, 4-16.

Cetiner, C. (2018). Analyzing the attitudes of translation students towards cat (computer-aided translation) tools. Jounral of Language and Linguistic Studies, 14(1), 153-161.

Cetiner, C., & Isiag, K. (2019). Undergraduate Level Translation Students’ Attitudes towards Machine Translation Post-Editing Training. International Journal of Languages’ Education and Teaching, 7(1), 110-120. doi:22.01.2019 20.02.2019 25.03.2019 10.18298/ijlet.3242

Concidine, J., Botti, M., & Thomas, S. (2005). Design, format, validity and reliability of multiple choice questions for use in nursing research and education. Collegian, 12(1), 19-24.

Cronbach, L., & Shavelson, R. (2004). My Current Thoughts on Coefficient Alpha and Successor Procedures Educational and Psychological Measurement, 64(3), 391-418.

Darancik, Y. (2016). The Effect of Data-Based Translation Program Used in Foreign Language Education on the Correct Use of Language. TOJET: The Turkish Online Journal of Educational Technology 15(4), 88-106.

Delaney, C. (2005). The Spirituality Scale, Development and Psychometric Testing of a Holistic Instrument to Assess the Human Spiritual Dimension. Journal of Holistic Nursing, 33(2), 145-167.

Devon, H., Block, M., Moyle-Wright, P., Ernest, D., Hayden, S., & Lazzara, D. (2007). A psychometric Toolbox for testing Validity and Reliability. Journal of Nursing scholarship, 39(2), 155-164.

Dongyun, S. (2017). Application of Post-Editing in Foreign Language Teaching: Problems and Challenges Canadian Social Science 13 (7), 1-5 doi:10.3968/9698

Dordi-nezhad F. (2015) Construction, Validation and Reliability of the Foreign Language Attitude and Motivation Questionnaire. IQBQ.; 6 (2): 107-128. URL: http://lrr.modares.ac.ir/article-14-10922-fa.html

Enkin, E., & Mejias-Bikandi, E. (2016). Using Online Translators in the Second Language Classroom: Ideas for Advanced level Spanish LACLIL, 9(1), 138-158 doi:10.5294/laclil.2016.9.1.6

Field, A. (2005). Discovering Statistics using SPSS. London: Sage.

Garcia, I., & Pena, M. (2011). Machine translation-assisted language learning: writing for beginners. Computer Assisted Language Learning, 24(5), 471-487. doi:10.1080/09588221.2011.582687

Hair, J., Anderson, R., Tatham, R., & Black, W. (1998). Multivariate Data Analysis. Upper Saddle River: Prentice-Hall.

Haladyna, T. (1999). Developing and Validating multiple-choice test items. New Jersey: Lawrence Erlbaum.

Harris, h. (2010). Machine Translations revisited: Issues and Treatment Protocol The Language Teacher, 34(3), 1-29.

Hayes, S. (2002). Acceptance, Mindfulness, and Science. American Psychological Association, 12, 101-105.

Heppner, P., Heppner, M., Lee, D., Wang, Y., & Park, H. (2006). Development and validation of a Collectivist Coping Style Inventory. Journal of Counselling Psychology, 53(1), 107-125.

Holt, A., Overgaard, D., & Engel, L. (2020). Health literacy, digital literacy and eHealth literacy in Danish nursing students at entry and graduate level: a cross sectional study. BMC Nurse, 19(22). doi:https://doi.org/10.1186/s12912-020-00418-w

Javdan, M. (2016). Construction and Standardization English learning Problems Inventory in High School. Biquarterly Journal of Cognitive Strategies in Learning, 3(5), 107-121.

Jimenez-Crespo, M. (2018). The role of translation technologies in Spanish language learning, Journal of Spanish Language Teaching. Jounral of Spanish Language Teaching, 1-13. doi:10.1080/23247797.2017.1408949

Kaiser, H. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31-36.

Keshavarz, M. (2011). Contrastive analysis and error analysis. Tehran: Rahnama Publication.

Khaleghizadeh, S., Pahlavannezhad, M., Vakilifard, A., Kamyabi Gol, A. (2020). Validation of Persian Language Learning Motivation Questionnaireas a Second Language. Journal of Teaching Persian to Speakers of Other Languages, 9(20), 25-63. doi: 10.30479/jtpsol.2020.9935.1421

Khalili, N., Vaziri-Harami, R., Jahani, Y. (2017). Construction and Validation of the Narcissism and Vulnerable Narcissism Scale in a Non-clinical Sample in Iran. Journal of Clinical Research, 2(Issue (1-2)), 23-37. doi: 10.22122/ajcr.2017.48985

Khani, R., Ghasemi, F. (2018). Development and Validation of Teacher Emotional Support Scale: a structural equation modeling approach. Journal of English Language Teaching and Learning, 10(21), 137-160.

Khatib, M., Rahimi, A. (2015). Attitudes towards English Language Norms in the Expanding Circle: Development and Validation of a new Model and Questionnaire. Journal of Teaching Language Skills, 34(1), 51-81. doi: 10.22099/jtls.2015.3234

Kheiri Langroudi, S. A, Badsar A R, Hosseini Z, Rouhi M. Validation of the Persian version of the Dundee Ready Educational Environment Measure (DREEM). Rme. 2012; 4 (2) :24-33. URL: http://rme.gums.ac.ir/article-1-133-fa.html

Kol, S., Schcolnik, M., & Spector-Cohen, E. (2018). Google Translate in Academic Writing Courses? The EUROCALL Review, 26(2), 50-57.

Lee, S. (2019). The impact of using machine translation on EFL students’ writing. Computer Assisted Language Learning. doi:10.1080/09588221.2018.1553186

Linn, R., & Gronlund, N. (2000). Measurement and assessment in teaching. In. New Jersey: Prentice Hall.

Lorenceau, A., Marec, C., & Mostafa, T. (2020). Upgrading the ICT questionnaire items in Pisa 2021. In: Organisation for Economic Co-operation and Development.

Lynn, M. (1996). Determination and quantification of content validity. Nursing Research, 35, 382-385.

Mirhosseini, Seyyed-Abdolhamid, Badri, Narges Sadat (2020). Studying the Awareness of Iranian University Students about the International Policies for the Spread of English. Journal of Foreign Language Research, 10 (2), 300-315. DOI: 10.22059/jflr.2020.292031.699

Modaresifar M, Abolfathi Momtaz Y, Delbari A. Validity and Reliability of the Persian Version of Relationship of Grandchild- Grandparents. Salmand: Iranian Journal of Ageing. 2020; 15 (1): 53-42. URL: http://salmandj.uswr.ac.ir/article-1-1702-en.html

Modarresi, G., Shams Hosseini, H. (2015). Construction, Validation and Application of CALL Knowledge Scale for EFL Learners and its Relationship with their L2 Writing Skills. Foreign Language Research Journal, 5(2), 309-333. doi: 10.22059/jflr.2015.62552

Munro, B. (2005). Statistical methods for health care research. Philadelphia: Lippincott: Williams & Wilkins.

Napitupulu, S. (2017). Analyzing Indonesian_English Abstracts Translation in View of Translation Errors by Google Translate. International Journal of English Language and Linguistics Research, 5(2), 15-23.

Nasiripour, A., Golalizadeh, A., Komeili, A., Tabibi, S., Mahmoudi, M. (2019).Designing and Validating a Questionnaire for the Health Insurance Management of the People with Mental Disorders . Journal of Medical Council of Iran. 37 (2) :102-109. URL: http://jmciri.ir/article-1-2908-fa.html

Nino, A. (2004). Recycling MT: A course on foreign language writing via MT post-editing.

Nino, A. (2008). Evaluating the use of machine translation post-editing in the foreign language class. Computer Assisted Language Learning, 21(1), 29-49. doi:http://dx.doi.org/10.1080/09588220701865482

Nordin, M., Ahamad, T., Zubairi, A., Ismail, N., Rahman, A., Trayek, F., & Ibrahim, M. (2016). Digital citizenship questionnaire. International Education Studies, 9(3), 71-80.

Nunnally, J., & Bernstein, I. (1994). Psychometric theory. New York: McGraw-Hill.

O’Neill, E. (2019). Training students to use online translators and dictionaries: The impact on second language writing scores. International Journal of Research Studies in Language Learning (IJRSLL), 8(2), 47-65.

Pilot, D., & Hunger, B. (1999). Nursing research: principals and methods. Philadelphia: Lippincott Williams & Wilkins.

Putri, G., & Ardi , H. (2015). Types of errors found in google translation: a modle of MT evaluation. Paper presented at the Proceedings of ISELT FBS Universitas Negeri Padang.

Rasafiani M, Sahaf R, Shams A, Vameghi R, Zareian H, Akrami R. Validity and Reliability of the Persian Version of the World Health Organization Quality of Life Questionnaire – the Older Adults Edition. Salmand: Iranian Journal of Ageing. 2020; 15 (1) :41-28. URL: http://salmandj.uswr.ac.ir/article-1-1622-en.html

Salehi, M., Homaee Shandiz, F., Najaf Najafi, M., Salehi, M. (2014). Development and Validation of a Questionnaire to Assess the Attitudes of Mastectomy Patients about Breast Reconstruction. The Iranian Journal of Obstetrics, Gynecology and Infertility, 16(90), 16-24. doi: 10.22038/ijogi.2014.2592

Stevens, J. (2002). Applied Multivariate Statistics for the Social Sciences. Mahwah, NJ: Lawrence Erlbaum Associates.

Sukkhwan, A. (2014). Students’ Attitudes and Behaviours towards the Use of Google Translate. (Master Thesis). Prince of Songkla University, Thailand,

Trochim, W. M. K. (2001). Research methods knowledge base (2nd ed. ed.). Cincinnati, Ohio: Atomic Dog Pub.

Tu, Z., Lu, Z., Liu, Y., Liu, X., & Li, H. (2016). Modeling Coverage for Neural Machine Translation. Paper presented at the Proceedings of ACL 2016.