Philosophy of Research in Language Testing: Investigating Papers Published in Iranian, Peer-reviewed, Domestic Journals from 2008 to 2018

Document Type : research article

Author

Department of English Language and Literature, Faculty of Letters and Humanities, Shahid Chamran University of Ahvaz, Iran.

Abstract

Research in language testing should ideally move in tandem with movements in philosophical paradigms of science. Some scholars maintain that research in language testing has been slow to catch up with paradigmatic changes in ontology and epistemology of knowledge. The current study addresses the philosophical underpinnings of research in language testing in Iran. To this aim, all language testing papers published online in Iranian domestic journals were gathered and studied. Paper titles, the abstracts, methods, and the theoretical frameworks of the studies were content analyzed. Their approach to the definition of validity and validation were also investigated. Results indicated that positivist and realist ontological assumptions underlie two thirds of the papers. Moreover, one third of papers were conducted within the constructivist philosophy and the pragmatic philosophy seems to be to a large extent absent in language testing research in Iran (only two papers). The implications for the pedagogy of language testing and assessment are further discussed.

Keywords

References

لطف‌آبادی, ح. (2007). کاستی‌های معرفت شناسی و روش شناسی در مقالات پژوهشی روانشناسی در ایران. فصلنامه حوزه و دانشگاه روش شناسی علوم انسانی, 13(51), 9-44.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2. doi:10.1207/s15434311laq0201_1
Bachman, L. F., & Palmer, A. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: Oxford University Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.
Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics: Cambridge University Press.
Brown, J. D., & Hudson, T. (1998). The alternatives in language assessment. TESOL Quarterly, 32(4), 653-675.
Cantley, I. (2017). A quantum measurement paradigm for Educational Predicates: Implications for validity in educational measurement. Educational Philosophy and Theory, 49(4), 405-421.
Chalhoub‐Deville, M., & Deville, C. (2008). Utilizing psychometric methods in assessment. In N. Hornberger & E. Shohamy (Eds.), Encyclopedia of language and education (pp. 2349-2361). NY: Springer.
Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (2011). Building a validity argument for the Test of English as a Foreign LanguageTM: Routledge.
Chappelle, C. A. (2012). Conceptions of validity. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 21-33). London: Routledge.
Cheng, L., & Fox, J. (2013). Review of doctoral research in language assessment in Canada (2006–2011). Language Teaching, 46(4), 518-544.
Corson, D. (1997). Critical realism: An emancipatory philosophy for applied linguistics? Applied Linguistics, 18(2), 166-188.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological bulletin, 52(4), 281-302.
Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25(3), 327-347.
Farhady, H., Jafarpur, A., & Birjandi, P. (1994). Testing language skills: From theory to practice. Tehran: SAMT Publications.
Foucault, M. (1977). Discipline and punishment: The birth of the prison. London: Allen Lane.
Fulcher, G. (2010). Practical language testing: Routledge.
Fulcher, G. (2013). Philosophy and language testing. In A. J. Kunnan (Ed.), The companion to language assessment (Vol. 3, pp. 1431-1451). Chichester: John Wiley & Sons.
Fulcher, G. (2015). Reexamining language testing: A philosophical and social inquiry.
Fulcher, G., & Davidson, F. (2007). Language testing and assessment. New York: Routledge
Galaczi, E., & Taylor, L. (2018). Interactional Competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly, 15(3), 219-236. doi:10.1080/15434303.2018.1453816
Hacking, I. (1999). The social construction of what? Cambridge,Mass.: Harvard university press.
Hamp-Lyons, L., & Lynch, B. K. (1998). Perspectives on validity: A historical analysis of language testing conference abstracts. In A. J. Kunnan (Ed.), Issues in Language Testing Research: Conventional Validity and Beyond (pp. 253-277). NJ: Lawrence Erlbaum, Mahwah.
Hathcoat, J. D. (2013). Validity semantics in educational and psychological assessment. Practical assessment, research & evaluation, 18(9), 1-14.
Hood, S. B. (2009). Validity in psychological testing and scientific realism. Theory & Psychology, 19(4), 451-473.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed Methods Research: A Research Paradigm Whose Time Has Come. Educational Researcher, 33(7), 14-26. doi:10.3102/0013189x033007014
Johnson, R. C., & Riazi, A. M. (2015). Accuplacer Companion in a foreign language context: An argument-based validation of both test score meaning and impact. Papers in Language Testing and Assessment, 4(1), 31-58.
Jordan, G. (2004). Theory construction in second language acquisition. Amsterdam: John Benjamins Publishing.
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73.
Knoch, U., & Chapelle, C. A. (2018). Validation of rating processes within an argument-based framework. Language Testing, 35(4), 477-499.
LaFlair, G. T., & Staples, S. (2017). Using corpus linguistics to examine the extrapolation inference in the validity argument for a high-stakes speaking assessment. Language Testing, 34(4), 451-475. doi:10.1177/0265532217713951
Lazaratan, A. (2005). Quantitative research methods. In E. Hinkle (Ed.), Handbook of Research in Second Language Teaching and Learning (pp. 209-244). NJ: Lawrence Erlbaum Associates, Publishers, Mahwah.
Lumley, T., & Brown, A. (2005). Research methods in language testing. In E. Hinkle (Ed.), Handbook of research in second language teaching and learning (pp. 857-880). NJ: Lawrence Erlbaum Associates, Publishers, Mahwah.
Lynch, B., & Shaw, P. (2005). Portfolios, Power, and Ethics. TESOL Quarterly, 39(2), 263-297. doi:10.2307/3588311
Markus, K. A., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. New York: Routledge.
Maul, A., Irribarra, D. T., & Wilson, M. (2016). On the philosophical foundations of psychological measurement. Measurement, 79, 311-320.
Maxwell, J. A., & Mittapalli, K. (2010). Realism as a stance for mixed methods research. In A. Tashakkori & C. Teddie (Eds.), Handbook of mixed methods in social & behavioral research (pp. 145-168). California: Sage.
McNamara, T., & Roever, K. (2006). Language testing: The social dimension. Oxford: Blackwell.
Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement. New York: Macmillan.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241-256. doi:10.1177/026553229601300302
Michell, J. (1990). An introduction to the logic of psychological measurement. Hillsdale, NJ: Erlbaum: Psychology Press.
Mislevy, R. J. (2018). Sociocognitive foundations of educational measurement. New York: Routledge.
Moss, P. A., Girard, B. J., & Haniford, L. C. (2006). Chapter 4: Validity in educational assessment. Review of research in education, 30(1), 109-162.
Newton, P., & Shaw, S. (2014). Validity in educational and psychological assessment. California: Sage.
Nisbett, R. (2004). The geography of thought: How Asians and Westerners think differently... and why. New York: Free Press.
Noddings, N. (2018). Philosophy of education. New York: Routledge.
Norris, S. P. (1983). The inconsistencies at the foundation of construct validation theory. New directions for program evaluation, 19, 53-74.
Packer, M. J. (2017). The science of qualitative research. Cambridge: Cambridge University Press.
Popper, K. (2014). Conjectures and refutations: The growth of scientific knowledge. New York: Routledge.
Ryan, A. G., & Aikenhead, G. S. (1992). Students' preconceptions about the epistemology of science. Science education, 76(6), 559-580.
Shohamy, E. (2014). The power of tests: A critical perspective on the uses of language tests. New York: Routledge.
Sutton, C. (1996). Beliefs about science and beliefs about language. International Journal of Science Education, 18(1), 1-18.
Toulmin, S. E. (2002). The uses of argument. Cambridge: Cambridge university press.
Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing (pp. 41-58). Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers.
Weideman, A. (2013). Applied linguistics beyond postmodernism. Acta Academica, 45(4), 236-255.
Weir, C. J. (2005). Language testing and validation: Hampshire: Palgrave McMillan.
Xiaoming, X. (2010). How do we go about investigating test fairness? Language Testing, 27(2), 147-170. doi:10.1177/0265532209349465
Journal of Foreign Language Research
Volume 9, Issue 3
January 2020
Pages 831-860

Files

Share

How to cite

Statistics