Can an AI Agent Replace Human Examiners in High-Stakes Interactive Speaking Tests? A Debate

Abstract

Generative artificial intelligence (GenAI) is advancing at a remarkable speed, and its promise in transforming current practice in language assessment has been articulated by applied linguistics researchers. An emerging application of GenAI is to integrate the technology into Spoken Dialogue Systems (SDSs) to simulate human interlocutors for the purpose of speaking practice or assessment. Despite rapid technological advances, the issue of whether an AI agent can replace a human examiner in one-on-one, high-stakes interactive speaking tests remains contentious. Building on a lively academic debate on this topic at the 2025 Language Testing Research Colloquium (LTRC) in Bangkok, this Viewpoint presents arguments both for and against this proposition in terms of construct theory, practicality, washback, fairness, ethical uses of AI, and so forth.

Keywords

AI agent fairness generative artificial intelligence human examiner interactive speaking test interactional competence spoken dialogue systems validity washback

Get full access to this article

View all access options for this article.

References

Association of Language Testers in Europe. (2020). ALTE principles of good practice. ALTE. https://www.alte.org/Materials

Bachman

L. F.

(1991). What does language testing have to offer? TESOL Quarterly, 25(4), 671–704. https://doi.org/10.2307/3587082

Brown

(2003). Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20(1), 1–25. https://doi.org/10.1191/0265532203lt242oa

Burton

J. D.

(2024). Evaluating the impact of nonverbal behavior on language ability ratings. Language Testing, 41(4), 729–758. https://doi.org/10.1177/02655322241255709

Chapelle

C. A.

(2024). Open generative AI changes a lot, but not everything. The Modern Language Journal, 108(2), 534–540. https://doi.org/10.1111/modl.12927

Chapelle

C. A.

(2025). Generative AI as game changer: Implications for language education. System, 132, 103672. https://doi.org/10.1016/j.system.2025.103672

Davis

(2021). Rater and interlocutor training. In Fulcher

Harding

(Eds.), The Routledge handbook of language testing (pp. 322–338). Routledge.

Fulcher

(2003). Testing second language speaking. Pearson Longman.

Galaczi

(2010). Face-to-face and computer-based assessment of speaking: Challenges and opportunities. In Araújo

(Ed.), Proceedings of the Computer-Based Assessment (CBA) of Foreign Language Speaking Skills (pp. 29–51). European Union.

10.

Galaczi

Pastorino-Campos

(2025). Ethical AI for language assessment: Principles, considerations, and emerging tensions. Annual Review of Applied Linguistics, 45, 294–314. https://doi.org/10.1017/S0267190525100081

11.

Galaczi

Taylor

(2018). Interactional competence: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15(3), 219–236. https://doi.org/10.1080/15434303.2018.1453816

12.

Ginther

(2020). Assessment of speaking. In Chapelle

C. A.

(Ed.), The encyclopedia of applied linguistics (pp. 1–7). John Wiley. https://doi.org/10.1002/9781405198431.wbeal0052.pub2

13.

Gokturk

Chukharev

(2024). Exploring the potential of a spoken dialog system-delivered paired discussion task for assessing interactional competence. Language Assessment Quarterly, 21(1), 60–99. https://doi.org/10.1080/15434303.2023.2289173

14.

Harding

(2025). Utopian and dystopian visions: Steering a course for the responsible use of artificial intelligence (AI) in language testing and assessment. Language Testing, 42(4), 561–575. https://doi.org/10.1177/02655322251350717

15.

Suvorov

(2026). Interactive videos in multimodal listening assessments: Examining language learners’ perspectives. TESOL Quarterly. Advance online publication. https://doi.org/10.1002/tesq.70099

16.

Holstein

Doroudi

(2022). Equity and artificial intelligence in education. In Holmes

Porayska-Pomsta

(Eds.), The ethics of artificial intelligence in education: Practices, challenges, and debates (pp. 151–173). Routledge. https://doi.org/10.4324/9780429329067-9

17.

Karatay

Glasson

Mullooly

(2025). AI-generated voices and accent diversity: Towards fairer listening assessment [Paper Presentation]. Paper presented at the 45th Language Testing Forum, Southampton, UK.

18.

Karatay

(2022). Development and validation of spoken dialog system-based oral communication tasks in an ESP context (Publication No. 29165842) [Doctoral dissertation, Iowa State University]. ProQuest Dissertations Publishing. https://doi.org/10.31274/td-20240329-228

19.

Karatay

Chapelle

C. A.

(this issue). Evaluating the validity of a spoken dialog system-mediated role-play test for assessing second language oral communication. Language Testing.

20.

Karatay

(2025). Exploring the potential of conversational AI for assessing second language oral proficiency. TESOL Quarterly, 59, S220–S250. https://doi.org/10.1002/tesq.70003

21.

Lazaraton

(1996). Interlocutor support in oral proficiency interviews: The case of CASE. Language Testing, 13(2), 151–172. https://doi.org/10.1177/026553229601300202

22.

Litman

Strik

Lim

G. S.

(2018). Speech technologies and the assessment of second language speaking: Approaches, challenges, and opportunities. Language Assessment Quarterly, 15(3), 294–309. https://doi.org/10.1080/15434303.2018.1472265

23.

Lomas

J. D.

van der Maden

Bandyopadhyay

Lion

Patel

Jain

Litowsky

Xue

Desmet

(2024). Evaluating the alignment of AI with human emotions. Advanced Design Research, 2(2), 88–97. https://doi.org/10.1016/j.ijadr.2024.10.002

24.

Machová

Szabóova

Paralič

Mičko

(2023). Detection of emotion by text analysis using machine learning. Frontiers in Psychology, 14, Article 190326. https://doi.org/10.3389/fpsyg.2023.1190326

25.

Mislevy

R. J.

Chapelle

C. A.

Chung

Y.-R.

(2008). Options for adaptivity in computer-assisted language learning and assessment. In Chapelle

C. A.

Chung

Y.-R.

(Eds.), Towards adaptive CALL: Natural language processing for diagnostic language assessment (pp. 9–24). Iowa State University. https://apling.engl.iastate.edu/wp-content/uploads/sites/221/2015/05/5thTSLL2007_proceedings.pdf

26.

Nakano

Takeuchi

Yoshikawa

Matsuyama

Komatani

(2025). Dialogue systems engineering: A survey and future directions. arXiv [Preprint]. https://doi.org/10.48550/arXiv.2508.02279

27.

Norton

(2013). Performing identities in speaking tests: Co-construction revisited. Language Assessment Quarterly, 10(3), 309–330. https://doi.org/10.1080/15434303.2013.769549

28.

Ockey

G. J.

Chukharev-Hudilainen

(2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. https://doi.org/10.1093/applin/amaa067

29.

Pepe

Divekar

Timpe-Laughlin

Sydorenko

Dombi

(2025). Error and recovery analysis of intent-based and LLM-based SDS. In Proceedings of the10th Workshop on Speech and Language Technology in Education (pp. 86–90). Radboud University. https://doi.org/10.21437/SLaTE.2025-18

30.

Qian

Lange

Evanini

(2020). Automatic speech recognition for automated speech scoring. In Zechner

Evanini

(Eds.), Automated speaking assessment: Using language technologies to score spontaneous speech (Vol. 3, pp. 61–74). Routledge. https://doi.org/10.4324/9781315165103

31.

Raina

McGhee

Knill

Gales

(2024). Muting whisper: A universal acoustic adversarial attack on speech foundation models. arXiv [Preprint]. https://doi.org/10.48550/arXiv.2405.06134

32.

Roever

Ikeda

(2022). What scores from monologic speaking tests can(not) tell us about interactional competence. Language Testing, 39(1), 7–29. https://doi.org/10.1177/02655322211003332

33.

Roever

Kasper

(2018). Speaking in turns and sequences: Interactional competence as a target construct in testing speaking. Language Testing, 35(3), 331–355. https://doi.org/10.1177/0265532218758128

34.

Saeki

Takatsu

Kurata

Suzuki

Eguchi

Matsuura

Takizawa

Yoshikawa

Matsuyama

(2024). InteLLA: Intelligent language learning assistant for assessing language proficiency through interviews and roleplays. In Demberg

Ultes

Inoue

Mehri

Howcroft

Komatani

(Eds.), Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and dialogue (pp. 385–399). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.sigdial-1.34

35.

Suvorov

(2022). Visuals in the assessment and testing of second language listening: A methodological synthesis. International Journal of Listening, 36(2), 80–99. https://doi.org/10.1080/10904018.2021.1941028

36.

Timpe-Laughlin

Divekar

Sydorenko

Dombi

(2025). Intent-based versus GPT-based conversational agents: Benefits and challenges for practicing and assessing oral interaction. TESOL Quarterly, 59, S117–S149. https://doi.org/10.1002/tesq.3409

37.

Voss

Cushing

S. T.

Ockey

G. J.

Yan

(2023). The use of assistive technologies including Generative AI by test takers in language assessment: A debate of theory and practice. Language Assessment Quarterly, 20(4–5), 520–532. https://doi.org/10.1080/15434303.2023.2288256

38.

Weir

C. J.

Vidaković

(2013). The measurement of listening ability 1913–2012. In Weir

C. J.

Vidaković

Galaczi

E. D.

(Eds.), Measured constructs: A history of Cambridge English language examinations 1913–2012 (pp. 347–419). Cambridge University Press.

39.

Zhu

Liu

Wang

Shi

Xiong

Xing

Niu

Peng

(2026). The Impact of chatbots on adolescent mental health development: A comprehensive literature review. Journal of Multidisciplinary Healthcare, 19, 579872. https://doi.org/10.2147/JMDH.S579872

40.

(2023). Advancing language assessment with AI and ML–Leaning into AI is inevitable, but can theory keep up? Language Assessment Quarterly, 20(4–5), 357–376. https://doi.org/10.1080/15434303.2023.2291488