The new horizon of human-computer communication and the changing nature of language education.
This report examines Generative AI (GenAI) from a perspective of language education and language test preparation. While there is already a growing body of research on GenAI and language learning, the role of prompting, understood as a novel form of human-computer communication, has been neglected. To address this gap, we carried out a study in two parts.
In the first part, we conducted a scoping review that focused on how prompting GenAI language models can support English language learning and assessment.
The review identified three application scenarios for prompts: text generation, test item generation and automated assessment. For each scenario, we examined how prompts were constructed and how they could be replicated. In addition to these scenarios, the review also highlighted the emerging importance of 'meta prompts' which are distinct from user-oriented prompts in that they operate in the back-end of AI models and are not visible or modifiable. We also found that effective prompts can certainly increase the sophistication and precision of human-AI interactions, but the outputs of these interactions still display limitations in terms of contextual awareness, bias, reliability and performance consistency.
In the second part, we conducted qualitative fieldwork to understand how prompting as a communicative practice is emerging in a real context of language learning and language test preparation. To this end, we carried out interviews and observations in a language school located in a large Australian city. We found that both teachers and students engage in highly contextual forms of sensemaking that influence informal theories about GenAI use and prompting. We also found evidence of a form of prompting amongst students that we termed 'tactical': unsophisticated but reflecting pragmatic and subjective priorities.
In our conclusion, we reflect on the significance of prompting as pragmatic communication and suggest some implications and future research directions.