This summary overview reports on a systematic review of the IELTS Writing criteria and band descriptors conducted in response to ongoing monitoring of test performance, including anecdotal feedback from IELTS Writing examiners over several years. The cross-partner project was conducted between March 2019 and May 2023 and entailed multiple phases.
Phase 1 investigated quantitative and qualitative data collected via a survey questionnaire and focus groups from at least 400 IELTS Writing examiners worldwide. Data analyses from Phase 1 suggested a need to remove perceived ambiguities in some scales and band descriptors through adjustment of the existing wording.
Building on the Phase 1 findings, Phase 2 involved a team of IELTS senior examiners and researchers making targeted adjustments to the existing IELTS scales and band descriptors to prepare a draft revised version for field trialling.
The Phase 3 field trial collected quantitative scoring data from 103 examiners marking 882 responses using the adjusted band descriptors, accompanied by questionnaire and focus group feedback. Score data analyses, using Linear Mixed Effects (LME) and Multi-Faceted Rasch Measurement (MFRM) models, indicated that the adjusted scales demonstrated appropriate technical performance qualities, while examiner feedback showed that raters perceived the adjustments as a positive improvement.
Although work in Phases 1, 2 and 3 had been completed by February 2020, the onset of the Covid-19 pandemic in March 2020 made the planned implementation of the adjusted scales and band descriptors for IELTS Writing difficult at that time. Instead, the IELTS partners prioritised innovations in the existing test system which could respond to and keep pace with rapidly changing market requirements due to the pandemic (e.g., IELTS Indicator).
Phase 4 of the review project was initiated in January 2022 in preparation for operational implementation of the adjusted scales and band descriptors. Between May and July 2022, a cross-partner team of subject matter experts made further refinements to the descriptor wording and developed revised examiner training materials. The appropriate functioning of the revised scales was confirmed in a study that examined score data from 1,000 IELTS Writing responses marked twice by 100 examiners, once using the existing descriptors and then using the adjusted descriptors. Feedback from examiners was also gathered for analysis. The outcomes of the Phase 4 study showed no cause for concern and the adjusted band descriptors were introduced operationally from May 2023.