Test-takers’ performance appraisals, appraisal calibration, state-trait strategy use, and state-trait IELTS listening difficulty in a simulated IELTS Listening test

This study investigates the nature of test-takers’ appraisal confidence and its accuracy (calibration), reported trait and state strategy use and IELTS Listening difficulty levels in a simulated IELTS Listening test.

Appraisal calibration denotes a perfect relationship between appraisal confidence in test performance success and actual performance outcome. Calibration indicates an individual’s monitoring accuracy. The study aims to examine four aspects theoretically related to IELTS Listening test scores: 

1. Test-takers’ trait (meaning: generally perceived) and state (meaning: context-specific) cognitive and metacognitive strategy use for IELTS Listening tests; 
2. Test-takers’ calibration of appraisal confidence for each test question (meaning: single-case confidence) and for entire test sections (meaning: relative-frequency confidence); 
3. Trait and state test difficulty perception in IELTS Listening tests; and 
4. Test difficulty and test-takers’ ability as key factors affecting the above variables.

The study recruited 376 non-English speaking background (NESB) international students in Sydney, Australia. Quantitative data analysis techniques including Rasch Item Response Theory, Pearson Product-Moment correlations, t-tests, analysis of variance (ANOVA), and structural equation modelling (SEM) were used.

It was found that test-takers were miscalibrated in their performance appraisals, exhibiting a tendency to be overconfident across the four test sections. Their appraisal calibration scores were found to be worst for very difficult questions. Gender and academic success variables were also examined as factors affecting test-takers’ calibration. The SEM analysis conducted suggests that there are complex structural relationships among test-takers’ appraisal confidence, calibration, trait and state cognitive and metacognitive strategy use, IELTS Listening difficulty, and IELTS Listening performance.

The study has advanced our knowledge of strategic processes, including appraisal calibration and strategy use that affect IELTS Listening test performance. The outcomes of the study can inform IELTS by providing empirical evidence of the reasons for test score variation among different success levels. Recommendations for future research are discussed.