Recent technological advances offer test developers the opportunity to develop new assessment tasks which closely replicate behaviour in real-world domains such as academic lectures, thus achieving greater ecological and cognitive validity. However, developing new task types that demand the integration of auditory and textual information relies on understanding the relationship between what is said and what is written on the slides, and how students integrate these two streams of information.
With a dual focus, this study first examined the relationship between the lecturer's speech and the textual information on the lecture slides before investigating the processes students use to integrate these two streams of information to develop an understanding of the lecture content.
In Phase 1, five lecture recordings were collected from universities across the UK. Following Hallewell and Crook (2019), the lecturer’s speech was segmented and mapped to slide-text units to form discourse units. The discourse units were coded according to the role that the lecturer's speech performed in relation to the slide text.
In Phase 2, the most obvious speech/slide variable—the extent to which the information delivered verbally also appears in the slide text—was manipulated to assess the effect on participants’ processes. Two lecture excerpts were selected from Phase 1 and two versions of the slides were created for each lecture clip: 1) topic headings plus a short text outlining the main teaching points, and 2) topic headings only. Using a counter-balanced presentation, eye-tracking and stimulated recall were used to investigate participants' perceptual processes as they read the slides and listened to the lecture excerpts.
Phase 1 resulted in a taxonomy of discourse relations and Phase 2 reported the factors which impacted participants’ ability to develop an understanding of the lecture content. The implications for teaching and assessing academic listening are discussed.