Authors:
Bammel, Moritz, moritz.bammel@leuphana.de, Leuphana University Lueneburg
Sanches de Oliveira, Guilherme, gui.cogsci@gmail.com, Technical University Berlin
Keywords: reading comprehension, eye movements, recurrence quantification analysis, reading time regularity
Abstract:
Previous research on assessing reading comprehension using aggregate eye movement measures (e.g., number of fixations, mean fixation duration, percentage of regressions) has yielded inconclusive findings as to which eye movement features serve as reliable predictors of reading comprehension (e.g., Mézière et al., 2023; Southwell et al., 2020; Wallot et al., 2015). Against this background, the concept of reading time regularity has been proposed, hypothesizing that more regularity in process measures capturing the temporal evolution of the reading process (e.g., recurrences in gaze steps, fixation durations, or reading times) will be predictive of proficient reading comprehension (Tschense & Wallot, 2022).
In this study, we conducted a qualitative comparison of aggregate versus process measures of eye movements during reading based on data from the Potsdam Textbook Corpus (Jäger et al., 2021, OSF). Beginners and experts in either biology or physics read short scientific texts (~160 words) taken from undergraduate-level textbooks in biology and physics, followed by comprehension questionnaires. In addition to extracting aggregate eye movement measures (total number of fixations, mean fixation duration, and percentage of regressive eye movements), we applied recurrence quantification analysis to raw gaze step data to obtain process measures quantifying regularity of the reading process.
We found that aggregate eye movement measures did not predict reading comprehension reliably, whereas recurrence measures based on gaze step data systematically co-varied with differences in comprehension proficiency between beginners and experts. However, contrary to our hypothesis, experts with high reading proficiency showed more irregular, rather than more regular, eye movement dynamics compared to beginners with poorer comprehension.
These findings demonstrate the potential of using process instead of aggregate eye movement measures to assess reading comprehension. Yet, the reading time regularity hypothesis needs to be reconsidered critically since the present results do not support its main prediction.
References:
Jäger, L. A., Kern, T., & Haller, P. (2021). Potsdam Textbook Corpus (PoTeC): Eye-tracking data from experts and non- experts reading scientific texts. Open Science Framework. https://doi.org/10.17605/OSF.IO/DN5HP
Mézière, D. C., Yu, L., Reichle, E. D., Von Der Malsburg, T., & McArthur, G. (2023). Using eye‐tracking measures to predict reading comprehension. Reading Research Quarterly, 58(3), 425-449. https://doi.org/10.1002/rrq.498
Southwell, R., Gregg, J., Bixler, R., & D’Mello, S. K. (2020). What Eye Movements Reveal About Later Comprehension of Long Connected Texts. Cognitive Science, 44(10). https://doi.org/10.1111/cogs.12905
Tschense, M., & Wallot, S. (2022). Using measures of reading time regularity (RTR) to quantify eye movement dynamics, and how they are shaped by linguistic information. Journal of Vision, 22(6), 1–21. https://doi.org/10.1167/jov.22.6.9
Wallot, S., O’Brien, B. A., Coey, C. A., & Kelty-Stephen, D. (2015). Power-law fluctuations in eye movements predict text comprehension during connected text reading. In Proceedings of the 37th Annual Meeting of the Cognitive Science Society, 2583–2588.