Authors:
Mézière, Diane, diane.meziere@utu.fi, University of Turku
von der Malsburg, Titus, titus.von-der-malsburg@ling.uni-stuttgart.de, University of Stuttgart
Keywords: eye-tracking, reading assessment, random forests
Abstract:
Research on the relationship between eye movements and reading comprehension has shown that eye-movement measures can be used to predict performance on different reading comprehension tasks. In previous work, we used logistic regression and cross-validation to investigate the relationship between eye-tracking measures and reading comprehension and examine the usefulness of these measures in predicting comprehension scores. These models explained up to 46% of the variance in the data, with an average of 37% across tasks. However, these models are simple, and it is likely that more expressive modelling techniques can increase both the amount of variance explained and prediction accuracy. In this study, we use random forests to predict reading comprehension outcomes across comprehension tasks by re-analyzing datasets from Mézière et al. (2023, 2024). Random forests are particularly well-suited because they account for interactions between predictors, handle multicollinearity of predictors well, and make efficient use of small datasets – all of which are common challenges in typical eye-tracking datasets. We examined two datasets that include eye movement and comprehension data from 141 participants (N = 79 and 62 respectively) while they read texts according to five reading tasks: 1) the GORT-5; 2) the YARC; 3) the sentence comprehension sub-test of the WRAT-4; 4) reading for recall; and 5) reading without any additional task. Random forests were fit for each reading task to predict comprehension scores from nine canonical eye-movement measures used in previous work. We fit forests in two ways: first with data aggregated per test and participant (participant-level analysis), and then with data aggregated per test item (i.e., text-level analysis). Preliminary results from both analyses show that the most important predictors differed across reading tasks, which is in line with previous work. R² values suggest that random forests explain more variance than logistic regression (participant level: 40% vs. 37%, averaged across tasks). Explained variance was further increased to 53% in the text-level analysis. Overall, these results further support the usefulness of eye-tracking for the assessment of reading comprehension.