Digital text comprehension: An eye-movement study of text and multimedia integration

Digital text comprehension: An eye-movement study of text and multimedia integration

First Author: Gal Ben-Yehudah -- The Open University of Israel
Additional authors/chairs: 
Orly Azulai; Yael Gilutz; Yoram Eshet-Alkalai
Keywords: Text Comprehension, Eye movements, Digital literacy
Abstract / Summary: 

Digital texts are often displayed with multimedia elements; however, it is not clear how strategies for integrating information from different sources (i.e., integration behavior) are related to text comprehension. Purpose: We used eye-tracking technology to investigate the integration of textual and visual-multimedial information. The research questions were (1) is there a relationship between integration behavior and digital text comprehension (experiment 1); (2) does an explicit cue to move from the textual to the multimedial elements improve comprehension (experiment 2)? Method: In both experiments young adults read an expository digital text that included a scientific illustration and an interactive simulation. The number of eye-movements between text and multimedia areas (i.e., transfers) served as a proxy for the amount of integration. Results: In experiment 1 (n=39), a median-split analysis of the transfers showed that in the high-transfer group 70% had above median comprehension scores, compared to only 26% in the low-transfer group. In experiment 2 (n=13), transfers were induced by visual cues embedded in the text, which prompted participants to gaze at the multimedia elements. There were significantly (p<.005) more transfers in experiment 2 than experiment 1; however, this did not improve comprehension scores (68.2% and 69.6%, respectively). Interestingly, high-transfer participants in experiment 1 had higher comprehension scores (72.5% success) than high-transfer participants in experiment 2 (63.6% success). Conclusions: An interpretation of these findings is that adult readers have formed stable learning strategies that fit them; therefore, an explicit cue to gaze at the multimedial material interferes with their personal learning process.