Concurrent and predictive validity of reading assessment based on eye tracking and machine learning

Concurrent and predictive validity of reading assessment based on eye tracking and machine learning

First Author: Elizabeth Meisinger -- University of Memphis
Additional authors/chairs: 
Mattias N. Benfatto; Markus Andersson; Gustaf Ö. Seimyr
Keywords: Assessment, Screening, Eye-tracking, Fluency, Efficiency
Abstract / Summary: 

Purpose: This study examined the concurrent and predictive validity of Lexplore, an innovative method for reading assessment based on eye tracking and machine learning.
Method: Lexplore was incorporated into the standard benchmark assessment battery in grade 1-6 at several elementary schools (N = 1,484 students) during the 2018–2019 school year. Fall and spring screening results from Lexplore was compared to i-Ready, words correct per minute scores (WCPM), and the California end-of-year state test (CAASPP) for grades 3-6. Concurrent and predictive validity was examined through correlational analyses, receiver operating characteristic (ROC) curve analyses, and classification accuracy statistics.
Results: Strong correlations were observed between Lexplore and i-Ready (r = .75) and WCPM scores (r = .92). The correlation between the CAASPP and Lexplore was r = .65 in both spring and fall, the results for the WCPM scores were similar. Correlations between the CAASPP and i-Ready benchmarks were r = .80 in fall and r = .85 in spring. The CAASPP result (standard met or exceeded) was used as a cut-off for predictive analysis. The area under the curve (AUC) for fall benchmarks (n = 927) were .79 for Lexplore, .81 for WCPM, and .83 for i-Ready. For spring benchmarks (n = 590), the AUC was .83 for Lexplore, .82 for WCPM, and .90 for i-Ready.
Conclusions: The results from Lexplore were very similar to WCPM and correlated well with i-Ready. The correlation between Lexplore and CAASPP was somewhat lower than i-Ready, although both tests performed similarly when predicting end-of-year reading proficiency.