Getting to grips with poor comprehenders: stability of group membership across test conditions

Getting to grips with poor comprehenders: stability of group membership across test conditions

First Author: Hanne Trebbien Daugaard -- University of Copenhagen
Additional authors/chairs: 
Carsten Elbro; Jane Oakhill
Keywords: Poor Comprehenders, Assessment, Reading comprehension
Abstract / Summary: 

Purpose
Reliable identification of poor comprehenders is difficult because reading comprehension tests do not measure the same thing (Keenan et al., 2014). In addition to different contents, differences in how the same test is administered may also cause instability of the definition of poor comprehenders.

Method
First, 21 poor comprehenders and 26 good comprehenders (aged 12 years) were identified from a sample of 168 students by means of a group-administered reading comprehension test (with silent reading, written responses, and a time limit). Then, the good and poor comprehenders were re-tested with a second form of the same reading comprehension test, which was individually administered (oral reading, oral responses, and no time limit). The stability of the identification of poor comprehenders across the two administrations of the test was compared to the parallel-forms stability (and reliability) of the test using only one format (group testing, in a separate study of 95 participants).

Results
Despite different administrations of the parallel forms (group versus individual administration) 81 % of the participants could be classified in the same way. For the group of poor comprehenders stability was only 59 %. However, the correlation between the two forms of the reading comprehension tests was significantly higher when the test condition was constant (both group) than when the test condition varied (group versus individual) (r = .80 and r = .60, z = 2.14, p < 0.05). Hence, test reliability did not fully account for the instability of the poor comprehender profile when the different conditions were applied.

Conclusions
The findings suggest that test condition (group versus individual, and/or silent versus oral, and/or timed versus untimed) may be a source of variation in reading comprehension test scores. This variation may in turn limit the stability of the identification of poor comprehenders.