Investigating the reliability and validity of a measure of text structure awareness in grades 4-5

Investigating the reliability and validity of a measure of text structure awareness in grades 4-5

First Author: John Strong -- University at Buffalo
Keywords: Assessment, Grade 4-5, Informational Text, Measurement, Text structure
Abstract / Summary: 

Purpose

Text structure awareness is related to reading comprehension and varies according to characteristics of readers and texts (Ray & Meyer, 2011). The purpose of this study was to develop and refine a measure of text structure awareness, the Text Structure Identification Test (TSIT), by investigating its internal consistency reliability and construct validity.

Methods

Students in grades 4 (n = 84) and 5 (n = 72) were administered the TSIT. Students read 20 informational texts and identified which text structure was used. Reliability was assessed using the KR20 formula to compute coefficient alpha. Validity was evaluated using an exploratory factor analysis to examine the test’s internal structure. Independent and paired samples t-tests were used to analyze differences in text structure awareness by grade and type of structure.

Results

Five items with item-total correlations below .20 were removed. EFA results revealed that 12 items showed salient loadings on a single factor, explaining 22.2% of the variance. Three items with item-factor associations below .32 were removed, resulting in a 12-item test with acceptable reliability (⍺ = .72). Results on the revised test revealed that fifth-grade students in demonstrated higher text structure awareness (p = .01). Awareness of problem/solution and sequence were higher than compare/contrast and cause/effect (p < .05).

Conclusions

This study provides evidence of the reliability and validity of the TSIT, which adds to the literature by filling a need for classroom-based assessments of text structure awareness to identify students who might benefit from text structure interventions (Ray & Meyer, 2011).