Reliability and Predictive Validity of Expressive Measures Designed to Teach Science Words in Depth

Reliability and Predictive Validity of Expressive Measures Designed to Teach Science Words in Depth

First Author: Thu Le -- Southern Methodist University
Additional authors/chairs: 
Doris Baker
Keywords: second language learning, consistency, Predictive Reading, Grade 1-2, science
Abstract / Summary: 

Purpose
This poster will present results of the reliability and predictive validity of expressive vocabulary and content measures used in an English language vocabulary program (ELVA) designed to improve the vocabulary knowledge and language proficiency of second-grade students in the US. Reliability and predictive validity will be examined based on Standards for Educational and Psychological Testing (Test Standard, 2014). In this study, we analyze student utterances related to Earth Sciences such as Rocks and Volcanoes. Students were exposed to the lessons through a Learning Management System (LMS) for about 30 minutes per day for one month.

Method
Research Design. Participants were randomly assigned to receive science vocabulary lessons through the LMS.
Participants. 46 second-grade girls with varying levels of English language proficiency from four classrooms in a school that focuses on STEM learning for girls.
Measures. An adapted standardized picture science assessment (PSA), the Expressive One Word Picture Vocabulary Test (EOWPVT) at posttest, and the Depth of Knowledge (DOK) at pretest and posttest.

Results
Reliability of two raters on student responses were high with 0.86 for PSA, 0.88 for DOK pretest and 0.82 for DOK posttest. This indicated a strong interrater reliability. The correlation between DOK pretest and DOK posttest was 0.67, and the correlation between DOK pretest and PSA posttest was 0.47. The correlations were significant (p<0.05), suggesting that science measures had predictive validity. The findings also indicated that DOK measure predicted EOWPVT.

Conclusion
The preliminary findings indicated that there was a strong interrater reliability on student responses. Additionally, DOK Pretest can significant predict student responses on science measures. Consequently, reliability and predictive validity of expressive vocabulary and content measures used in ELVA were significantly strong.