Critical evaluation of the conceptualizations and operationalizations of letter knowledge

First Author: Jason Lon Anthony -- University of South Florida

Conference:

Keywords: Assessment, Letter knowledge, English, Early childhood age 3-8, English Language Learners (ELL)

Abstract / Summary:

Prereaders’ and early readers’ knowledge of the names and sounds associated with letters is highly predictive of their reading achievement. Yet only recently have studies begun to critically investigate letter knowledge (LK) as a construct. This symposium thoroughly examines various conceptualizations and operationalizations of English LK. Paper One uses item response theory analyses to test alternative models of the dimensionality of LK. Paper Two employs nominal item response models and partial credit models to help identify which letter-sound correspondences should and should not be accepted as correct responses on a test of letter sound knowledge. Paper Three identifies specific letters and entire tasks that demonstrate bias against particular gender, ethnic, and linguistic groups. Paper four demonstrates a novel Monte Carlo approach to creating parallel forms and reports psychometric evaluations of the sociolinguistically fair, short and long forms that were developed from this large-scale measurement project.

Symposium Papers:

Dimensionality of letter knowledge across names, sounds, case, and response modalities.

First Author/Chair:Jason Anthony -- University of South Florida

Purpose: Prereaders’ and early readers’ knowledge of names and sounds of letters is highly predictive of their reading achievement. This study examined the factor structure of children’s English letter knowledge (LK) to inform construction of assessments aimed to guide early literacy instruction. It was hypothesized that children’s performances would be influenced by construct (names vs sounds), case (uppercase vs lowercase), and task (multiple choice vs free response) but that influences of case and task on dimensionality would be minimal.

Method: Over 3,400 3- to 7-year-old children were administered four tests of LK. Letter name identification asked children to provide the name of individual letters. Letter sound identification asked children to provide a sound associated with individual letters. Letter sound recognition asked children to point to one of four letters that made a particular sound, which had been digitally recorded, presented by computer, and heard via headphones. Letter name recognition asked children to point to one of four letters that was labeled by the computer.

Results: Item response theory analyses found that factors distinguished by task and construct were reliably distinguishable (2differences=[1]77-545, ps< .0001). Factors could not be reliably distinguished by case, even with the large sample size. Moreover, factors distinguished by case and task were very highly correlated (rs=.98 and .95, respectively). Factors distinguished by construct were highly correlated (r=.75), but not to the extent that they should practically be considered redundant skills.

Conclusions: Results confirmed the need to develop separate tests of letter name knowledge and letter sound knowledge so that educators can monitor pupils learning in each area separately.

Which sounds should be scored as correct on an English test of letter sounds?

First Author/Chair:Janelle Montroy -- University of Texas Health Science Center at Houston

Additional authors/chairs:

Jason Anthony; Jeffrey Williams

Purpose: The ability to identify sounds associated with letters (letter sound knowledge [LSK]) is crucial to literacy development. English letters make many sounds, and researchers vary in how they score sounds that are only infrequently associated with a given letter, i.e., “low frequency sounds”. This lack of scoring clarity makes it difficult to accurately and reliably measure LSK. A compounding measurement concern is that it is unknown how well all letter sound correspondences index latent LSK.

Method: Approximately 2300 3-7 year old (M= 4.71, SD= 0.91), monolingual English children participated. The receptive test of LSK asked children to point to 1 of 4 letters that made a given sound. The expressive test asked children to produce a sound made by individual letters. Items were re-administered if children provided a ‘low frequency sound’, and both responses were recorded.

Results: Nominal item response models were used to evaluate how well each response discriminated individuals’ LSK. Results indicated (a) “instructed” letter sounds well discriminate LSK, (b) vowel names weakly discriminate LSK, (c) a few “low frequency sounds” well discriminate LSK, and (d) certain letters (Cc, Gc, Qq and vowels) may be better scored via partial credit. Follow-up analysis will confirm children who provide certain low frequency sounds indeed know the commonly associated sound(s), as evidenced by their responses to the readministrations and to the receptive test of LSK.

Conclusions: An understanding of how well letter-sound correspondences index LSK will lead to improved measurement, advanced research, and better instructional decision making.

Bias in the assessment of English letter name and English letter sound knowledge: Group differences by gender, racial, and language status groups.

First Author/Chair:Matthew Foster -- University of South Florida

Additional authors/chairs:

Jason Anthony; Jeffrey Williams; Janelle Montroy

Purpose: This study evaluated tasks and test-items commonly used to assess English letter
knowledge for bias against gender, racial, ethnic, and linguistic groups.

Method: Approximately 2,350 monolingual English speakers and 1,300 Spanish-English bilinguals participated. Participants ranged from three to six years of age and were administered four tests of LK. Letter name identification asked children to provide the names of individual letters. Letter sound identification asked children to provide a sound associated with individual letters. Letter sound recognition asked children to point to one of four letters that made a particular sound. Letter name recognition asked children to point to one of four letters labeled by the computer.

Results: We tested differential item functioning (DIF) in discrimination and difficulty by gender (male vs female), ethnicity (Caucasian vs African American vs Hispanic/Latino), and English language status (monolingual English vs bilingual). A couple letter name free response items demonstrated DIF by gender, appearing to reflect the first initial phenomenon. English language status was a minor source of DIF in three item pools but a major source of DIF in letter sounds free response.

Conclusions: Asking children to say a sound associated with a given English letter is not a good method of assessing English LK among Spanish speaking English language learners (ELLS). Their performances may reflect English sound knowledge, Spanish letter sound knowledge, or a mix of both. Thus, we advocate use of the unbiased receptive task to assess English letter sound knowledge among Spanish speaking ELLs.

Psychometric evaluations of the short and long form tests of letter knowledge on the School Readiness Curriculum Based Measurement System (SRCBM).

First Author/Chair:Jeffrey Williams -- University of South Florida

Additional authors/chairs:

Jason Anthony

Purpose: This project developed ten short form assessments of English letter name and letter sound knowledge to be used by educators for universal screening, benchmarking, and progress monitoring. Three long forms were also developed for use by researchers and program evaluators. This paper presents the methods used to create parallel forms, leveled forms, and long forms that draw from different item pools. Results from psychometric evaluations are also reported.

Method: The validity study included 400 children in PreK-3, PreK-4, and kindergarten who were tested in fall of 2016, winter of 2016/2017, and spring of 2017 with the new LK measures, existing benchmark measures, and criterion measures.

Results: The 8- to 11-item short forms and the 31- to 38-item long forms were highly reliable (mean reliabilities= .93 and .97, respectively). Inspection of test information curves indicated that the Monte Carlo methods that were employed yielded parallel short forms. Correlational and regression analyses will evaluate convergent, discriminant, and predictive validity and compare validity coefficients with those from another widely used benchmark measure. The long and short form tests of letter knowledge are expected to have excellent convergent, discriminant, and predictive validity that are superior to those of the widely used benchmark assessment.

Conclusions: The 11,000+ educators who use the existing benchmark measures with over 150,000 children per year will be better equipped to make instructional decisions when SRCBM tests replace the current benchmark measures. Alternate, parallel forms will also be available to researchers and educators who desire them.

Synthesis of research that has examined the conceptualization, development, and operationalization of letter knowledge

First Author/Chair:DISCUSSANT Shayne Piasta -- Ohio State University

We will discuss the person-, letter-, and instructional- characteristics that influence development of letter knowledge, including those characteristics identified in the present studies and those identified in other published studies.

We will also summarize recent advances in what the field has learned about measurement of letter knowledge, Discuss ways that recent advances have led to improved measures, including expanding coverage, minimizing floor and ceiling effects, availability of parallel forms, exclusion of biased items, etc. and name a number of new measures that exemplify each advancement.

Bowles et al.(2011, 2014), Piasta & Wagner (2010), Piasta et al (2016), Drouin et al (2012), Philips et al (2012), Justice et al. (2006)

You are here

Critical evaluation of the conceptualizations and operationalizations of letter knowledge