Saturday, July 30, 2005

MAZE Craze

When I went through ed school, I was exposed to the MAZE craze. I suspect it is more WL mumbo-jumbo. (With MAZE every nth word is omitted in a text. Pupils must guess the missing word).

At ed school we were asked to apply MAZE in a clinical setting to determine the pupils’ supposed reading level. Since the MAZE method is based on false premises, the results are suspect. Harm is done if these questionable results are used to determine reading levels. I say harm because pupils can read at a higher level than MAZE would indicate.

An article on reading myths mentions a study that shows the false premises on which MAZE is based.

Excerpt from the reading myths article:

Myth #5 Skilled reading involves using syntactic and semantic cues to “guess” words, and good readers make many “mistakes” as they read authentic text.

Research indicates that both of these claims are quite wrong, but both are surprisingly pervasive in reading instruction. The idea that good readers use context cues to guess words in running text comes from a method of assessment developed by Ken Goodman that he called “miscue analysis” (which has given rise to the popular “running records” assessments). For his dissertation, Goodman examined the types of mistakes that young readers make, and drew inferences about the strategies they employ as they read. He noticed that the children in his studies very often made errors as they read, but many of these errors did not change the meaning of the text (e.g. misreading “rabbit” as “bunny”). He surmised that the reason must be that good readers depend on context to predict upcoming words in passages of text. He further suggested that for good readers, these context cues are so important that the reader only needs to occasionally “sample” from the text (i.e. look at the words on the page) to confirm the predictions. Children who struggle to sound out words, Goodman says, are over-depending on the letter / word cues, and need to learn to pay more attention to the semantic and syntactic cues.

Goodman’s model, that eventually gave rise to the “Three Cueing Systems” model of word recognition, is very influential in reading instruction, but unfortunately, it has never been supported by research evidence.

In fact, repeated studies have shown that only poor readers depend upon context to try to “guess” words in text. Good readers depend heavily upon the visual information contained in the words themselves (i.e. the letter / word cues) to quickly and automatically identify the word. Keith Stanovich has been especially critical of the three cueing systems model because the predictions made by the model are exactly the opposite of what has been observed in research studies.

Philip Gough and I addressed the second claim and showed that, in fact, good readers almost never make any mistakes at all when they read, which means the notion of conducting a “miscue analysis” is somewhat suspect. How can you perform a miscue analysis when there are typically no miscues? We had over 400 college students read a passage of text from Ken Goodman’s book Phonics Phacts, and showed that the modal number of mistakes made by these students was zero. Almost all of the students read the passage flawlessly. To suggest that good readers are correctly guessing the words in the passage with one-hundred percent accuracy stretched the boundaries of credulity.

However, to be sure, we examined how accurate people would be if they were forced to use semantics and context as their only cues. We concealed the passage of text and asked our college students to guess each of the words in the passage one at a time; after each guess, the correct word was revealed, and students were asked to guess the next word. This process was repeated for every word in the passage, so the students always knew the words leading up to the unknown word. We found that, given unlimited time to ponder, students were able to correctly guess one out of ten content words in the passage. That’s a ninety percent failure rate, as opposed to the zero percent failure rate seen in skilled readers who were not forced to make guesses based on context.

It is clear that good readers depend very heavily upon the visual information contained in the word for word identification (what is commonly called the graphemic information or orthographic information). The semantic and syntactic information is critical for comprehension of passages of text, but they do not play an important role in decoding or identifying words. Good readers make virtually no mistakes as they read because they have developed extremely effective and efficient word identification skills that do not depend upon semantics/context or syntax. For good readers, word identification is fast, fluent, and automatic. It needs to be so that their attention can be fully focused on using semantics and syntax to comprehend the text.

4 comments:

Catherine Johnson said...

fast, fluent, and automatic

Expertise always looks like this.

Universally, that's how people know they've mastered something: when they've become fast, fluent, and automatic.

This is why we speak of being fluent in a foreign language.

sigh.

Anonymous said...

Hey, I recently added a news widget from www.widgetmate.com to my blog. It shows the latest news, and just took a copy and paste to implement. Might interest you too.

Todd said...

To my knowledge MAZE is not based on Goodman's flawed assumptions. It is a modification of the CLOZE procedure and is a measure of comprehension not reading fluency or accuracy (though readers have to read both fluently and accurately to see success on this task).

pavlovsdog said...

The MAZE assessment is certainly not based on the whole-language theories of Goodman, and is in the vein of curriculum based measurement (CBM) that he vehemently opposes (e.g. The Truth About DIBELS). From limited research, MAZE passages appear to assess primarily syntactic features of reading comprehension. They are a decent measure for progress monitoring fluent readers who have an isolated difficulty with comprehension (a fairly rare phenomenon). However, if the student struggles at all with fluency, R-CBM (e.g. DIBELS ORF) is a better general indicator of reading proficiency.