National Clout of DIBELS Test Draws Scrutiny

Just a few years ago, a set of tests known as “dibbles” would have elicited little more than a chuckle from educators or anyone else. Today, they’re taking it seriously, because the acronym DIBELS has come to symbolize the standard for early-literacy assessment throughout much of the country.

Teachers in Reading First schools in more than 40 states now use the Dynamic Indicators of Basic Early Literacy Skills to screen K-3 pupils for potential reading problems and to monitor their progress. And state officials are collecting the data from the short reading-fluency tests to determine whether schools receiving some of the $1 billion given annually in federal Reading First grants are making adequate progress in getting students up to grade-level proficiency.

Several states have gone so far as to adopt the assessments for all schools to use regularly.

Seen as Reliable

Developed by researchers at the University of Oregon, DIBELS has become a catchphrase in the schoolhouse and the statehouse as officials look to test data to inform instruction, to identify children at risk of failure in reading, and to hold schools accountable for student achievement.

But while teachers, administrators, and researchers praise the tests for their ease of use and reliability in predicting which children may have reading difficulties later, the use of DIBELS has drawn criticism from some in the field. Critics cite the tendency of some educators to teach to the tests or give the measures too much weight in gauging reading ability, as well as the often-aggressive promotion of DIBELS by federal employees and consultants to the Reading First program.

“If you want a test of whether kids can read fast with low comprehension, then DIBELS is great,” said G. Michael Pressley, a professor of education at Michigan State University in East Lansing and a former editor of the Journal of Educational Psychology. “It is not as well developed as the claims that are being made.”

DIBELS has become the de facto national assessment for Reading First over the last three years, with the tests’ use in most of the 4,800 schools in the program and a number of other schools. Reading First, adopted under the No Child Left Behind Act, authorized $6 billion over six years to implement strongly skills-based reading instruction in the nation’s struggling schools.

Although the total number of schools using the tests is unknown, more than 8,200 schools in 2,600 districts have signed up for the associated data-management system offered by the University of Oregon at a cost of $1 per student.

DIBELS has become widely used because “it is pretty darn good,” Roland H. Good III, who developed the tests with Ruth A. Kaminski, wrote in an e-mail message to Education Week.

“DIBELS is a reliable and valid measurement tool…that helps teachers make decisions to change reading outcomes for students,” he wrote. “It is also very efficient to use and requires modest resources to implement.”

‘Lol’ and ‘Tob’

DIBELS measures such skills as letter naming, recognizing the sounds of each letter, sounding out words, and read-aloud fluency. Its appeal is that the tests are “brief, replicable, and relatively easy to administer,” said Natalie Rathvon, a Bethesda, Md.-based consultant and the author of Early Reading Assessment: A Practitioner’s Handbook. “DIBELS is great for testing those foundational skills.”

The tests for K-6 cover a range of tasks and include benchmarks for determining whether pupils are on grade level or at risk of problems with reading. On the test of letter-naming fluency for kindergartners, for example, youngsters are asked to identify as many randomly listed uppercase and lowercase letters as they can in one minute.

On the nonsense-word-fluency test, K-2 pupils must decode pseudo-words, such as “lol” or “tob,” as quickly as they can for one minute, a measure intended to gauge how well they know the letter sounds. First, 2nd, and 3rd graders are also given a test of their oral-reading fluency, which measures how many words in a given text passage they can read accurately in a minute.

At Spencer Math and Science Academy in Chicago last week, Rachel Woodrick, the Reading First coach, was in the midst of testing 3rd graders on how well they could read DIBELS passages. After spending a few minutes with each student, she entered the scores into a hand-held computer and received feedback on how each child measured up to the desired standard.

“Once you learn to do the test it’s really easy to do … and it gives me a fast result,” she said. “And the test does not push [the children] to an extreme frustration level” if they are struggling with their reading.

The battery of tests was not the first choice for Illinois and some states applying for the federal money. In fact, a number of states had intended to use other assessments for screening children and gauging progress in Reading First schools. They changed their plans, they maintain, after federal officials and consultants pressured them to include DIBELS in their grant proposal as a condition for approval. Federal officials deny those charges. (“States Pressed to Refashion Reading First Grant Designs,” Sept. 7, 2005.)

Still, observers say that DIBELS, which was in use years before the Reading First initiative, may have seemed the best fit for Reading First schools because the tests are quick and yield information about aptitude on a range of reading tasks. They were also designed around the same research on effective reading instruction that guides the federal initiative, Mr. Good said.

“It is not a coincidence … that the structure of our work corresponds so closely to Reading First and the National Reading Panel report,” the influential research review released in 2000, Mr. Good wrote. “We were all reading the same research and committed to following the evidence and the data.”

Competitive Edge

Some states were able to make a case for other tests. In Virginia, for example, Reading First schools use the Phonological Awareness Literacy Screening tests, or PALS. That system, developed by researchers at the University of Virginia in Charlottesville, has been the early-literacy assessment in that state since 1997 and is used by schools and districts in more than 40 other states as well.

Virginia was able to make its case for using the test in Reading First schools based on its use in nearly all of the state’s 135 school districts over the last 8 years. Several other states also initially chose PALS as their evaluation instrument. But federal reviewers, who questioned the validity and reliability of the tests, repeatedly returned those proposals. Marcia Invernizzi, the director of PALS, calls the federal claims “outrageous.”

“I would pit PALS’ validity and reliability studies against any Reading First assessment,” she said. “DIBELS is a terrific assessment tool, … but it lacks the instructional transparence and utility that teachers need.”

Ms. Rathvon, the Maryland-based consultant, agrees. While DIBELS provides good assessment tools, it is “a work in progress,” she writes in her book. Nor does it give teachers clear answers for designing instruction around students’ needs. For example, PALS pinpoints if a 2nd grader is reading at a level expected of a mid-year 1st grader, while DIBELS determines only if a pupil does or does not measure up to the benchmarks set for his or her own grade level.

The Virginia test system, however, was not included in a federally commissioned expert review of research-based early-literacy assessments that met Reading First requirements, although Ms. Invernizzi said she twice submitted the materials.

Only a few states were able to get tests other than DIBELS, such as the Developmental Reading Assessment and the Texas Primary Reading Inventory, approved under Reading First.

Some critics charge that DIBELS got the competitive edge not because of its superiority, but because its developers and their colleagues at the University of Oregon, located in Eugene, were key consultants to the U.S. Department of Education for Reading First. Mr. Good was on the assessment committee that evaluated 29 early-literacy tests, including DIBELS, his own product. That list was provided as a resource to states for drafting their Reading First plans.

Related Products

While the tests are free for download, slick, packaged sets and related products are sold by Sopris West, a Longmont, Colo.-based company that sells instructional materials and assessments and offers teacher training. The data-management program has been licensed to Wireless Generation, a New York City-based company selling hand-held computers to schools for assessment and data management.

States and districts have also paid for training sessions. Mr. Good said his influence on Reading First “has been negligible” and credits other researchers with guiding the federal initiative.

Some of the harshest criticism is aimed at the test content itself. Some researchers question whether children’s speed at reading nonsense words or carefully crafted passages has anything to do with the ultimate goal of comprehension, according to P. David Pearson, the dean of education at the University of California, Berkeley. Moreover, Mr. Pearson said, the assessment measures have not been studied adequately enough to support its widespread use in Reading First.

Research has shown that students’ ability to decode nonsense words, and their speed in reading grade-level passages, are valid and reliable predictors of whether they will have reading difficulties later on, according to Ms. Rathvon.

Even so, some researchers argue that DIBELS’ reliability and its usefulness to teachers is questionable.

Michigan State’s Mr. Pressley is completing a technical report on DIBELS. So far, he said, his analysis shows that it is an accurate indicator of how pupils will perform on high-stakes state tests in reading. But DIBELS does not show whether students are on target for developing higher-level reading skills, such as vocabulary and comprehension.

“These [tested skills] become your end goal,” Mr. Pressley said. “DIBELS is leading some teachers to infer the wrong end goal, which is to read the words fast.”