As the 2000 presidential campaign heats up, the rapid gains that Texas students are making on state tests are being called everything from a “miracle” to an “outright fraud” by commentators and academics from coast to coast. Almost certainly, neither of those extremes is true, but in the swirl of conflicting evidence surrounding the state’s school performance, it’s difficult to tell which tale, in the end, will be taller.
Reaching a final verdict on Texas’ school initiatives will be important, and not just because the governor of the Lone Star State is running for president. The state has been engaging in one of the longest and most controversial experiments to date with a system of high-pressure student tests.
Results from the Texas Assessment of Academic Skills, better known as TAAS, determine which schools are failing and which students get a high school diploma. And schools must show that scores are improving not only for their students as a whole, but also for certain subgroups—including specifically, African- Americans, Hispanics, and disadvantaged students—or risk an eventual state takeover.
In his bid for the Republican nomination, Gov. George W. Bush has often pointed with pride to his state’s nearly six-year-long rise in test scores, especially for minority students. The program actually predates Mr. Bush’s term of office but the system has come into full flower during his tenure. Now, he is one of its biggest boosters.
The numbers show overall pass rates on the test climbing from 53 percent in 1994, the first year it was fully implemented, to a record 80 percent this spring. And, while black and Hispanic students are falling further behind their white and Asian-American counterparts on some national tests, they appear to be making better progress in the Lone Star State.
Passing rates for African-Americans have shot up from 31 percent in 1994, the first year the tests were fully implemented, to 67 percent this year. Hispanics’ success rates grew from 39 percent to 72 percent over the same period. Schools in some districts, such as the Aldine Independent School District outside of Houston, have nearly succeeded in closing the racial and ethnic achievement gaps altogether. (“In a Texas District, Test Scores for Minority Students Have Soared,” April 5, 2000.)
Gaps Persist
But dissenters, who range from minority-group advocates to prominent academics, are also quick to note that students from minority groups are still far behind. They pass the tests at only two-thirds the rate of whites.
Texas’ test-score gains are a mirage, they claim, because: the system is causing large numbers of minority students to drop out, many special education students are being excluded, and mindless test preparation is elbowing out real learning.
“What is happening in Texas seems to me to be not just an illusion, but from an educational point of view, an outright fraud,” said Walter M. Haney, a Boston College education professor. He testified against the state last year in an unsuccessful lawsuit brought on behalf of groups of minority students who had failed the high school exit exams.
The truth, many other experts say, may lie somewhere in the middle.
“It’s the proverbial half- empty, half-full glass because, prior to TAAS and the accountability system, for many groups of kids no teaching had been going on,” said A. Gary Dworkin, a University of Houston sociologist. “I would certainly like to see more higher- order thinking skills and smaller class sizes,” he added. “But there are also fewer kids who fall through the cracks, fewer kids who are ignored, and fewer kids whose education is considered irrelevant.”
To back up their claims, both critics and boosters of the state’s reforms have looked to independent tests, such as the National Assessment of Educational Progress, which is periodically administered by the federal government to samples of students in most states, and the SAT.
On the seven NAEP tests given to 4th and 8th graders between 1990 and 1996, Texas and North Carolina made the largest average gains in the nation, according to a widely reported 1998 analysis by David W. Grissmer, a senior managing scientist at the RAND Corp., a Santa Monica, Calif.-based think tank.
In math, black 4th graders in Texas even ranked first among their counterparts nationwide in 1996. Hispanic students, ranking 6th on that test that year, were not far behind.
But, overall, Texas schoolchildren tend to score no better and no worse than the national average in key subjects, such as reading and math. And, while the shrinking racial and ethnic achievement gaps seen in TAAS are reflected in most of the NAEP results, those gulf shave actually widened in some areas. In 4th grade reading, for example, the gap between blacks and whites has worsened considerably since 1992. Critics, such as Mr. Haney, also contend the state’s high NAEP scores could be inflated because the state has relatively high retention rates, compared to many other states.
On the SAT, in comparison, Texas scores remained flat over the last decade. But experts said that particular lack of progress is easier to explain: A more selective group of students take the college-entrance exam.
“TAAS started as a basic-skills test and most of the improvements we see are coming in schools and among students who were very low-performing to begin with,” said Albert Cortez, the director of the institute for policy and leadership at the Intercultural Development Research Association, a San Antonio-based nonprofit group.
Many Texas-watchers suspect that higher-achieving students have “topped-out” on the state tests.
But critics look at the SAT results and see another picture. To them, the stagnant scores are further evidence that teachers are spending too much time teaching to the lower-level skills on the Texas tests—and not enough time on the higher-level skills that colleges demand.
Surveys from the front lines of Texas teachers, in fact, seem to bolster that argument. Asked on one recent survey whether the reforms have led to improved teaching in Texas, for example, just 27 percent of teachers said they thought better TAAS scores reflected genuine learning gains.
A clearer picture may emerge next year when the state administers a tougher version of the TAAS based on upgraded state standards. If the gap between racial and ethnic groups widens on that round, it could throw more cold water on the Texas “miracle.”
Confusion Over Dropouts
Mr. Haney, who has spent much of the last two years analyzing Texas data, also doubts that the state’s improvements are real for another reason: He contends that fewer likely-to-fail students are taking the tests.
“I would guess at least half of the apparent increases are a mirage resulting from increasing numbers of students being excluded from test results—either because they dropped out of school or they’ve been misclassified as special education students,” he said.
Estimates of actual dropout rates in the Lone Star State vary wildly. At the optimistic end of the scale, the Texas Education Agency reports a 71 percent decline over eight years in the number of students who drop out each year from Texas schools. The percentage dropped from 6.1 percent in 1988-89 to 1.6 percent in 1997-98, the agency says.
Researchers such as Boston College’s Haney and the Intercultural Research Development Association, in contrast, make a case for sizeable and uneven rises in dropout rates since the 1980s. The San Antonio researchers, for example, estimate that as many as two of every five Texas students leave school sometime between 9th and 12th grades.
Even more disturbing, they maintain, African-American and Hispanic students are more than 1.6 times as likely as white students to disappear from the attendance rolls.
The researchers “all have different methodologies and even the definition of who is a dropout and who isn’t a dropout differs,” said Edward J. Fuller, a research and policy specialist at the Charles A. Dana Center, a research group at the University of Texas at Austin. “The actual rate is probably somewhere between TEA’s estimate and IDRA’s and Walt Haney’s estimates. We really don’t know where.”
The official state count, based on the reports of local school administrators, does not include, for example, immigrant students who return to their home country, students who say they are leaving to pursue a high-school equivalency diploma, and students who enroll in other districts.
One problem with the last two categories, said Mr. Cortez, is that local school officials rarely verify whether students are indeed working on their GED or attending school somewhere else.
Plus, schools’ accountability ratings are based, in part, on their success in keeping dropout numbers low. Critics contend that kind of pressure is prompting school officials to misreport dropout data. As of February, four Texas districts— Austin, North Forest, Quitman, and Ysleta—had earned unacceptable ratings from the state because of concerns about the accuracy of their dropout data.
“If it’s self-reported, you basically can’t trust it,” said Mr. Haney.
In comparison, he and IDRA calculate dropout estimates by comparing enrollment rates in lower grades with 12th grade enrollment years later. Mr. Haney estimates, for example, that the percentage of minority 6th graders who make it to 12th grade has declined from a little more than 80 percent in the pre-TAAS 1980s to just under 70 percent last year. The persistence rate for white 6th graders, after suffering a similar but more gradual dip, now hovers around 80 percent—10 percentage points higher than the minority rate.
But other researchers point out some problems with that method, too. One is that it doesn’t adequately account for in- and out-migration of students. (The San Antonio group does, however, make some adjustments for overall student population changes in a district.)
The confusion has drawn the eye of a state legislative task force, which plans to issue a call for reforms in Texas’ counting methods this fall. The state education agency has also launched a study to track the same group of students over time, an approach considered more reliable.
Mr. Haney also believes the pass rates are inflated because more special education students are sitting out the tests. The percentage of disabled 10th graders who did not take the exams nearly doubled from 1994 to 1998, for example, growing from 3.9 percent to 6.3 percent.
Likewise, critics also charge that too many limited-English-proficient students have been excluded from the tests. Thanks to legislative changes, however, the state this year managed to nearly halve the number of students with limited English- speaking skills who were excused from the tests—and still post record-high test scores.
Retention Rates Debated
Almost everyone also agrees that the numbers of Texas students who stay back in 9th grade are peculiarly high. During the 1996-97 school year, for example, nearly 18 percent of all students were repeating a grade. And the percentages of 9th grade repeaters among minority students is more than twice as high as it is for white students. The timing of what is sometimes called the “9th grade pileup” is significant because students take the high school exit tests in 10th grade.
What experts disagree on, however, is whether the bottleneck is pushing more students, especially those who are minorities, out of school. Historically, research suggests that repeating a grade is the single biggest predictor of whether a student will drop out.
“Retention rates are going up,” said Martin Carnoy, a Stanford University researcher who is studying Texas high schools. “But it may also be the case that it is improving the finish rate for kids.” He noted, for example, that graduation and college-going rates for blacks and Hispanics were on the rise throughout the 1990s even as percentages of students held back climbed.
Mr. Carnoy has found that dropout rates have decreased rather than increased across the state in the high schools that have registered the biggest test-score gains over time.
A growing heap of anecdotal evidence, on the other hand, paint portraits of classrooms in which real teaching and learning grinds to a halt for weeks as teachers drill students with TAAS-like questions and offer tips on test-taking strategies.
“Many science teachers with poor and minority children are required by their principals to suspend the teaching of science for weeks, and in some cases, months in order to devote science class time to drill and practice on the math sections of the TAAS,” writes Rice University researcher Linda McNeil in a paper on the subject prepared for the annual meeting of the American Educational Research Association. And she says instruction in art, social studies, and other subjects are likewise put on hold in favor of test-practice sessions.
In contrast, said Mr. Fuller of the Dana Center, “We’ve done studies that looked at high-performing, high-minority, predominantly low-income schools and we’ve found exactly the opposite going on.’'
“It’s not that the TAAS assessment and accountability system necessarily results in teaching to the tests. It’s that some schools respond effectively to the system and some schools resist responding effectively to the system,” he said. “The thing to remember is that, previous to the accountability system, the achievement of minority students in Texas was basically hidden from everyone.”
Further clues will come later this year when scores from the latest round of NAEP testing become available. And politicians, as well as researchers and educators, will be watching closely.
In truth, however, both Mr. Bush and his likely Democratic opponent, Vice President Al Gore, have embraced the standards-and-testing movement that is sweeping the states.
“If this were not an election year, there may have been a little less attention on TAAS,’' said Mr. Dworkin. “But one needs to realize that most of the activities that led to the accountability system predate the incumbent governor and they occurred under both Democratic and Republican governors.”
