Success of College-Readiness Intervention Hard to Gauge

Stubbornly high college remediation rates have revealed a painful equation: High school completion does not equal college readiness. That disconnection has prompted national leaders to focus like never before on figuring out how to ensure that high school graduates are truly ready to succeed in college. In that quest, a California program is often cited as a model.

The Early Assessment Program draws praise for doing something few thought possible: It brought together K-12 and higher education and got them to agree on the knowledge and skills that constitute college-level mastery. They created a test that sends rising high school seniors an early signal about their readiness in mathematics and literacy, and allows those who meet the mark to go right into credit-bearing coursework as college freshmen, skipping remedial classes. To complete the picture, they crafted a suite of courses to bring lagging 12th graders up to college-level snuff and added training for preservice and in-service teachers.

Fusing all those elements into one system was unheard of a decade ago, when the EAP took shape. The most striking element was that the test, coursework, and training were infused with college expectations, since they were written jointly by faculty members from K-12 schools and California State University, one of the state’s two public university systems.

Early Assessment Program Results

The number of students who take California’s voluntary test of college readiness has soared since 2004, when the test was first offered. The proportion of students who meet the “college-ready” cutoff score for mathematics or English is rising slowly, but the vast majority still fall short of the mark.

SOURCE: California State University

“We’ve gone from a system [of state tests] that looks backward, asking how well we did, to one that looks ahead, asking if we have really gotten students ready for college,” said Douglas McRae, who helped design the state’s tests in the 1990s. “That’s a big mindset shift.”

Even some of the program’s admirers, though, are frustrated by the lack of key data about its impact and worry that execution of the EAP might not fulfill its promise.

“It’s been ballyhooed for its design, which is deserved,” said Michael W. Kirst, a professor emeritus of education at Stanford University who studies the system. “But there is scant information about its implementation on the ground.”

The accomplishments of California’s Early Assessment Program—as well as the unanswered questions about it—are particularly pertinent now, as educators and policymakers across the country try to define, build, and assess students’ readiness for college.

One part of that work is in carrying out new common academic standards that were written to reflect college-level skills and have been adopted fully or provisionally by all but seven states. Another part lies with two big groups of states that are collaborating to design new tests for those standards. Leaders of those efforts cite the EAP as a model as they endeavor to fold elements of it into work they hope will ultimately make college remediation unnecessary: aligning K-12 study and tests to college expectations, creating a feedback loop to inform learning, and providing supports to students and teachers.

“We look over at the EAP and ask, ‘How did they crack that nut?’ ” said Joe Willhoft, the executive director of the SMARTER Balanced consortium, one of the two assessment groups.

Measuring Impact

Few dispute the significance of the EAP’s pivotal victory: getting precollegiate and higher education to agree on what college-level mastery looks like. But gauging its impact has been trickier. Since the test was first given in 2004, the program has generated a confounding mix of results.

There are early signs that it reduces the need for college remediation: A 2009 study by Michal Kurlaender, a researcher at the University of California, Davis, found that students at one California State University campus who had taken the EAP—regardless of their scores—were 4 percentage points to 6 percentage points less likely to require remediation than those who hadn’t. The test likely prompts students to “buckle down” harder in 12th grade, Ms. Kurlaender said in an interview.

The number of students opting to take the EAP has soared, and the proportion meeting its readiness cutoffs, while small, is growing.

Some speculate that the rising scores are the payoff for the investment in teacher training. CSU has embedded the EAP’s ideas into preservice teacher education and offers online supports for teachers, as well as in-person courses to bolster their instruction. More than 9,600 teachers—nearly 20 percent of the state’s high school English teachers—have taken one or both of the literacy courses, and more than 2,000 have done so in math, according to CSU’s data.

CSU studies show that schools that fully implement the Expository Reading and Writing Course, which was designed to bolster instruction for students who fall short of the EAP’s English-readiness mark, improve on the state’s 11th grade test twice as fast as schools that don’t, and students who take it have a better chance of passing the university system’s own placement tests.

In CSU surveys, teachers rave about the course for its power to spark top-notch work from both students and teachers. Of California’s 1,400 public high schools, 376 have fully implemented the course; others use some of its modules.

Some data on the EAP are less encouraging. Remediation rates at CSU remain largely unchanged in the past six years. Even though all first-time freshmen enter the system having completed the system’s required “A-G” course sequence, and the average freshman carried a B average in high school, 49 percent must take remedial English, and 38 percent must take remedial math.

Too Little, Too Late?

Critics note that a crucial question about the EAP—whether it is a valid measure of college readiness—hasn’t been fully answered. CSU has not yet completed its study of how students fare in credit-bearing work after EAP exemptions from remedial courses. One small study, from Santa Rosa Junior College, did find that students with EAP exemptions had higher grade point averages in their college coursework than those of the college’s general population.

William G. Tierney, a professor at the University of Southern California’s Rossier School of Education, urged policymakers in a 2008 paper to “suspend the extensive accolades the EAP has gotten based on sketchy evidence” of its success.

In an interview, he said the program is weak on helping students once they fall short of the readiness cutoff. The expository-writing course is not as widely available as it should be, he said, especially in low-performing schools where it’s needed most. And too often, he found in his research, students don’t understand the letter notifying them of their EAP results and are unaware of catch-up options.

“Assessment is important, absolutely,” Mr. Tierney said. “But if students and parents don’t know what to do with the results, it’s useless.”

One small study, as yet unpublished, put a fine point on that problem. It found that four in 10 students didn’t remember taking the EAP. Of those who didn’t pass it, more than 75 percent opted to do nothing about it in 12th grade.

Even those who support the Early Assessment Program wonder if its early-warning function kicks in too late to be helpful. By the time students get the news that they are not college-ready—when they’re rising seniors—it’s often too late to rearrange their class schedules. Many students, also, are too far short of the mark to catch up in just one year.

Signals sent by the state’s own testing system might contribute to the problem. A study by James S. Lanich, the director of CSU’s Center to Close the Achievement Gap, back-mapped the EAP’s cutoff score to 4th grade California Standards Tests. He found that a student who didn’t score “proficient” or above on that test in 4th grade had only a 3 percent chance of meeting the EAP’s readiness mark seven years later. That suggests, he said, that the cutoffs on the state’s own tests aren’t progressively preparing students for college readiness.

California’s experience with the EAP has delivered abundant food for thought and revision.

Carolina C. Cardenas, the associate director of academic outreach and early assessment in the CSU chancellor’s office, said the system is getting better able to track EAP-exempt students’ performance in credit-bearing courses and is investigating ways to build early-warning indicators into middle grades. Even if the program hasn’t yet produced huge gains in college readiness, she said, it has been a potent force for improvement by clarifying and raising expectations for everyone in the school pipeline.

“When counselors tell us, ‘You give us the [EAP] results too late; I can’t change their class schedule,’ I get it. I really do,” Ms. Cardenas said. “But my feeling is, why wouldn’t the student already be signed up for a fourth year of college-prep English? And why wouldn’t you get more kids in a fourth year of math? These conversations lead us to move more in that direction.”

Moving in that direction, though, is chronically hobbled by a shortage of resources and multiple policy priorities, some longtime activists in the state say.

Scott Hill, who served as undersecretary of education under then-Gov. Arnold Schwarzenegger and now oversees education policy at the Rancho Cordova, Calif.-based consulting company School Innovations & Advocacy, said the EAP has done “an awful lot to start the conversation” about college preparedness. But he hasn’t seen as much of a “response of curiosity” from the K-12 system as he had hoped about what must be done to align it with higher education.

“People aren’t being evil,” Mr. Hill said. “It’s a matter of conflicting pressures and policy priorities and how difficult it is to get change in California on almost anything.”

How much districts have “embraced” the EAP as a way to improve college readiness varies greatly from place to place, said Joe Radding, who oversees college-preparation programs for the California education department. But he contended that while the program has sparked an important conversation, particularly about the state’s weakness in teaching writing and expository analysis, it can’t be expected to serve as the “major driver” in bolstering K-12 students’ college readiness.

“California still has a lot of work to do on things like high school graduation requirements and accountability,” he said. “We can’t leave those out of the discussion.”

Community Colleges Join In

The EAP has expanded its reach; community colleges are now part of the program. A 2008 law allowed them to participate, and as of this academic year, 48 of California’s 112 community colleges are accepting the EAP’s results in lieu of their own course-placement tests.

In an attempt to “mirror what the CSU is expecting” of transfer students, the community colleges are using the same EAP cutoff score that the university system uses, said Sonia Ortiz-Mercado, the community college system’s dean for matriculation and early assessment. Most high school students don’t think they need to prepare well for community college, she said, so using the same readiness score as the university system sends “a stronger signal that they do.”

Some educators are concerned that doing so could discourage high school students from enrolling in community colleges. But Ms. Ortiz-Mercado said the system has taken extra steps to clarify that while many students might not meet the EAP benchmark for credit-bearing courses, all students can still be admitted to the open-access institution.

Stanford’s Mr. Kirst, who is studying the inclusion of community colleges in the EAP and also serves on the state board of education, said that the program can’t easily be “retrofitted” for that population, since such students are traditionally less academically prepared than university-bound ones. It is especially important, he said, that earlier warning signals and better supports be built into the K-12 pipeline to help students who will ultimately attend community colleges.

Ms. Ortiz-Mercado said the community college system is weighing the possibility of using an additional “secondary cut score” that could provide more fine-tuned information to guide course-placement decisions, since there is wide variation in skill levels among the many students the EAP deems “not yet ready” for college.

Meanwhile, the University of California system, which draws from the state’s highest-achieving group of graduating seniors, supports the Early Assessment Program in concept, but has not agreed to use its results for course-placement purposes.

“Neither test was at a level that gave us confidence we could use it to measure college readiness for our purposes,” said Susan Wilbur, the UC system’s director of undergraduate admissions.

Key lessons from the EAP are informing college-readiness work around the country. Both common-assessment consortia, for example, have engaged the support of higher education to design tests that reflect their expectations. Though plans are still in the early stages, both groups envision testing systems that will deliver early and frequent signs of students’ progress and be paired with supports for students and teachers. Both plan research to validate their tests as true indicators of college readiness.

Designing all the moving parts of a system that builds college readiness is a massive undertaking, said David S. Spence, who was instrumental in the EAP’s early design when he served as executive vice chancellor and chief academic officer of the California State University system. Now president of the Southern Regional Education Board, Mr. Spence is working with six states as they adopt EAP-like approaches.

“I know the challenge of doing this,” he said. “But somehow, we have got to find a way to do it.”