Everybody knows that a little creativity, wedded with a good amount of common sense, can go a long way in life, whether it’s in a pickup basketball game or the thorniest corporate board meeting.
Yet those attributes aren’t always easy to quantify—and they aren’t measured with much success in today’s most widely used college admissions test, according to a prominent scholar and team of researchers working out of Yale University.
Robert J. Sternberg, a professor of psychology at Yale who has probed the norms and nuances of intelligence and wisdom for years, has completed the first phase of a project he believes could produce an admissions test that is a stronger predictor of success in college, and fairer to underrepresented minorities.
His premise is the belief that the SAT, taken about 2.6 million times by students each year, measures primarily memory and analytical ability, rather than creative or practical skills—the latter of which is usually associated with common sense.
Creative and practical talents are important for determining an individual’s success in college, and in life, Mr. Sternberg contends. He directs the study, called the Rainbow Project, which is financed by the College Board, the New York City-based nonprofit group that sponsors the SAT.
“The system fails to identify kids who might succeed in ways the [current SAT] doesn’t suggest they will,” said Mr. Sternberg, the director of the PACE Center at Yale in New Haven, Conn. He is also the president of the American Psychological Association.
For the project’s first phase, researchers designed an alternative exam, one that Mr. Sternberg believes eventually could augment, rather than replace, the SAT. The new test was given to 1,007 student volunteers at two high schools, five community colleges, and eight four-year colleges across the country, mostly from April to June of 2001.
Because researchers sought to gauge the predictability of success in college, not high school, the first phase of the study primarily used the data collected from the 793 first-year college students among the group of volunteers. Elena Grigorenko, the deputy director of the PACE center, took the lead in coordinating much of the testing process.
New Sections
The exam evaluated students in three overall areas: analytical, practical, and creative skills. It used a combination of multiple-choice questions and performance-based sections, in which the participants worked out their own answers.
Some of the sections, such as one asking students to figure out word meanings in the context of written passages, bear a resemblance to the current version of the SAT. Many other sections, though, share little with today’s model.
In one section, students were given cartoons, purchased from the archives of The New Yorker, and were asked to write captions for them. Test-takers were also asked to write two stories, taking 15 minutes each, based on a list of fictitious titles. A team of judges rated those stories on originality, evocativeness, complexity, and descriptiveness.
Other sections presented students with various troublesome situations they might encounter in everyday life and asked them to rate numerically the best solutions.
A supervisor has assigned you to work with an employee who is “rude, lazy, and rarely does a proper job,” according to one sample on the “common sense” questionnaire. Students were then asked to rate the quality of different responses, such as telling the colleague off, or working with him in a businesslike manner.
The current SAT is a good predictor of college success, Mr. Sternberg asserts, but he believes the Rainbow Project’s test does even better. He also says the new test showed smaller gaps between the scores of students from different racial and ethnic groups. In theory, the new exam could offer a college-admissions test that more accurately predicts postsecondary achievement, with fewer worries that the exam was biased in favor of one racial or ethnic group or another.
Diane F. Halpern, a professor of psychology at Claremont McKenna College, in Claremont, Calif., who assisted with the project, arranged to give the exams by computer to about 150 students at the campus where she used to work, California State University-San Bernardino. She saw value in the study.
With today’s technology, students have the ability to access more information more quickly than ever, Ms. Halpern noted. But the challenge in college, and in life, she said, is approaching data critically and gauging what is most important.
College-admissions tests, she argues, need to reflect the demand for those skills.
Creativity in a Can
One researcher familiar with Mr. Sternberg’s study, Paul R. Sackett of the University of Minnesota, disputes some of the project’s conclusions, though he says he admires its broader goal.
By giving the test to first-year college students, the Rainbow Project was evaluating an overly restricted population of test-takers, said Mr. Sackett, a professor of psychology at the university’s Twin Cities campus.
And in trying to gauge the success of the participating students in college, the Yale project did not go far enough in accounting for the differences in grading at the particular schools, Mr. Sackett contends.
Overall, while he sees merit in the study, Mr. Sackett believes that its claims about predicting college success, and reducing disparities between racial and ethnic groups, are overstated.
But Mr. Sternberg counters that his test-taking population was quite diverse, since it included high school and community college students, who might not get accepted or even choose to go to a four-year college at all. And he said adjustments had been made to account for college grading.
A central question, Mr. Sackett noted, is whether the project has any value for a broad swath of the college-going population. He had doubts: The ability to assess students’ creative and practical abilities could suffer from college applicants’ determination to manipulate and beat the system.
“Will it still measure creativity when people can go to two-day workshops aimed at writing cartoon captions?” Mr. Sackett said.
But Mr. Sternberg asserts that because sections of the test grade students on innovation and the novelty of their ideas, their scores would suffer if they were simply to repeat stale answers suggested by test-prep courses.
Rather than supplanting the SAT, Mr. Sternberg envisions that a portion of college-bound students could take the new test, either because certain colleges asked them to or because the students voluntarily signed up for it. The second phase of the project will focus on giving the test to a larger sample of students—anywhere between 5,000 and 10,000—and shortening its time from about three hours, among other improvements.
Mr. Sternberg is optimistic about where the project is headed.
“I’m very pleased,” he said. “We see this as not just a project, but a step to transform education ... that will change the way we teach in the classroom.”
