The Educational Testing Service and an Illinois insurance firm have reached an out-of-court settlement in an eight-year-old lawsuit over the issue of racial bias in examination questions.
But the two sides in the case strongly disagree over the significance of the settlement.
The insurance company calls it “precedent-setting” and predicts that ets will have to revise all of its standardized tests, including those used by educational institutions, or face a host of similar lawsuits. Lawyers for the testing company insist that the settlement has “zero legal significance.”
Under the terms of the settlement, which was announced last week, the nation’s largest test manufacturer will for the first time make public the proportions of blacks and whites that correctly answer each question on one of its examinations, a test used to license insurance agents in Illinois and some 22 other states.
The agreement also calls for the company to construct future versions of the test, when possible, from questions on which the passing rates of blacks and whites differ least. The content of the examination will not be changed under the agreement, nor will it be made less difficult.
As a group, blacks have scored on the average substantially lower than their white counterparts on other standardized tests produced by the ets, such as the National Teacher Examination and the Scholastic Aptitude Test. In the last few years, states’ increased use of the nte as an admissions requirement for the teaching profession has begun to reduce the proportion of blacks entering the profession, state figures show.
The suit was initiated in an Illinois state state court by the Golden Rule Insurance Company, a 1,300-employee firm. The plaintiffs also included several black individuals who contended they were harmed by purported bias in the examination.
Parties in the case last week vigorously disputed the implications of the agreement.
Representatives of the Golden Rule firm, and the testing experts and civil-rights advocates assembled last week by the company to comment on the settlement, described the steps to be taken by the testing organization as “precedent-setting.”
“It adopts for the first time an objective, publicly reviewable standard for eliminating biased items from standardized tests,” said David White, director of a project of the National Conference of Black Lawyers to eliminate cultural bias from the Law School Admissions Test.
“It is likely that students taking the sat, the Graduate Record Examinations, and other [standardized] tests will want similar safeguards, now that it has been shown that it can be done,” said J. Patrick Rooney, chairman and chief executive officer of Golden Rule.
“How can ets refuse to extend the same kind of reform to other tests, now that it has been shown that we can be sensitive to bias without compromising the integrity of a test?” said Chuck Stone, director of minority affairs at ets from 1970-72 and now senior editor of the Philadelphia Daily News. “If ets doesn’t extend these reforms to their other tests, they will soon face dozens of similar lawsuits.”
“I would predict that revisions of the sat, the act [The American College Testing Program tests], and other admissions examinations would be called for in light of this settlement,” said Asa G. Hilliard 3rd, associate professor of education at Georgia State University and a leading black educator.
Claims Denied
Spokesmen and lawyers for ets last week denied these claims.
“It is of zero legal significance,” said Joy McIntyre, director of corporate information services at ets, “because it was an out-of-court settlement involving only one occupational test.” She said ets will not voluntarily offer similar item-by-item reporting of the performance of different racial groups on its other licensing and admissions examinations.
In a telephone interview, the organization’s general counsel, Stanford von Mayrhauser, rejected the assertion that the settlement obligates ets to construct future tests from questions that have been found to have the smallest difference in passing rates between blacks and whites.
“The timing will be our own,” he said. “We will do it at our own option.”
Bias-Reduction Queried
Mr. von Mayrhauser also rejected the notion that selecting test questions on the basis of similar passing rates for blacks and whites will reduce the amount of “bias” in standardized tests.
“This is not going to have anywhere near the effect that is being claimed,” he said. “It will have little effect on the content of the test and the outcomes by subgroup on it.”
“To conclude that bias exists on the basis of [differing] performances on a test is to engage in an unwarranted inferential leap,” he added. “It’s a conclusion that is not supported in the literature on testing. It is reached by those who for political reasons find it easier to blame a test than to get at the possible underlying causes of the problem” of low minority scores on standardized tests.
Mr. von Mayrhauser acknowledged that the public reporting of item-by-item results within racial groups would be unprecedented for ets and that the settlement may result in efforts to seek similar remedies involving other ets tests. He said the organization would defend its other tests “very vigorously” in court.
Ms. McIntyre, the firm’s spokesman, said ets, which is a nonprofit organization that administered approximately six million tests and had total revenues of $133 million in 1983, currently safeguards against bias in its tests though a “carefully monitored, 18-month” internal process and external reviews by panels of experts.
Exclusions Said Important
Martin M. Shapiro, a professor of psychology at Emory University and a testing expert who helped shape the out-of-court settlement on behalf of Golden Rule and the five individuals who joined the company’s challenge, said last week that it is important to exclude questions that blacks answer correctly less often.
There are questions on the same level of difficulty and content “that will produce different rates of correct answers from race to race, whereas other questions in the same content area will not,” he said. ''There is no plausible reason for saying that a question that produces a large difference should be used instead of one in the same content area and at the same level of difficulty that gives a small or no difference. This is what is being said in this agreement and it is remarkably simple.”
8-Year-Old Case
The case, Golden Rule Life Insurance Company v. John E. Washburn, was initiated by the plaintiffs in an Illinois circuit court eight years ago. It sought damages and fees from ets and the director of the Illinois Department of Insurance on the grounds that the examination was not job-related and unfairly discriminated against blacks.
The court dismissed the suit as moot in 1978 on the grounds that ets had made sufficient changes in the content, level of difficulty, and question format of the test. It dismissed an amended suit in 1979.
Golden Rule appealed the decision to the Appellate Court for the Fourth Circuit of Illinois. That court reversed the dismissal and remanded the case to the circuit court for trial. It ruled that ets, by virtue of playing a central role in a state licensing process, was not immune, as ets lawyers had argued, from legal challenges advanced under Constitutional and federal civil-rights provisions.
The court also ruled that discrimination suits could be brought against ets if it could be shown that the organization knew its tests had a racially disparate effect but did not takes steps to alleviate the disparities.
The Illinois director of insurance and ets petitioned the state’s supreme court for the right to appeal the decision, but the court denied their request, thereby letting stand the appellate court’s decision for a trial to take place. Last week’s settlement averted that trial.
Settlement Involves Reviews
The out-of-court settlement calls for ets and Golden Rule to ask the U.S. Social Security Administration, within 60 days of the dismissal of the suit, to review the responses to questions on each Illinois insurance examination given between Jan. 1, 1983, and June 30, 1984, and report the percentage of correct answers by race. Mr. Shapiro said he suggested the idea of the role for the ssa, which is authorized to assist in the litigation of civil-rights actions.
Within two months of receiving the ssa report, ets must classify all items within one of two groups: those for which the correct answer rate for blacks, whites, and all examinees is not lower than 40 percent and for which the correct-answer rates of blacks and whites differ by less than 15 percent; and those that do not meet such standards.
ets and Golden Rule will share the cost of the ssa study.
Then, within 60 days of classifying the items, ets is to write new tests that will use questions that meet the standards “so long as they are available in sufficient numbers.” Questions that do not meet the criteria may be used in creating the new tests “to the extent that [the preferred questions] are not available in sufficient numbers,” the settlement says.
The agreement also calls for ets to review the answers by blacks and whites on sample items that are to be included in all new tests. If these “pretested” items meet the standards, ets would be required to try to include them in subsequent versions of the Illinois insurance examination.
New tests will be written in this fashion annually and they will be monitored by an advisory panel that is to include representatives of both sides in the suit. Test results will be published annually.
Earlier Demand
In an early stage of negotiation, Golden Rule had sought to have all test answers that did not meet the new standard removed from the examination, those familiar with the case said. The firm’s later offer to compromise, according to Mr. von Mayrhauser, prompted ets to agree to settle out of court.
ets estimates, he said, that the “overwhelming majority” of the questions on the present Illinois examination meet the standards outlined in the settlement, which applies to no minority group other than blacks.
Under the agreement, which will remain in place for seven years, ets will not pay damages or lawyers’ fees, nor will it make any admission of guilt.
Mr. Rooney of Golden Rule said the company pursued the case out of an interest in “doing something socially responsible” and that it agreed to the out-of-court settlement because “what we got was the same as what we would have gotten if it went to trial; it was a superb outcome.”
Mr. von Mayrhauser added: “In light of the favorable agreement reached, we saw no good reason to devote large amounts of time, energy, and money to continue the litigation.”
