April 2000 

Classifieds
Letters
Display Ads
Archives
Article Index
Apr '00 Issue
Latest Issue
MSBA Home Page

The Board of Law Examiners recently conducted studies of the Bar Examination and the minimum passing score, and concluded that the minimum passing score should be raised ten points, to 270, in two-point increments over each of the next five years. It reported its conclusion to the Supreme Court and requested its approval for the change. The Board and the Court have invited comments on the suggested change, a hearing was set for March 9, 2000, and the matter remains under consideration at this writing.

Reproduced below are excerpts from communications of the Board and its testing consultant, explaining their studies and recommendations; and a response from the deans of Minnesota’s four law schools, expressing their views and concerns about the proposal.

The Rules of the Supreme Court for Admission to the Bar authorize the Minnesota State Board of Law Examiners to examine the fitness of applicants to practice law, and to fix the minimum satisfactory grade for success on examinations. Neither these Rules nor the Rules of the Board of Law Examiners specify the form of the examination or the minimum passing score. These are established by other actions of the Board.

The current examination consists of two parts: the Multistate Bar Examination (MBE), a 200-question multiple-choice test; and an eight-question essay test. An applicant’s score on the MBE is the number of correct answers. Each essay answer is given a grade from one to seven, with higher numbers assigned to better answers. An applicant’s total essay raw score is the sum of the applicant’s scores on the eight questions. The total essay raw scores of all applicants are converted to a 200-point scale, with the same mean and standard deviation as the applicants’ MBE scores. An applicant’s total Bar Exam score is the total of the MBE and essay scores. A total score of 260 or higher is required to pass the examination.

Excerpts from communications of the Board of Law Examiners

Deans' commentary on the proposed increased in the passing score

"a higher standard will be an encourage-
ment to those who sit for the Bar to be better prepared"
--Board of Law Examiners

In January 1998, the Board of Law Examiners conducted a standard review study of the Minnesota Bar Examination. The "standard" under review was the passing score an applicant must achieve in order to be considered successful on the Minnesota Bar Exam. Since the 1970s, the Minnesota Bar Exam passing score has been fixed at 260 points out of 400 possible points on an exam that consists of eight essay questions (afforded 200 points) and a 200-point multiple-choice exam.

The study was undertaken because it had been 20 years since the Board had considered what the passing score should be. Best practices in testing require that a study of the pass-fail standard be conducted periodically in order to insure that that point is a valid reflection of the profession’s current standards. The Board’s purpose is to ensure that those who are admitted to the Bar possess the minimum level of legal competency so that clients, as well as the justice system are adequately protected.

Stephen Klein is a nationally recognized psychometric consultant with expertise in bar examining issues. He was retained by the Board to conduct the Minnesota study. Dr. Klein has conducted similar studies in other testing environments and is familiar with the unique characteristics of bar exams. He recently completed similar studies in Virginia and Ohio and is currently conducting a study for the state of Florida.

The July 1997 Minnesota Bar Exam supplied the data upon which the study was based. The July 1997 Minnesota Bar Exam was chosen because it had been recently graded, and all test and score data, as well as the graders themselves, were easily accessible.

The first part of the Minnesota study involved the use of "internal reviewers" -- eight attorneys who are experienced graders of the Minnesota Bar Exam.

The readers who graded the essay answers for all the July 1997 applicants were asked what raw score (on the seven-point scale) they felt most closely coincided with "passing." These standards were then compared to the score distribution on their assigned question. For example, if the reader said a score of three or higher was passing and if 70 percent of the applicants had a score of three or higher on the question that the reader graded, then the "passing rate" for that question was set to 70. The average of these rates over all the essay questions in the test was 75.8 percent. An inspection of the distribution of total scores found that a total score of 272 would produce a 75.8 percent passing rate; i.e., the average of the readers’ pass/fail standards was a 272.

The second and major part of the study involved assembling 36 external reviewers. The Board recruited attorneys to participate in the external review process who were representative of all aspects of the Bar. An effort was made to include among the reviewers judges, law professors, attorneys of color, women, attorneys from all areas of the state, and attorneys who had a variety of legal practice experience. In addition to the demographic mix, each of the individuals was selected because each is recognized as an expert in one of the content areas covered by the eight questions under review.

Each panelist was then given a set of 35 answers and asked to place each answer in one of the following four categories: 1 = clear fail; 2 = marginal fail; 3 = marginal pass; or 4 = clear pass. Panelists were told to make their judgments independently and that they could place as many papers as they wished in each category. There was no limit on the number they could classify as passing (or failing). All the members of a team evaluated the same set of 35 answers. These answers were spread evenly across the range of reader-assigned scores on the question (i.e., there were five answers from each of the seven possible score levels). The panelists did not see the score the regular reader assigned to each answer nor did they know the distribution of the reader scores in their set of 35 answers.

The panelists on a question usually agreed with each other as to whether an answer was passing or failing even though no effort was made to calibrate the panelists to a common standard. They also generally agreed with the regular readers regarding the relative quality of the 35 answers they graded in common. In other words, answers designated as passing by the panelists on a question (i.e., those that received a rating of three or four on their four-point scale) were more likely to receive relatively high scores from the regular readers while those designated as failing by the panelists tended to receive low reader scores. These relationships were used to estimate the reader score on a question that most closely coincided with an average panelist rating of 2.5 or higher on that question. The value of 2.5 was used for this purpose because it falls halfway between a "marginal fail" and a "marginal pass" on the panelists’ four-point scale.

The next step involved determining the percentage of the 644 applicants taking the test who had reader-assigned scores that were as high or higher than the panelists’ passing score on the question. This percentage was designated as the passing rate on the question. The average of these rates across the eight questions was 78 percent. In other words, if the average of the panelists’ standards was used as the pass/fail line, then 78 percent of the applicants would have passed the July 1997 exam. The panelists’ standard corresponds to a total score of 271, i.e., if the July 1997 Minnesota applicants had to achieve a total score of 271 or higher in order to pass, then 78 percent of them would have passed. A more statistically sophisticated method of analyzing the data indicated that the panelists’ standard was a total scale score of 270.

The foregoing estimates of the panelists’ overall pass/fail standard were derived from eight separate and completely independent mini-studies, i.e., each question was its own mini-study. Put another way, there were essentially eight replications of the study design. The average standard derived from these replications provides a reliable index of the panelists’ judgments of where the passing line should be drawn. Moreover, these judgments were based solely on the panelists’ evaluations of answer quality. The panelists were not asked to recommend a passing score or a passing rate and they could not determine what this rate would be for the standard they used.

To sum up, if the average of the regular readers’ standards was used to set the passing score, then this score would be 272. If the independent panelists’ standard had been used, then the passing score would be 270 or 271. In other words, the readers and panelists had almost identical standards.

The readers’ and panelists’ standards are much higher than the 260 Minnesota now employs. Most states have pass/fail standards that would correspond to Minnesota total scores of 270 to 272 (i.e., 135 to 136 on the MBE scale). Consequently, the standards employed by both the regular readers and the independent panelists are much more consistent with the standards that are now used in other states than they are with Minnesota’s current standard of 260. (emphasis in original)

These studies suggest that some examinees are passing the Minnesota Bar Exam on the basis of essay answers that do not, in the opinion of the panelists, show a minimum level of legal competency. In addition to considering these studies, the Board also surveyed the minimum passing scores set by other states in the United States and found that Minnesota’s minimum passing score of 260 placed Minnesota in the bottom 25 percent of all states.

Based upon all of the above, the Board recommended to the Supreme Court that Minnesota increase the passing score by two points each July for the next five years, resulting in a ten-point increase (from 260 to 270) by 2005. The gradual increase was recommended over five years because the Board wanted to give adequate notice of the rising standard.

Bar exam testing experts have advised the Board that raising the passing score will result in a gradual increase in the mean or average performance of all examinees. This occurs because a higher standard will be an encouragement to those who sit for the Bar to be better prepared. The Board’s hope is that they will not only be better prepared for the exam but better equipped to represent the most important legal interests of the public.

The Board is not recommending an increase in the passing score in order to reduce the number of attorneys who are being admitted in Minnesota. It is recommending this increase in order to assure that the passing standard reflects an accepted level of legal competence. By raising the score, the Board expects that examinees will have chosen classes in law school which will prepare them for the practice of law, even if those classes are not required for graduation. For example, although a course such as Evidence may not be required to graduate from law school, students who plan to be licensed would be more likely to select that course if they knew that the passing standard was being raised. The Board hopes that raising the passing standard will better serve the public by providing additional assurances that attorneys are well-prepared to provide the full range of legal representation that a law license permits.

"studies suggest that some examinees are passing the Minnesota Bar Exam on the basis of essay answers that do not, in the opinion of the panelists, show a minimum level of legal competency"
--Board of Law Examiners

"We are concerned that the proposed increase may disproportionally affect students of color and nontraditional students by either preventing or delaying their admission to the bar."
--Minnesota law school deans

We oppose the proposal by the State Board of Law Examiners to raise the score required to pass the Bar Exam by ten points. This change will have a substantial impact on individuals who have graduated from law school and are seeking admission to the bar and should be implemented only if necessary to address documented problems that have arisen under the current standards. The percentage of first-time takers who pass the Bar has not increased significantly in recent years and the Board has not presented any evidence that a competency problem exists among young lawyers. Even if such a problem existed, it would be incumbent upon the Board to demonstrate that the problem was caused by individuals with marginally passing scores on the Bar and that an increase in the score would alleviate the problem. This has not been done.

In addition to our belief that the Board has not documented any necessity for an increase in the passing score on the Bar Exam, we have the following specific concerns about the effect of such an increase:

• The timing of the proposal is ill-advised; it comes at the same time as the introduction of the "Practice Question" on the Bar Examination. The introduction of another significant change will make it difficult, if not impossible, to evaluate what effect, if any, either or both of these changes has produced.
• This significant change will be implemented at the very time the University of St. Thomas will be producing graduates who will be sitting for the Bar Examination. We need to consider whether the proposed increase will be viewed or could be viewed as an anti-competitive attempt to limit access to the profession.
• Approximately 10 percent (70-80 students) of applicants who now pass the Bar do so with a score that would not be a passing score under the proposed increase. We are concerned that the proposed increase may disproportionally affect students of color and nontraditional students by either preventing or delaying their admission to the bar. A disproportionate impact did occur in Ohio when it implemented a similar proposal.

• The vast majority of students who are unsuccessful on their first attempt to pass the Bar do pass the Bar eventually. However, delaying admission imposes substantial costs -- the costs of retaking the Bar, the costs of preparing for the Bar, and the opportunity costs associated with lost or deferred employment until the second Bar is completed -- which adversely affect those without substantial economic resources. This may cause some individuals likely to succeed on a second attempt to drop out of the process.
• The assertion that the increase will not result in fewer applicants passing the Bar has not proved true in Ohio, which recently implemented a similar proposal. Even if true, if the same individuals will be passing the Bar, why raise the score? By studying harder in order to attain a marginally higher score students will not become more competent attorneys, they will simply become better test takers.
• Under the current grading process, graders are not required to make a judgment as to absolute competence, but only assign scores on a relative basis. If there are marginal students who are passing the Bar but perhaps should not be, the Board could address this problem by a final check or filter in the grading process that reviews the exams with very low scores on the essay to determine, from an absolute quality standard, whether the exams should receive a passing score. The fact that an assessment of competency is not a part of the current grading procedure for the essay questions is startling and calls into question the premises underlying the proposal to increase the passing score.
• The Board has relied on the opinion of a single expert whose methodology has been criticized by other experts in the field. The Board should weigh the opinions of a variety of experts before imposing a substantial change on the Bar Exam process.

In summary, the Board’s proposal to increase the passing score on the Bar Examination is not a response to any documented problems resulting from the current standard. The score should not be raised simply because other states might be raising their score; other states may be addressing issues and concerns not present in Minnesota.

-- E. Thomas Sullivan
-- Harry J. Haynsworth
-- Edwin J. Butterfoss
-- David T. Link

"the Board has not presented any evidence that a competency problem exists among young lawyers."
--Minnesota law school deans