Do law reviews need reform? A survey of law professors, student editors, attorneys, and judges.
From their origins in the late nineteenth century, student edited law reviews have generated controversy. Yet, despite the hundreds of articles criticizing or defending law reviews, there is little research about what the legal community thinks about them. Consequently, we surveyed 1,325 law professors, 338 student editors, 215 attorneys, and 156 judges to determine their beliefs about the current system of law reviews and the need for reforms. Generally, law professors were the most critical and student editors the least critical of law reviews. Respondents identified several problems with law reviews. They believed that law review articles are too long. The law professors were dissatisfied with the effects of law reviews on their careers. The law professors, attorneys, and judges in the survey believed that law reviews are not meeting the needs of attorneys and judges. The vast majority of respondents indicated that reforms are needed. Surprisingly, all four groups of respondents agreed on the three most important reforms needed to improve law reviews. All four groups rated blind reviews, peer review, and better training of students as the most important reforms. In this article, we review the controversy about law reviews, present the results of our study, and discuss its implications.
I. INTRODUCTION II. CRITICISMS OF LAW REVIEWS A. SELECTION OF ARTICLES B. EDITING OF ARTICLES C. EFFECTS ON LEGAL SCHOLARSHIP, THE LAW, AND THE LEGAL COMMUNITY III. ARGUMENTS FAVORING THE CURRENT SYSTEM OF LAW REVIEWS A. BENEFITS TO STUDENTS B. BENEFITS TO LAW PROFESSORS C. BENEFITS TO LAW REVIEWS IV. PRIOR EMPIRICAL STUDIES OF LAW REVIEWS V. STUDY'S METHODOLOGY A. RESPONDENTS B. QUESTIONNAIRE C. DATA ANALYSIS VI. RESULTS A. SELECTING ARTICLES FOR PUBLICATION B. EDITING ARTICLES C. FACTORS THAT MAY HARM LAW REVIEWS D. SATISFACTION WITH LAW REVIEWS E. WHAT REFORMS SHOULD BE IMPLEMENTED F. MOST IMPORTANT REFORMS G. OTHER RESULTS VII. DISCUSSION A. LEGAL PROFESSIONALS AND STUDENT EDITORS' RESPONSES B. PROBLEMS WITH LAW REVIEWS C. REFORMING LAW REVIEWS D. LIMITATIONS E. CONCLUSIONS
Law reviews and law journals (hereafter referred to as law reviews) are vital to the law. They are the primary repositories of legal scholarship. (1) Accordingly, they influence how attorneys argue cases, how judges decide cases, what regulations administrative agencies adopt, and what laws legislatures enact. (2) They also have a significant impact on law professors' careers. (3) They help determine who is hired as a law professor, which law school hires them, whether law professors receive tenure and promotions, what compensation they receive, and what influence their research has on legal scholarship and the law. (4) In addition, law reviews are an important component of legal education. (5) Law review editors select and edit articles; engage in legal analysis, research, and writing; interact with legal scholars; and manage an important legal enterprise. (6) Law reviews also improve their members' job prospects and provide law schools with publicity and prestige. (7)
Unlike other scholarly journals, the vast majority of law reviews are run by students rather than professors or professional editors. (8) This characteristic has generated a major controversy about law reviews almost from their inception in the late nineteenth century. (9) Because of the controversy about law reviews, hundreds of articles have been written about them over the years. (10) Moreover, the debate has intensified in recent decades. (11)
For example, in 1992, the American Bar Association conducted a study about legal education and professional development that concluded practitioners increasingly viewed law review articles as irrelevant and believed most law professors pursued their own interests rather than researching topics that benefited the bar. (12) In 1994, the University of Chicago Law Review published a series of essays that debated the merit of student-edited law reviews. (13) The next year, the Stanford Law Review held a conference about law reviews that included editors from the top law reviews and prominent judges, attorneys, and law professors. (14) At the conference, they discussed areas of conflict between student editors and the legal community such as the selection and editing of articles and student management of law reviews. (15) They also discussed issues pertaining to the internal operations of law reviews. For example, how they are administered, produce articles, and select notes and members. (16)
In 2002, former Solicitor General Seth Waxman commented, "at the Supreme Court, academic citations are viewed as largely irrelevant--only a true naif would blunder to mention one at oral argument." (17) In 2004, the Harvard Law Review surveyed approximately 780 law faculty and determined that more than 85% of them thought that law review articles were too long. (18) In 2009, Aaron Twerski, a former dean of Hofstra University School of Law, stated that law review articles are increasingly irrelevant to attorneys and judges because new professors are discouraged from publishing traditional doctrinal articles. (19) The following year, Chief Justice John Roberts said that he pays little attention to law reviews because they are not "particularly helpful for practitioners and judges." (20) His remarks were the culmination of over two decades of judicial criticisms of law reviews from influential federal judges such as Richard Posner and Harry Edwards. (21) Despite the long-standing controversy about law reviews, their importance to the law, and the plethora of articles that have been written about them; there has been little research (22) about what the legal community thinks about them. (23) This question is critical to evaluating how well law reviews meet the needs of the law and legal education, and whether they require reform, because the opinions expressed about them in law review articles and elsewhere may not be representative of the legal community. Consequently, we surveyed law professors, student editors, attorneys, and judges to determine what they think about the current system of law reviews, the need for reforms, and what reforms should be implemented. We received an excellent response to the survey. A total of 1,325 law professors, 338 student editors, 215 attorneys, and 156 judges participated.
The remainder of this article is divided into six parts. Part II discusses common criticisms of law reviews. Part III presents arguments supporting the current system of law reviews. Part IV summarizes prior research on what the legal community thinks about law reviews. Part V describes the questionnaire used in our study and the study's methodology. Part VI reports the results of the study. Part VII examines the results and their implications for law reviews.
II. CRITICISMS OF LAW REVIEWS
Criticisms of law reviews can be divided into three categories: (1) their selection of articles; (2) their editing of articles; and (3) their effect on legal scholarship, the law, and the legal community.
A. SELECTION OF ARTICLES
Critics have advanced several reasons why they believe law reviews do a poor job of selecting articles for publication. First, students lack sufficient knowledge, experience, training, expertise, and time to select the best articles. (24) These deficits are exacerbated because a law review's entire staff turns over every two years, not only preventing the staff from gaining sufficient knowledge and experience to select the best articles, (25) but also causing a lack of institutional memory and long term plans, (26) which further hinders article selection. Because student editors generally do not use peer review, (27) and rarely consult law faculty (28) when selecting articles, they have no means of compensating for their deficiencies in article selection. Instead, critics maintain that they frequently rely on an author's reputation and law school affiliation when choosing articles because they cannot assess an article's quality. (29) According to critics, another negative consequence of students' lack of knowledge and experience is that authors have to write long introductions to place their articles in context. (30) The long introductions not only add significantly to the length of articles and the time required to evaluate and edit them, but also complicate article assessment. (31) Critics also contend that law reviews frequently use an article's length as an indicator of quality, which further impairs article selection. (32)
Second, more interdisciplinary articles are being submitted to law reviews. (33) Critics maintain that the evaluation of interdisciplinary articles requires both legal expertise and expertise in another discipline, a capability that student editors generally lack. (34) Third, the huge volume of articles submitted, especially to the more elite law reviews, hurts article selection. (35) For example, the elite law reviews receive thousands of articles every year even though they can publish only a small number of articles. (36) Multiple, simultaneous submissions of articles, a practice generally prohibited by other scholarly journals, (37) are primarily responsible for the large quantity of articles submitted to law reviews. (38) The creation of ExpressO, an electronic service that permits the simultaneous submission of an article to more than 750 law reviews, (39) has increased the problem in recent years. (40) Expedited reviews of articles (41) and "trading up" (42) to a higher ranked law review also aggravate the problem of law reviews having too many articles to evaluate. As a result, a student editor often spends little time reading an article before rejecting it. (43)
Fourth, critics assert that law reviews tend to publish articles about a limited range of topics. Critics claim law review editors favor articles about "hot, trendy, or cute topics," or topics that personally interest them. (44) Critics further allege that law reviews tend to be conservative--avoiding articles that are original and innovative. (45) Consequently, law reviews frequently do not select the best articles for publication because the best articles often do not concern hot, trendy topics, are not interesting to the student editors, or are original and innovative. Next, the poor quality of many submissions to law reviews makes article selection more difficult. Articles submitted to law reviews are frequently poorly written and researched. They often contain numerous grammatical and spelling errors, as well as incorrect citations and citation forms, because many law professors expect students to correct these deficits for them. (46) Because of these deficits, law review editors not only have to evaluate the quality of an article's arguments, but also the work required to prepare it for publication. (47) Lastly, law reviews do not always select the best articles because student editors often feel pressured to publish articles from their own law school faculty, (48) or to select an article for publication to help a junior professor at their law school. (49)
B. EDITING OF ARTICLES
Perhaps the most acerbic criticisms have been directed towards law reviews' editing of articles. (50) Critics assert the following about student editing of law review articles: Student editors lack the knowledge and experience to edit articles and often do not understand the articles they are editing. (51) Moreover, the constant turnover of student editors exacerbates the problem. It prevents students from accumulating the knowledge and experience they need to be good editors, and from developing a consistent approach to editing. (52) Students over-edit articles, sometimes even engaging in line by line editing that can fundamentally change the character of the article they are editing and reduce its quality. (53) They rigidly adhere to stylistic rules and manuals, even when it makes no sense to do so, because they lack the experience and training to recognize good writing. (54)
Student editors frequently "treat authors badly" and are not respectful of their time, often subjecting them to multiple, extensive edits of their articles (55) and imposing absurdly short deadlines for reviewing revisions of their articles. (56) Student editing increases the length and the number of footnotes of articles, which discourages attorneys and judges from reading them. (57) Law reviews slavishly adhere to Bluebook requirements, (58) demanding citations even for the most obvious fact, (59) which stifles creativity and originality (60) and encourages the piling on of footnotes that contain meaningless minutiae. (61) Finally, student editors are slow in publishing articles (62) and do not provide authors with feedback explaining why their articles were rejected and what they can do to improve them. (63)
C. EFFECTS ON LEGAL SCHOLARSHIP, THE LAW, AND THE LEGAL COMMUNITY
Law reviews have been criticized for producing too much legal scholarship that is useless and of poor quality. (64) Critics allege not only that the diminishing quality of law reviews significantly decreases attorneys' and judges' use of them, but even law professors use them less frequently. (65) For example, a 2007 study found that 43% of articles had never been cited even once by another law review article or by a court. (66) As one critic states: "There is powerful evidence ... that most of the legal academy are not even talking to each other but to the mirror." (67)
Critics also complain there are too many law reviews (68) and that their excessive numbers decrease article quality, are a "colossal" waste of law school resources, (69) and make it difficult for attorneys and judges to find relevant articles. (70) Law reviews have also been disparaged for their sameness (71) and for their imperviousness to market forces, (72) which allow them to flourish despite their poor quality and their failure to attract an audience. (73)
Most importantly, however, critics have attacked law reviews for their alleged failure to meet the needs of the legal community. For example, critics assert the following about the effects of law reviews on their student members: The costs of being on law review outweigh the benefits. (74) Too much time on law review is spent in "mindless scut work," such as learning the details of the Bluebook, source checking, and line editing--tasks with limited utility to students. (75) The pedagogical value of law reviews has substantially diminished over the years because of the large increase in submissions, the increasing complexity of peripheral tasks such as choosing new members, and the decreasing involvement of faculty in law reviews. (76) In addition, law review consumes so much time that it significantly decreases members' study time and causes them to cut classes. (77) There are better uses of student time than serving on law reviews, such as participating in legal clinics, simulated law practice, internships, and faculty directed writing courses. (78) Lastly, critics contend that even if law reviews did benefit their members, their value would still be limited because only a small percentage of law students can serve on them. (79)
Law reviews have also been criticized for their effects on law professors. As stated previously, article placement has important consequences for law professors' careers. (80) Yet critics assert that articles selection is not primarily based on merit but rather on author prestige and other extraneous factors. (81) The lack of impartiality in article selection especially harms new law professors, particularly ones at non-elite law schools; female law professors; and foreign law professors. (82) Critics maintain that the current system of law reviews encourages law professors to be lazy and not produce their best work because they know student editors will correct deficiencies in their articles for them; (83) quality frequently does not determine article placement; (84) and law reviews are so numerous that any article can be published somewhere. (85) Critics further claim that because student editors have limited legal knowledge and editing experience, professors are forced to write long introductions for their articles and to waste their time correcting poor editing, which decreases the number of articles professors can publish. (86) Lastly, because law reviews favor certain topics and tend to be conservative, they limit the subjects law professors can research and write about. (87)
Critics have also attacked law reviews for their alleged failure to meet the needs of judges and attorneys. They state that most law review articles have become too theoretical, too abstract, and too devoid of doctrinal analysis to be useful to attorneys and judges. (88) Critics further assert that other traditional sources of doctrinal analysis such as treatises, casebooks, and student notes are inadequate to fill the void. (89) Furthermore, attorneys and judges frequently lack the time, resources, neutrality, and expertise to adequately analyze legal doctrine by themselves, and therefore cannot replace the analysis of legal doctrine that law reviews have traditionally provided. (90) Moreover, attorneys' and judges' deficiencies in legal analysis are exacerbated by the increasingly complex and difficult legal issues they face. (91)
Critics contend that the failure of law reviews to meet the needs of attorneys and judges will not only have serious, long term, adverse consequences for the law, but also for law professors and law schools. They point out that law reviews' lack of useful scholarship diminishes the prestige of law professors and law schools. (92) Because law schools depend on attorneys and judges for financial support and to give their graduates jobs, their opinions are important. (93) In addition, U.S. News and World Report rankings are based in part on attorneys' and judges' assessment of a law school's reputation. (94) As David Hricik and Victoria Salzmann state: "In short, what the bench and bar think about a law school matters, and the usefulness of a school's scholarship is a factor that the bench and bar consider when forming their opinions." (95)
III. ARGUMENTS FAVORING THE CURRENT SYSTEM OF LAW REVIEWS
Although fewer in number, law reviews have their defenders as well as their detractors. (96) The defenders maintain that student-run law reviews benefit students, law professors, and law reviews.
A. BENEFITS TO STUDENTS
Most defenders of law reviews believe that the most important benefit of student-run law reviews is the educational experience that they provide to students. (97) Service on law reviews improves students' legal reasoning, writing, editing, research, and citation skills. (98) It teaches students the importance of attention to detail and accuracy in the law and increases their legal knowledge. (99) Students on law review learn about the latest trends in legal scholarship and have an opportunity to personally contribute to legal scholarship and the law by writing a note or comment. (100) Students on law review interact with legal scholars. For instance, they may work closely with a faculty member when writing a note or comment. (101) They may consult with a faculty member when deciding whether to accept an article for publication. (102) They have extensive communications with legal scholars who publish articles in their law review. (103) In short, law reviews allow students to closely interact with legal scholars, and in a context in which legal scholars care not only about the process but also about the outcome. (104)
Students on law review interact and work with their peers on a daily basis. They learn from and teach their peers; argue, defend, and exchange ideas with their peers; give and receive criticisms from their peers; and manage an important legal enterprise with their peers. (105) As Judge John Noonan stated, peer education may be the "best kind of education." (106) Lastly, law review membership is a source of pride for most students and gives them an impressive credential that improves their job prospects, especially for elite positions such as a law professor, a federal judicial clerkship, or an associate in a large law firm. (107)
B. BENEFITS TO LAW PROFESSORS
Student-run law reviews benefit law professors in several ways. They give law professors more time to publish, teach, do pro bono work, and spend time with their families. (108) Because student-run law reviews allow law professors to pursue more of their interests, they help attract highly qualified candidates to law schools who might otherwise prefer the more lucrative private practice of law. (109) They also vastly increase the number of law reviews and therefore ensure that any worthwhile article can be published somewhere. (110) Students make possible the multiple, simultaneous, submission of articles to law reviews and the "trading up" of articles to more prestigious law reviews. (111) Accordingly, law professors, unlike other scholars, do not have to submit their article to one journal at a time, wait months to determine if their article is accepted, and then publish the article with a journal if it is accepted, (112) Furthermore, student-run law reviews free law professors from the drudgery of selecting and editing articles, serving as peer reviewers, and ensuring that their citations are correct and in proper form. (113)
C. BENEFITS TO LAW REVIEWS
Defenders of law reviews maintain that law reviews benefit from having students run them. Students are more open to new ideas, theories, and perspectives than law professors and are less likely than law professors to require authors to conform to "methodological and intellectual orthodoxies." (114) Because law students are much more numerous than law professors and are not compensated for their time, student-run law reviews permit law schools to publish a large number of journals, which ensures that a wide array of legal ideas, theories, and perspectives are published. (115) Students tend to select articles they understand and edit them to improve their clarity and readability. (116) Consequently, they force law professors to write articles that are less abstract and theoretical, which makes articles more useful to attorneys and judges than if professors edited law reviews. (117)
Defenders also contend that readers benefit from the long introductions and the large number of footnotes that student-run law reviews require. The long introductions place articles in context and therefore make law review articles comprehensible to a broad audience. They also help readers learn about unfamiliar areas of the law. (118) The large number of footnotes ensures that law review articles are well researched, (119) provide a measure of article quality, (120) and help readers research the law. (121)
Defenders of law reviews assert that students are capable of selecting articles for law reviews despite their lack of legal knowledge and experience. (122) They claim that it is often easy to differentiate good articles from mediocre or poor articles, and, therefore, article selection does not require an expert. (123) Moreover, a law review article must be comprehensible to a diverse audience. (124) Consequently, if bright student editors cannot understand an article then it is not suitable for publication.
Defenders of law reviews also point out that some subjectivity is always present in article selection. (125) Because article selection is not a completely objective process, the goal of student editors is not to select the "best" articles for publication, but rather to select high quality articles that are timely, interesting, and appropriate for their law review. (126) These goals are much more manageable for students than selecting the "best" articles and are achievable through hard work, research, and preemption checks. (127) Law reviews also compensate for the subjective nature of article selection by striving to collect "a portfolio of strong varied articles" that appeals to a wide audience. (128) Thus, even if students err in selecting some articles for publication, the errors are unlikely to significantly affect the overall quality of the law review's "portfolio." (129)
In addition, defenders of law reviews maintain that the importance of students' selection of articles has diminished over the years because of the large increase in the number of law reviews, which makes it possible for any "meritorious" article to be published somewhere, (130) Moreover, once an article is published, it is likely to be available on a large number of sites such as Social Science Research Network (SSRN), the author's blog, the author's webpage, the law review's webpage, as well as on Westlaw, LexisNexis, Findlaw, and Google. (131) Because of the wide accessibility of articles today, the impact an article has on legal scholarship and the law is more likely to be determined by its merit rather than by which law review published it. (132) In short, the information age has greatly diminished the importance of article selection. (133) Finally, article selection should not affect tenure decisions, because law schools should evaluate professors on the quality of their articles and not on their placement of articles. (134)
Defenders of law reviews also contend students are capable of editing articles for the following reasons: editing is expensive; when editors of peer review journals accept an article, they do little to improve its writing quality, do little proofreading of the article, and do no cite checking. (135) Consequently, the use of professional editors would not improve the editing of law review articles. Furthermore, most law review articles undergo an elaborate vetting process by an author's colleagues and friends before it is submitted for publication. (136) Accordingly, most law review articles do not require extensive editing.
Student editors are adults with college degrees who generally have good verbal skills and therefore are capable of editing articles. (137) "[M]any of the greatest editors were not as talented as the writers they edited." (138) Consequently, student editors are capable of editing law professors' articles even if we make the dubious assumption that they are not as talented as law professors. (139)
Law reviews have created safeguards to compensate for students' lack of legal knowledge and editing experience. (140) For example, law reviews frequently use multiple editors. (141) Moreover, in each round of editing, several students are involved in the editing process and a student editor evaluates the suggested edits before sending them to the author. (142) Another safeguard is that many law reviews permit authors to decide which edits to accept or reject. (143) Multiple editing may be inefficient and frustrating, but it gives authors different perspectives on how to improve their articles and generally produces a well-edited article. (144) Student editing also has strengths that compensate for its weaknesses. (145) Because law reviews use students rather than law professors, they have large staffs that can carefully review and edit accepted articles and can ensure that articles' citations are correct and in the proper form before publishing them. (146)
Having examined the arguments of critics and defenders of law reviews, we discuss in the next section prior studies of law reviews.
IV. PRIOR EMPIRICAL STUDIES OF LAW REVIEWS
There are two earlier empirical studies about law reviews. In 1992, Max Stier, Kelly Klaus, Dam Bagatell, and Jeffrey Rachlinksi surveyed 90 law professors, 166 attorneys, and 124 judges. (147) They determined how often respondents used law reviews, whether respondents believed law reviews successfully accomplished their goals, and what changes they believed law reviews should make. (148) They found that law professors frequently used law reviews, primarily for academic purposes; while attorneys and judges used them much less frequently and primarily for practical purposes. (149) Stier et al. also determined whether law review membership helped students obtain employment. Respondents to the survey believed that law review membership was a particularly useful credential for obtaining elite jobs such as federal judicial clerkships and law professor positions. (150) Next, Stier et al. inquired whether law reviews benefit students. Most respondents believed that law reviews improved students' research, writing, and editing skills but were of limited value in improving their legal knowledge. The majority of respondents reported that they were satisfied with their own law review experience. (151)
Stier et al. asked respondents if law reviews were successful in achieving the following goals:
stimulating academic interest; suggesting theoretical frameworks for analysis; evaluating the effectiveness of existing law or alternatives; providing a general overview of existing law; tracking current developments in general practice areas; identifying new approaches toward legal topics; finding cases or support for specific positions in legal documents; and training students as writers, editors, and researchers. (152)
Respondents generally believed that law reviews were somewhat successful in achieving all these goals. (153) Stier et al. then inquired what respondents thought of the different types of material that law reviews publish. Respondents uniformly believed that full-length articles were the most useful type of law review publication. (154)
Stier et al. also asked whether law reviews should implement the following reforms: shorten articles, make them less theoretical, and decrease the number of footnotes. All three groups of respondents favored these reforms. (155) Next, they requested respondents to rate how much attention law reviews should devote to twelve legal topics. The responses of the three groups of participants varied widely, though all three groups wanted more articles about legal ethics, corporate and commercial law, and tort law. (156) Lastly, Stier et al. ascertained whether respondents thought students should select and edit articles for law reviews. All three groups of respondents strongly favored students selecting and editing articles for law reviews, though their support was stronger for student editing than for student selection of articles. (157) In addition, many respondents who favored student editing and selection of articles wanted greater faculty supervision and participation in law reviews. (158)
Stier et al. concluded about their survey:
Henceforth, the critical literature ought to be informed by empirical data instead of personal anecdotes. While this survey constitutes but a first step toward a more meaningful debate, it does support several conclusions. Our results suggest that radical change is neither necessary nor desired: Student selection and editing of law review articles are quite popular among all segments of the legal community, and the members of that community find the selected articles themselves to be useful. (159)
In 2004, the Harvard Law Review conducted an online "Usage Study" of approximately 780 law professors. (160) Besides determining that more than 85% of the respondents thought law review articles are too long or somewhat too long, the survey also found the following: (161) The two most common reasons law professors in the survey used law reviews were for "keeping abreast of developments" in their fields and for "specific research questions." (162) The two most frequent methods law professors employed to identify relevant law review articles were "online database research" and "from a citation in another article." (163)
A total of 69.4% of the law professors in the survey were at least somewhat satisfied with student editing of articles. (164) Of the law professors in the survey, 40.9% believed they received the right amount of feedback during the editing process. (165) In addition, 64.9% of the law professors found the quality of the editing feedback very useful, useful, or somewhat useful. (166) The survey also addressed the quality of law review publications. A total of 40.2% of the law professors thought that the "current scholarship" in law reviews was excellent or good with a plurality of 43.2% of professors believing that it was fair. (167) As for the quality of student writing in law reviews, 35.3% of the law professors thought it was excellent or good, and another 48.5% of professors thought it was fair. (168)
While these prior studies offer some useful information about whether the legal community is satisfied with law reviews, whether law reviews require reforms, and what reforms should be implemented; the information they provide is limited, incomplete, and outdated. Moreover, they do not include the beliefs of one of the primary stakeholders in law reviews: student editors. Accordingly, we conducted our own survey to more comprehensively answer these critical questions.
V. STUDY'S METHODOLOGY
We surveyed law professors, student editors, attorneys, and judges to assess what they think about law reviews, whether reforms are needed, and what reforms should be implemented.
The questionnaire for the survey was posted on SurveyMonkey. An email with a link to the survey was sent to most student law reviews listed on the website of Washington and Lee's Law Journals: Submissions and Ranking. Individual emails with a link to the survey were sent to law professors listed on the websites of approximately 90% of the 200 ABA approved law schools. A few law school websites did not list professors' email addresses, and some law school websites made it difficult to contact their law professors by email. Attorneys were recruited from firms' webpages and from members of a state bar association and a state attorneys' association. Individual emails with a link to the survey were sent to attorneys who were members of firms. Group emails with a link to the survey were sent to the attorneys who were the members of the legal organizations.
We used the American Bench to determine judges' email addresses. We sent individual emails with a link to the survey to federal and state judges from Alabama, California, Colorado, Indiana, Montana, New York, South Carolina, Tennessee, and Washington. Many judges who are listed in the American Bench do not publish their email addresses. The above states were selected because they had the most judges who listed their email addresses. Nonetheless, even in the selected states, many judges did not list an email address, and in some cases the email addresses were incorrect or out of date. Therefore, not all federal and state judges in the aforementioned states could be contacted about the survey. All emails were sent in 2010. One email with a link to the survey was sent to the respondents except for the student editors, who received two emails.
A total of 1,325 law professors, 338 student editors, 215 attorneys, and 156 judges took the survey. Of the 1,325 law professors who participated in the survey, 866 (65%) were tenured, 383 (29%) were not tenured, and 76 (6%) did not indicate whether they were tenured. Of the 215 attorneys who participated in the survey, 96 (45%) were members of law firms, 29 (13%) were solo practitioners, 15 (7%) were corporate counsels, 10 (5%) were state prosecutors, 6 (3%) worked for a corporation, 4 (2%) were federal prosecutors, 3 (1%) were private defense attorneys, 3 (1%) were federal defense attorneys, 2 (1%) were state defense attorneys, 44 (20%) listed "other" for their current position, and 3 (1%) did not list their current employment. The law firms of the attorneys who participated in the survey ranged in size from 1 attorney to a 1,000 attorneys.
Of the 156 judges in the survey, 146 (94%) were state judges, 6 (4%) were federal judges, and 4 (3%) judges did not indicate whether they were a state or federal judge. A total of 133 (85%) judges were trial court judges, 19 (12%) were appellate judges, and 4 (3%) did not indicate what type of judge they were. The percentages for each of the four groups of participants were rounded off and therefore do not total 100%.
The questionnaire took about 20-30 minutes to complete and consisted of seven sections. There was some missing data, but data from all respondents was included even if they did not complete the questionnaire. The first author, who is both an attorney and a psychologist, and has published several law review articles, drafted the questionnaire. Several law professors, student editors, and psychologists reviewed and suggested revisions for the questionnaire prior to its distribution. In Section I of the questionnaire, respondents answered 8 statements pertaining to the quality of law reviews' selection of articles for publication. In Section II, respondents answered 4 statements about law reviews' editing of articles. Section III contained 8 statements about factors that may affect the ability of law reviews to publish the best articles.
Section IV contained 7 statements that related to respondents' satisfaction with the current system of law reviews. Section V contained statements about 11 proposed reforms of law reviews. If a respondent favored a reform, in some instances there were follow-up questions to better ascertain the nature of the reform that the respondent favored. Respondents could recommend reforms that were not listed in the questionnaire. In Section VI, respondents also indicated 3 of the 11 proposed reforms they thought were the most important to implement to improve law reviews. Respondents could also answer that no reforms were needed, or write in a reform that was not listed. If they answered that no reforms were needed, they were instructed not to choose any of the proposed reforms or write in a reform.
Section VII requested demographic information and differed depending on whether the respondent was a law professor, student editor, attorney, or judge. Except for the follow-up questions, the question about the three most important law review reforms, and the demographic questions, the respondents used 7-point Likert-type scales to rate the statements in the questionnaire. The 7-point Likert-type scales had labels of 1 = strongly disagree, 4 = neutral, and 7 = strongly agree.
C. DATA ANALYSIS
We primarily used one-way independent ANOVAs to analyze the data because there were four groups of respondents. ANOVAs determine if there are significant differences between three or more groups. (169) Because ANOVAs only determine that there is a difference in group means, but do not specify which groups differ, we also conducted post hoc tests (i.e., follow-up tests) to determine which groups differed. (170) We used two types of follow up tests: the Games-Howell test when the population variances were not equal, and the Hochberg's GT2 when the population variances were equal. (171)
A. SELECTING ARTICLES FOR PUBLICATION
The 8 statements in Section I of the questionnaire evaluated whether respondents believed that law reviews do a good job in selecting articles for publication. (172) The respondents used 7-point Likert-type scales with labels of 1 = strongly disagree, 4 = neutral, 7 = strongly agree to evaluate the 8 statements in Section I. Table 1 and Figure 1 below summarize the results. (173)
To obtain a global assessment of respondents' beliefs about the quality of article selection by law reviews, we combined the 8 statements in Section I into a "selection scale." (174) There was a significant difference in the groups' scores on the selection scale. (175) A follow-up test showed that the law professors' (M = 3.11, SD = 1.07) (176) scores on the selection scale were significantly lower than the attorneys' (M = 4.08, SD = 1.02), judges' (M = 4.36, SD = .92), and student editors' scores (M = 4.57, SD = 1.07) (all ps < .001). The attorneys' scores on the selection scale were significantly lower than the student editors' (p < .001) and the judges' scores (p < .05). The judges' and the student editors' ratings on the selection scale did not significantly differ (p > .05).
Not only did the law professors have the lowest score on the selection scale, but they also gave significantly lower ratings than the other three groups for all 8 statements (177) in Section I (all ps < .01). The law review editors gave significantly higher ratings than the other three groups on four of the 8 statements in Section I: statements 1 (an article's contribution to legal scholarship), 3 (quality of an article's reasoning), 4 (an article's writing quality), and 8 (overall article selection quality) (all ps < .05). Attorneys and judges did not have significantly lower or higher ratings than the other groups on any of the 8 statements in Section I.
As indicated in the introduction, two of the major criticisms of law reviews' selection practices are that law reviews frequently select articles on the basis of the author's credentials and law school affiliation rather than on article quality (statement 6) and that they do not give all articles adequate consideration before making a publication decision (statement 7). (178) The law professors agreed (M = 6.01, SD = 1.26) that law reviews place too much emphasis on an author's reputation and law school affiliation in selecting articles.
We determined whether the ranking of a professor's law school affected his or her answer to this statement. We divided law schools into the Top 15, Top 15-25, Top 26-50, Top 51-100, Top 101-145, and unranked law schools. (179) There was a significant difference in professors' responses to this statement depending on their law school's ranking, (180) Professors at the Top 15 law schools gave a significantly lower rating to this statement (M = 5.07, SD = 1.42) than professors at all the lower ranked law schools. Professors at the Top 16-25 rated law schools (M = 5.70, SD = 1.38) also gave a significantly lower rating to this statement than professors at the unranked law schools (M = 6.28, SD = 1.24) (p < .05), but their ratings did not significantly differ from ratings of the professors at the other lower ranked schools. Professors' ratings of this statement at the Top 26-50 (M = 5.99, SD = 1.25), Top 51-100 (M = 6.14, SD = 1.07), Top 101-145 (M = 6.18, SD = 1.20), and unranked law schools did not significantly differ. In short, though professors at the top ranked law schools believed that law reviews place too great an emphasis on a law professors' reputation and law school affiliation in article selection, they believed it was significantly less of a problem than did the professors at lower ranked schools.
The student editors (M = 4.92, SD = 1.75) also agreed to a lesser extent that law reviews place too much emphasis in article selection on an author's reputation and law school affiliation.
The student editors' responses to this statement were significantly higher than the responses of the attorneys (M = 4.61, SD = 1.53) and the judges (M = 4.24, SD = 1.43), whose responses did not significantly differ (p > .05).
The law professors further agreed (M = 5.30, SD 1.51) that law reviews do a poor job of giving all articles adequate consideration before making a publication decision (statement 7). The law professors' responses to statement 7, however, varied somewhat depending on their law school's ranking. (181) Law professors at the Top 15 (M = 4.83, SD = 1.64) ranked law schools gave significantly lower ratings to this statement than professors at the Top 26-50 ranked law schools (M = 5.42, SD = 1.60) (p < .05) and at the unranked law schools (M = 5.41, SD = 1.48), but not than professors at the Top 16-25 (M = 4.97, SD = 1.60), Top 51-100 (M = 5.35, SD = 1.63), or Top 101-145 (M = 5.37, SD = 1.44) ranked law schools (all ps > .05). Law professors' ratings of this statement at the Top 16-25, Top 26-50, Top 51-100, Top 101-145, and unranked law schools did not significantly differ (all ps > .05). (182) The student editors, attorneys, and judges were neutral in their responses to whether law reviews give adequate consideration to all articles before making a publication decision (all Ms between 3.83 and 4.16). Their responses to this statement did not significantly differ (all ps > .05).
The law professors expressed two other major concerns about article selection, (183) They generally agreed that law reviews do a poor job evaluating an article's contribution to legal scholarship (M = 3.07, SD = 1.56) (statement 1) and assessing how original, creative, and innovative an article is (M = 2.97, SD = 1.54) (statement 5). Law professors' ratings of these two statements did not vary by their law school's ranking (p >.05), and as previously stated, their ratings were significantly lower than the ratings of the other three groups for both statements (all ps < .001). In contrast, the student editors (M = 4.83, SD = 1.48) gave significantly higher ratings than the other three groups on law reviews' ability to evaluate an article's contribution to legal scholarship (p < .05). They also gave significantly higher ratings (M = 4.59, SD = 1.55) than the law professors and attorneys (M = 4.17, SD = 1.37) (p < .01) to law reviews' ability to determine how original, creative, and innovative an article is. Their ratings on this statement, however, did not differ significantly from the judges' ratings (M = 4.56, SD = 1.29) (p > .05).
In summary, the law professors believed that law reviews need to do a better job of selecting articles for publication. The attorneys were generally neutral about this important issue. The student editors and, to a lesser extent, the judges, generally had a more favorable view of the quality of law reviews' article selection than the other two groups.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Abstract through VI. Results A. Selecting Articles for Publication, p. 1-43|
|Author:||Wise, Richard A.; McGough, Lucy S.; Bowers, James W.; Peters, Douglas P.; Miller, Joseph C.; Terrell|
|Publication:||Loyola Law Review|
|Date:||Mar 22, 2013|
|Previous Article:||Great Lakes Reinsurance V. Durham Auctions: why the lack of clearly defined fifth circuit "maritime choice of law principles" has the potential to...|
|Next Article:||Do law reviews need reform? A survey of law professors, student editors, attorneys, and judges.|