Is peer review in decline?I. INTRODUCTION
For the past half-century Noun 1. half-century - a period of 50 years
period, period of time, time period - an amount of time; "a time period of 30 years"; "hastened the period of time of his recovery"; "Picasso's blue period"
century - a period of 100 years or more, peer-reviewed journals have played a central role in the evaluation and dissemination dissemination Medtalk The spread of a pernicious process–eg, CA, acute infection Oncology Metastasis, see there of scientific research. The Internet Internet
Publicly accessible computer network connecting many smaller networks from around the world. It grew out of a U.S. Defense Department program called ARPANET (Advanced Research Projects Agency Network), established in 1969 with connections between computers at the has enhanced scientific communication in many ways, and there is considerable excitement around new institutions for disseminating dis·sem·i·nate
v. dis·sem·i·nat·ed, dis·sem·i·nat·ing, dis·sem·i·nates
1. To scatter widely, as in sowing seed.
2. research. (1) A more sobering thought, however, is that new technologies can also be disruptive disruptive /dis·rup·tive/ (-tiv)
1. bursting apart; rending.
2. causing confusion or disorder. .
This paper begins with some documentation of recent trends in economics publishing. Roughly, the trends involve economists from top-ranked departments publishing fewer papers in many top journals. This could be a welcome change but is also potentially troublesome: one way in which the Internet could be disruptive is that it could allow high-profile researchers to disseminate dis·sem·i·nate
v. dis·sem·i·nat·ed, dis·sem·i·nat·ing, dis·sem·i·nates
1. To scatter widely, as in sowing seed.
2. their work without subjecting it to peer review, which in turn could lead to a broader unraveling of the peer-review system. The majority of the paper is then devoted to analyses of several additional data sources with the goal of understanding better what may be causing the observed trends.
The facts part of this paper documents two main facts:
1. Economists in top-ranked departments now publish very few papers in top field journals. There is a marked decline in such publications between the early 1990s and early 2000s.
2. Comparing the early 2000s with the early 1990s, there is a decline in both the absolute number of papers and the share of papers in the top general interest journals written by Harvard Harvard, town (1990 pop. 12,329), Worcester co., E central Mass.; inc. 1732. A Shaker house and cemetery, a Native American museum, and a Harvard observatory are there. economics department faculty.
Although the second fact just concerns one department, I see it as potentially important to understanding what is happening because it comes at a time when Harvard is widely regarded (I believe correctly) as having ascended to the top position in the profession.
The "decline-of-peer-review" theory I allude to allude to
verb refer to, suggest, mention, speak of, imply, intimate, hint at, remark on, insinuate, touch upon see see, elude in the title is that the necessity of going through the peer-review process has lessened less·en
v. less·ened, less·en·ing, less·ens
1. To make less; reduce.
2. Archaic To make little of; belittle.
To become less; decrease. for high-status authors: in the old days peer-reviewed journals were by far the most effective means of reaching readers, whereas with the growth of the Internet high-status authors can now post papers online and exploit their reputation to attract readers.
Many alternate explanations are possible. I focus on four theories: the decline-in-peer-review theory and three alternatives.
1. The trends could be a consequence of top-school authors' being crowded out of the top journals by other researchers. Several such stories have an optimistic op·ti·mist
1. One who usually expects a favorable outcome.
2. A believer in philosophical optimism.
op message, for example, there is more talent entering the profession, old pro-elite biases are being broken down, more schools are encouraging faculty to do cutting-edge research, and the Internet is enabling more cutting-edge research by breaking down informational barriers that had hampered researchers outside the top schools. (2)
2. The trends could be a consequence of the growth of revisions at economics journals discussed in Ellison El·li·son , Ralph Waldo 1914-1994.
American writer whose novel Invisible Man (1952) is a naturalistic depiction of a young Black man's struggle against American society.
Noun 1. (2002a, 2002b). In this more pessimistic pes·si·mism
1. A tendency to stress the negative or unfavorable or to take the gloomiest possible view: "We have seen too much defeatism, too much pessimism, too much of a negative approach" theory, highly productive researchers must abandon some projects and/or seek out faster outlets to conserve the time now required to publish their most important works.
3. The trends could simply reflect that field journals have declined in quality in some relative sense and become a less attractive place to publish. This theory is meant to encompass also the rise of new journals, which is not obviously desirable or undesirable.
The majority of this paper is devoted to examining various data sources that provide additional details about how economics publishing has changed over the past decade. These are intended both to sharpen understanding of the facts to be explained and to provide tests of auxiliary auxiliary
In grammar, a verb that is subordinate to the main lexical verb in a clause. Auxiliaries can convey distinctions of tense, aspect, mood, person, and number. predictions of the theories. Two main sources of information are used: data on publications and data on citations. The publication data include department-level counts of publications in various additional journals, an individual-level dataset See data set. containing records of publications in a subset A group of commands or functions that do not include all the capabilities of the original specification. Software or hardware components designed for the subset will also work with the original. of journals for thousands of economists, and a very small dataset containing complete data on a few authors' publication records. The citation Citation
(foaled 1945) U.S. Thoroughbred racehorse. In four seasons he won 32 of 45 races, finished second in ten, and third in two. He won the 1948 Triple Crown, and became the first horse to win $1 million. He set a world record in 1950 by running a mile in 1:33 3/5. data include citations at the paper level for 9,000 published papers and less well-matched data that is used to construct measures of citations to authors' unpublished works, to departments as a whole, and to various journals.
The analysis part of the paper contains numerous small analyses of these auxiliary data sources. Each is designed to provide information relevant to one or more of the theories. I hope that readers will also find some of them to be of independent interest. Among the interesting observations from the publication data are that the facts appear to pertain to pertain to
verb relate to, concern, refer to, regard, be part of, belong to, apply to, bear on, befit, be relevant to, be appropriate to, appertain to economists in top departments, rather than reflecting general trends among top economists, and that economists in top departments continue to publish at a high rate in many other outlets. The citation analyses provide some evidence that economists in top departments are less reliant on journals for the dissemination of their work: Harvard economists are garnering a large number of citations to their unpublished research; and the citation benefit of publishing in top general interest journals appears to be small for members of several top departments.
Keeping track of the various analyses and which ones favor which theories is difficult. I provide a summary table and refer to it repeatedly in Sections IV and V to help with this, but the nature of the paper makes it a bit unwieldy. No one theory fits all the additional facts and none of the analyses could rule out the relevance of any theory. The decline-in-peer-review theory appears to do relatively well. There are also multiple sources of support for the slowdown theory.
This paper belongs to a growing literature on publication processes and academic productivity. (3) Ellison (2002a) develops a theoretical model of the peer review process in which quality standards are endogenously en·dog·e·nous
1. Produced or growing from within.
2. Originating or produced within an organism, tissue, or cell: endogenous secretions. determined. McCabe and Snyder Snyder, city (1990 pop. 12,195), seat of Scurry co., NW Tex., in a prairie and mesquite region; inc. 1907. Oil production is the city's main industry; natural gas is also refined and processed. (2005) develop a model of journals as certification intermediaries. Ellison (2002b) documents the increased time costs of going through the peer-review process and notes several other trends. Azar (2007) discusses incentive effects of these time costs. Kim Kim
orphan wanders streets of India with lama. [Br. Lit.: Kim]
See : Adventurousness , Morse, and Zingales (2009) examine whether being in a top economics department provides a productivity benefit reflected in increased published output in good journals. It develops an extensive database and exploits the movement of faculty across universities to estimate a model with individual- and department-decade fixed effects and finds that the productivity benefit of being in a top department declined from the 1970s to the 1990s and was gone by the latter decade. Oyer OYER, pleading. Oyer is a French word signifying to hear; in pleading it is a prayer or petition to the court, that the party may hear read to him the deed, &c., stated in the pleadings of the opposite party, and which deed is by intendment of law in court, when it is pleaded with a (2006) is related: it uses initial-year job-market tightness as an instrument for obtaining an initial placement in a top department and finds that there is a causal causal /cau·sal/ (kaw´z'l) pertaining to, involving, or indicating a cause.
relating to or emanating from cause. effect of better initial placement on published output (and on long-term placement). It does not explicitly consider changes over time but can be regarded as obtaining a somewhat contrasting conclusion because its sample is largely from the 1990s.
The citation analysis Citation Analysis is the most common method of bibliometrics. Citation analysis uses citations in scholarly works to establish links to other works or other researchers.
Co-citation coupling and bibliographic coupling are specific kinds of citation analysis. presented here is related to the literature on the determinants of citations to academic research. Some of the noteworthy papers on the economics discipline are Stigler and Friedland Friedland, town, Russia
Friedland: see Pravdinsk.
Friedland, town, Czech Republic
Friedland: see Frýdlant, Czech Republic. (1975), Laband (1986), and Laband and Tollison (2000). Davis and Fromerth (2007) present an analysis of citations to math papers which also relates to the themes of this paper in that they analyze an·a·lyze
1. To examine methodically by separating into parts and studying their interrelations.
2. To separate a chemical substance into its constituent elements to determine their nature or proportions.
3. the impact of dissemination via the ArXiv arXiv (pronounced "archive", as if the "X" were the Greek letter Chi or Χ) is an archive for electronic preprints of scientific papers in the fields of mathematics, physics, computer science and quantitative biology which can be accessed via the Internet. .
The basic facts about the two trends are presented in Section II. Section III discusses potential explanations for the facts. Section IV contains additional analyses of publication records. Section V contains the citation analyses. I review the various findings and attempt to bring them together in Section VI. I conclude that the decline-of-peer-review theory is probably the most important and that the growth-of-revisions theory probably also plays some role.
II. TWO FACTS
In this section, l document the two facts highlighted in the introduction.
The primary data used in this section are counts of publications in 18 economics journals by the faculty of ten economics departments. (4) The departments are the top ten in one particular published ranking. (5) They are listed in Table 1 in the order of their NRC NRC
1. National Research Council
2. Nuclear Regulatory Commission
Noun 1. NRC - an independent federal agency created in 1974 to license and regulate nuclear power plants rating and I will also use the NRC ratings at times to separate them into "top five" and "six to ten" groups. The journal set includes five general interest journals and 13 field journals. (6) When I discuss "recent trends" in publishing, I am referring to differences between the contents of journals in 2000-2003 and the contents of the same journals in 1990-1993. Except where otherwise noted, publication counts in this paper assign each author partial credit for coauthored papers, for example, a three-authored paper with one author who is a faculty member at Princeton Princeton, borough (1990 pop. 12,016) and surrounding township (1990 pop. 13,198), Mercer co., W central N.J.; settled late 1600s, borough inc. 1813, township est. 1838. A leading education center, it is the seat of Princeton Univ. will count as one-third of a paper by Princeton.
A. Field Journal Publications
In this section, I discuss Fact 1:
Fact 1 Comparing the early 2000s with the early 1990s, there is a decline in the share of papers in the top field journals (and the absolute number) written by faculty members from the top five economics departments. Economists in these departments now publish in top field journals at a rate of about one paper per faculty member per decade.
Table 1 presents counts of publications in 13 field journals. (7) The set of journals was chosen to include the most widely cited field journals and to include at least one journal from most of the major fields of economics. The list of journals can be found in the table. The left columns present counts of 1990-1993 publications, and the right columns present 2000-2003 publications.
The "decline" part of Fact 1 comes through clearly in the top row of the table. It indicates that members of the top five economics departments published 86.4 papers in these journals in 1990-1993 and 71.2 papers in 2000-2003, an 18% decrease. (8) Moreover, this decline in publication counts came during a period when many of the journals were substantially increasing the number of papers they published. If one looks at the share of papers by top five departments, the decline is more dramatic: the 2.7% share in 2000-2003 is about one-third less than the 4.0% share of 1990-1993.
The "levels" part of Fact 1 is perhaps even more striking. The total of 71.2 papers by five departments over a 4-yr period is about 3.5 papers per department per year. These departments have about 40 faculty members on average, so this is a publication rate of less than one paper per faculty member per decade.
The breakdown for top five departments shows that the pattern is fairly consistent across departments within the top five. All top five schools show a decline in field journal publications in the share measure. All but MIT MIT - Massachusetts Institute of Technology are below the one-paper-per-faculty-member-per-decade output rate. The journal-by-journal breakdown for the top five schools shows that publications are down fairly consistently. The top five share has declined by more than 50% at five journals. Only two journals, Journal of Finance and Journal of Public Economics, have an increase in the top five share.
The data on the other top-ten departments and the journal-by-journal breakdown indicate that the pattern is somewhat broader than just what is noted in Fact 1. Publications by Yale and Berkeley Berkeley (bûr`klē), city (1990 pop. 102,724), Alameda co., W Calif., on the E shore of San Francisco Bay just N of Oakland; inc. 1878. Originally (1820) part of a Spanish rancho, the site was purchased by Americans in 1853. , which are top five departments in some rankings, are also down in both the count and share measures.
The pattern at the other three departments--Pennsylvania, Northwestern, and School Z--is different, but in a way that is not as different as it first appears. The data on aggregate publications in the 13 field journals show that these departments are not reducing their publications and that they now publish more papers in the top field journals than do most of the top five schools. If one looks further into the journal-by-journal detail, however, one finds a common pattern. The departments have increased their publication counts at three of the more theory-oriented (and more widely cited) field journals--Games and Economic Behavior, Journal of Economic Theory, and Journal of Monetary Economics--and reduced publications at the others.
I would summarize sum·ma·rize
intr. & tr.v. sum·ma·rized, sum·ma·riz·ing, sum·ma·riz·es
To make a summary or make a summary of.
sum the additional data as indicating that the decline in field journal publications extends to a broadly defined top five and may extend even further if one omits the more theory-oriented field journals.
B. Top General Interest Journal Publications In this section, I discuss Fact 2:
Fact 2 Comparing the early 2000s with the early 1990s, there is a decline in the number of papers in the top general interest journals written by faculty members from the Harvard economics department.
Table 2 presents publication counts for the "top five" general interest journals. (9) The first row of Table 2 looks very different from the first row of Table 1. One big difference is that there is no substantial decline in publications by the top five departments. The total number of papers published by these journals has declined, so the top five share has actually increased from 13.4% to 14.4%. A second striking difference is that the counts and shares in Table 2 are much larger than in Table 1. The top five departments publish more than twice as many papers in the five general interest journals as they do in the 13 field journals. Their share is more than five times as large in the general interest journals.
Fact 2 is apparent in the department-by-department breakdown. In the early 1990s, Harvard and Princeton were well ahead of the other departments. Between the early 1990s and the early 2000s, Harvard's top general interest publications had declined by about one-third (or by 28% if one uses the share measures). (10)
Publication counts are roughly constant at most of the other top five departments. All of them have at least a small increase in the share measure. Stanford shows a large increase from a low initial level. Again, these conclusions are fairly robust to how one defines the top five: Berkeley and Yale also have roughly constant general interest publication counts. Despite the decline, Harvard remains ahead of all departments other than Princeton and MIT.
Harvard's journal-by-journal breakdown is distinctive. Publications in Econometrica Econometrica is an academic journal of economics, publishing articles not only in econometrics but in many areas of economics. It is published by the Econometric Society via Blackwell Publishing. , Journal of Political Economy, and Review of Economic Studies are down by more than 50%. Publications in the Quarterly Journal of Economics The Quarterly Journal of Economics, or QJE, is an economics journal published by the Massachusetts Institute of Technology and edited at Harvard University's Department of Economics. Its current editors are Robert J. Barro, Edward L. Glaeser and Lawrence F. Katz. have stayed roughly constant. They now account for more than 45% of Harvard's top general interest publications. Harvard would rank ninth among the ten schools if one omitted QJE QJE Quarterly Journal of Economics publications from the counts.
C. Review of Facts
I have emphasized two facts: the decline in field journal publications at top five departments and a decline in top general interest publications at Harvard. It should be noted that the tables also show many nonfacts of this variety, that is, sets of schools and sets of journals for which publications have not declined. These should also be kept in mind when assessing whether potential explanations for the facts can explain all trends.
One way to organize all facts and nonfacts into a single finding may be to say that there appears to be a general pattern of top schools publishing fewer papers in journals below some cutoff threshold in the journal hierarchy, with the threshold being higher for higher-ranked departments. The chart below is an illustration. The rows of the chart correspond to the schools listed in order of their NRC ranking. The columns correspond to three journal sets: the non-QJE top general interest journals; the three field journals that were noted to have a somewhat different pattern; and the other ten field journals. School journal set combinations for which publications are declining in percentile terms are in bold. The grouping of journals is, however, somewhat arbitrary Irrational; capricious.
The term arbitrary describes a course of action or a decision that is not based on reason or judgment but on personal will or discretion without regard to rules or standards. and reflects an ex post attempt to convey patterns in the data, so I would not regard the table as providing evidence for this hypothesized pattern.
Non-QJE Top GEB, JET, Other Field General Interest and JME Journals Harvard Harvard Harvard Chicago Chicago Chicago MIT MIT MIT Stanford Stanford Stanford Princeton Princeton Princeton Yale Yale Yale Berkeley Berkeley Berkeley Pennsylvania Pennsylvania Pennsylvania Northwestern Northwestern Northwestern School Z School Z School Z
III. POTENTIAL EXPLANATIONS
In the introduction, I mentioned four theories that one might give to account for the two facts. In this section, I describe each mechanism in a little more detail and comment on how it comports with the facts described above.
A. Decline in the Importance of Peer Review
Journals serve two roles: they disseminate papers and provide quality certification. The Internet aids dissemination in many ways: papers are posted to authors' websites and working paper archives; e-mail is used to inform potential readers about papers; Internet search tools help readers find papers; journals have been made more accessible; and so on. Many of these are substitutes for traditional journal dissemination, so the primary role of journals may increasingly be to provide quality certification. (11) A shift in the role of journals could lead authors from top schools to withdraw from publishing in many journals for two reasons. First, highly regarded and highly visible authors will be able to make their work widely known (and widely read) without publishing it in journals. Such authors receive diminished di·min·ish
v. di·min·ished, di·min·ish·ing, di·min·ish·es
a. To make smaller or less or to cause to appear so.
b. dissemination benefits from journal publication. Second, highly regarded authors may traditionally have been publishing in journals mostly for the dissemination benefits. Consider questions of the form:
Which do you think is of higher quality: a paper by Author X that just appeared in Journal Y or the same author's most recent working paper that has yet to be submitted for publication?
If the answer to this question is not clear, then Journal Y is providing little in the way of quality certification benefits to Author X.
This story can naturally fit each of the main facts described in the previous section. For many faculty in top departments, the fact that a paper is published in a top field journal will only have a small impact on potential readers' beliefs about the paper's quality and will only provide minimal career-concerns benefits. Hence, a decline in the dissemination role should lead to a decrease in field journal publications. My impression is that even the most highly regarded economists in the profession still receive a reputational benefit (which helps both with disseminating the particular paper and for career-concerns reasons) from publishing in the top general interest journals. However, even in this case, certification benefits are likely smaller for high-status authors and the dissemination benefits may be smaller for authors who can make their work widely known without publishing it in a top journal. The decline in Harvard's general interest publication could reflect that it has a disproportionate dis·pro·por·tion·ate
Out of proportion, as in size, shape, or amount.
dispro·por number of very high-status economists and that its faculty are uniquely visible.
B. Top-Department Quality/Productivity
The share of papers in top journals written by faculty in a given department is a measure of the output of that department relative to the profession as a whole. Hence, our facts could be explained as a consequence of a relative decline of the departments considered. Several plausible mechanisms could account for such a decline.
First, the traditionally top-ranked departments may have been less successful in attracting and retaining the most productive economists. This could be due to relative increases in salaries and working conditions at economics departments at various competing employers: other top economics departments, business schools, and at institutions (e.g., foreign schools) that are attractive to particular faculty members. Anecdotally, I know that MIT has seen a large share of its top new PhDs take jobs in business schools over the last decade and has lost several faculty to and failed to entice many more faculty away from other departments.
Second, the top-ranked departments could have been as successful as ever in assembling the most productive economists, but still had their share of output decline because of changes in the productivity distribution. For example, this would be the case if the number of economists trying to publish in top journals has increased or if there is a flattening of the productivity distribution.
Third, the top-ranked departments could be as strong as ever in their productivity share but still see a decline in their output share because of changes in the distribution of resources. For example, Kim, Morse, and Zingales (2009) argue that improvements in communication technology have reduced barriers that made it more difficult for faculty outside the top schools to do cutting-edge research. Another potential source of resource reallocation Noun 1. reallocation - a share that has been allocated again
allocation, allotment - a share set aside for a specific purpose
2. reallocation is increased attention paid, for example, by the United Kingdom, to top-journal publications. This might increase resources allocated to producing top-journal publications either via direct shifts in resource allocations, or indirectly as faculty increase the share of their effort devoted to publishing in top journals.
These stories are all plausible, but I regard the decline-in-top-department-quality theory as not fitting the facts outlined in the previous section as neatly as the other theories. In particular, it does not seem so natural that a decline-in-top-department quality would reduce field journal publications but not top general interest publications. One could hypothesize hy·poth·e·size
v. hy·poth·e·sized, hy·poth·e·siz·ing, hy·poth·e·siz·es
To assert as a hypothesis.
To form a hypothesis. that economists at the top five departments have reacted to increased competition by concentrating more on general interest publications or that barriers to doing field journal work have come down more than barriers to doing general interest work, but it is not obvious why these should be true. The theory also seems hard to reconcile with Harvard's decline in top general interest publications. Harvard hired a large number of highly regarded and highly productive faculty between 1993 and 2001 and is commonly perceived to have ascended to the top position in the profession during this period. (12)
C. The Slowdown
The title of Ellison (2002b) emphasizes the "slowdown" of the publication process. An important observation of that paper, however, is that publication does not simply take longer in calendar time--the process also requires more effort from authors. The slowdown continued through the 1990s at both general interest and field journals. Hence, it is potentially relevant to why behavior might have changed between the early 1990s and the early 2000s. (13)
The increased burdens of publishing in top journals should affect economists' submission strategies for two reasons analogous analogous /anal·o·gous/ (ah-nal´ah-gus) resembling or similar in some respects, as in function or appearance, but not in origin or development.
adj. to the substitution Substitution
put her own son in place of Orestes; her son was killed and Orestes was saved. [Gk. Myth.: Zimmerman, 32]
robber freed in Christ’s stead. [N.T.: Matthew 27:15–18; Swed. Lit. and income effects of consumer theory. First, economists would be expected to substitute away from journals at which the process became more time consuming and/or more arduous ar·du·ous
1. Demanding great effort or labor; difficult: "the arduous work of preparing a Dictionary of the English Language" Thomas Macaulay.
2. toward outlets where this did not occur. Second, aggregate time constraints CONSTRAINTS - A language for solving constraints using value inference.
["CONSTRAINTS: A Language for Expressing Almost-Hierarchical Descriptions", G.J. Sussman et al, Artif Intell 14(1):1-39 (Aug 1980)]. may lead economists to publish fewer papers in peer-reviewed outlets. The papers they choose not to publish (or not to write) would presumably pre·sum·a·ble
That can be presumed or taken for granted; reasonable as a supposition: presumable causes of the disaster. be those for which the benefit to publication (per unit time required) is low.
Table 1 of Ellison (2002a) indicates that the 1990s slowdown was most severe at the top general interest journals (other than the QJE). Mean submit-accept times at the top non-QJE general interest journals increased from 17.5 mo in 1990 to 24.1 mo in 1999. (14) The QJE's mean submit-accept time was reduced from 22 to 13 mo. Mean submit-accept times at the seven top field journals for which data is available show a smaller increase: from 14.8 mo in 1990 to 16.4 mo in 1999.
The publication counts for the top general interest journals are consistent with the hypothesis that the slowdown is an important determinant determinant, a polynomial expression that is inherent in the entries of a square matrix. The size n of the square matrix, as determined from the number of entries in any row or column, is called the order of the determinant. of the observed changes. There is a shift in publications away from the other general interest journals toward the QJE. The decline in field journal publications relative to general interest publications could be attributed to the income effect. Economists at top schools publish a nontrivial nontrivial - Requiring real thought or significant computing power. Often used as an understated way of saying that a problem is quite difficult or impractical, or even entirely unsolvable ("Proving P=NP is nontrivial"). The preferred emphatic form is "decidedly nontrivial". fraction of their papers in top general interest journals, so it is plausible that increases in the time cost of publishing in general interest journals could lead them to spend less time trying to publish papers in field journals. The decline of Harvard's general interest publications is a little harder to fit into this theory. The direction of the change makes sense, but it is harder to argue that the slowdown alone should result in Harvard dropping below some other departments in general interest publications.
D. Field Journal Quality
Journals provide two main benefits to authors: they disseminate research, and they provide a quality certification that can bolster This article is about the pillow called a bolster. For other meanings of the word "bolster", see bolster (disambiguation).
A bolster (etymology: Middle English, derived from Old English, and before that the Germanic word bulgstraz the author's reputation. Each benefit will be reduced if the average quality of papers in a journal declines.
It is plausible that field journals suffered a relative decrease in quality over the period studied for a few reasons. First, Ellison (2002b) notes that citations to articles in field journals did not grow as rapidly as citations to articles in general interest journals. Recent articles in nine top field journals received only 30% as many citations as articles in the top general interest journals in 1998, down from 52% in 1990. Second, many top journals are for-profit operations that have raised prices substantially in recent years. This has reduced their availability and led to discontent that may be affecting submissions. (15) Third, new journals continue to be introduced and existing journals continue to attempt to move up in the journal hierarchy. For example, two introductions from late in the period studied are the B.E. Press journals (first published in 2001) and the Journal of the European Economic Association (in 2003).
The decline-in-field-journal-quality hypothesis fits well with the observation that most top departments are reducing publications in field journals but not in the general interest journals. A separate explanation would be needed to account for the reduction in Harvard's general interest publications.
Throughout this paper, I will try to organize my discussion of the coherence coherence, constant phase difference in two or more Waves over time. Two waves are said to be in phase if their crests and troughs meet at the same place at the same time, and the waves are out of phase if the crests of one meet the troughs of another. between the various theories and the evidence by referring to Table 3. Each column of the table corresponds to one of the theories discussed in this section. The rows correspond to the various pieces of evidence considered. The first row is concerned with Facts 1 and 2. Here, my summary is that each of the theories could explain the facts, but that the decline-in-top-department quality is somewhat problematic.
The remaining rows of the table summarize the coherence between the theories and the various data items presented in Sections IV and V. I will refer back to this table and discuss these summaries at the end of each subsection subsection
any of the smaller parts into which a section may be divided
Noun 1. subsection - a section of a section; a part of a part; i.e. .
IV. MORE PUBLICATION DATA
In this section, I present additional publication counts. They provide a more detailed view of the changes that have occurred and bear on which theories might be most important.
A. Departmental Publication Counts
Journal-Specific Declines and the Slowdown. If top authors are withdrawing from publishing in top journals because of the slowdown of the publication process, then one would expect that there would be a greater decrease in top-school publications at journals that have experienced more severe slowdowns. Figure 1 shows how these variables are related across journals: the difference between the journal's 1999 and 1990 submit-accept time (in months) is on the x-axis and the change in the share of papers by authors in the top five economics departments is on the y-axis. (16)
The top general interest journals are marked by solid boxes. The QJE obviously stands out in both dimensions: it has sped up rather than slowing down; and the share of papers by authors at the top five departments has increased dramatically. The two general interest journals that lost top-department authors, Econometrica and Review of Economic Studies, do not stand out in the figures for having slowed down more than the others during the 1990s. It may be relevant, however, that they are the two slowest journals in 1999.
The field journals are marked by outlined boxes. Journal of Public Economics and Journal of Urban Economics have sped up over the course of the 1990s (and were the two fastest journals in 1999). Neither shows a decline in its top-school share. Six field journals slowed down by more than 3 mo, and five of the six show substantial declines in their top-school shares. The Journal of Finance is an exception to this pattern.
Overall, I would conclude that the cross-journal pattern appears to be consistent with the slowdown being a factor contributing to the observed declines in top-school publications.
[FIGURE 1 OMITTED]
Publication Counts for Other Outlets. Table 4 presents publication counts for six widely read journals that publish many invited papers and/or have review processes that are less burdensome than those at standard journals. Top-school shares are quite large for all of these journals. The NBER NBER National Bureau of Economic Research (Cambridge, MA)
NBER Nittany and Bald Eagle Railroad Company Macroeconomics macroeconomics
Study of the entire economy in terms of the total amount of goods and services produced, total income earned, level of employment of productive resources, and general behaviour of prices. Annual's top five share is larger than the QJE's. The other journals' top five shares are comparable with or larger than those of the top general interest journals. They are much larger than the top five shares of the field journals we discussed in Section II. Looking across decades within each journal, there is a large decline in the number and share of Brookings Brookings, city (1990 pop. 16,270), seat of Brookings co., E S.Dak., on the Big Sioux River; inc. 1883. A trade center in a livestock and grain region, Brookings is an important seed-processing point. Papers by economists in top departments. (17) Beyond this, however, the general pattern appears to be that publication counts are roughly constant across the decades. Harvard's publication counts would not stand out if I had given a department-by-department breakdown: they are slightly above the average of the other top five departments at each journal and are roughly constant across decades (apart from a decline at Brookings Papers).
Of the 13 field journals discussed earlier, 12 published at least one special issue between 2000 and 2003. (18) Most often, these issues are a collection of papers presented at a conference. The review process usually differs from the procedure for regular issues: authors are often contacted personally and invited to submit papers; and the review and revision process must fit within a tighter time frame. Invariably in·var·i·a·ble
Not changing or subject to change; constant.
in·vari·a·bil , journals state that papers in special issues were peer-reviewed and subject to the same standards that the journal applies to all papers. Whether standards are really the same is subject to debate.
Table 5 reports counts of "special" articles in the 13 field journals. The first row gives counts for the top five departments and the second row for departments 6-10. The primary observation I would emphasize is that special issues of field journals do not look like regular issues of field journal issues: the top-department share is approximately three times as large, and there is no substantial decline in top five publications across decades. Harvard's publication counts again would not stand out in a department-by-department breakdown: Harvard published six articles in field-journal special issues in 1990-1993 and 8.1 in 2000-2003.
I see the data in this subsection as problematic for the decline-in-top-department-quality theory in a couple of ways: economists in top departments appear to be as strong as ever in their ability to garner slots in invited journals and special issues of field journals, and there is nothing to suggest that Harvard's decline in general interest publications is due to its having a more severe dropoff in productivity. The data seem supportive of the slowdown theory in that we are seeing economists from top departments publishing as they always have in outlets that have not been subject to the slowdown. The data are also somewhat at odds with the decline-in-field-journal-quality story in that there is no evidence that economists who have the opportunity to publish in special issues are dissuaded by a reduction in dissemination and prestige benefits.
Business School Publication Counts. One reason that could be given for why top economics departments may have declined relative to the profession as a whole is that more top economists may now be working in business schools. To provide some evidence on this, I collected publication counts like those in Section II for three top business schools: Chicago Chicago, city, United States
Chicago (shĭkä`gō, shĭkô`gō), city (1990 pop. 2,783,726), seat of Cook co., NE Ill., on Lake Michigan; inc. 1837. , Northwestern Kellogg, and Stanford. Although these schools publish a large number of top-journal papers, the data do not support the hypothesis that economists in business schools might be increasingly crowding economists from economics departments out of the top journals.
Table 6 presents the data. The Chicago GSB GSB Graduate School of Business (Stanford)
GSB Graduate School of Business (Chicago)
GSB Government of the Student Body (Iowa State University, Ames, IA) had a very high general interest publication count in 2000-2003, but the aggregate count for the three business schools was higher in the earlier period. Although Kellogg and Chicago both have more field journal publications in 2000-2003 than any of the economics departments studied, again, the aggregate count for 2000-2003 is less than the same figure for 1990-1993. I conclude that these business schools do not appear to be a source of increasing competition.
In Table 3, I summarize the evidence from this section by putting - signs in the top-department quality and field-journal-quality columns and a + sign in the slowdown column. The - for top-department quality reflects both the business school evidence and the fact that top departments are not publishing fewer papers in invited journals and special issues of field journals. The - for field journal quality also reflects top department economists' willingness to publish in special issues of field journals. The + in the slowdown column reflects the (albeit weak) support provided by the scatter plot See scatter diagram. of changes in top-department shares versus changes in the length of the review process.
B. Author CV Data
An obvious question to ask about the decline in top-journal publications is where the papers are going: Are economists in top departments publishing fewer papers? Are they publishing more in other outlets? If so, where are they publishing? Answering such questions is difficult, however. Complete publication lists can only be obtained by gathering CVs, and even then it can be hard to classify clas·si·fy
tr.v. clas·si·fied, clas·si·fy·ing, clas·si·fies
1. To arrange or organize according to class or category.
2. To designate (a document, for example) as confidential, secret, or top secret. publications in nonstandard non·stan·dard
1. Varying from or not adhering to the standard: nonstandard lengths of board.
For this paper, I present a smaller analysis of the setting that I thought would potentially be most informative. I collected CVs for Harvard faculty members who were both (a) tenured ten·ured
Having tenure: tenured civil servants; tenured faculty.
Adj. 1. tenured and (b) less than 40 yr old in the fall of 1993 and the fall of 2003. Harvard is a leader in withdrawing from top journals, and this design also lets me make comparisons for which age/experience differences will not be a potential confounding confounding
when the effects of two, or more, processes on results cannot be separated, the results are said to be confounded, a cause of bias in disease studies.
confounding factor factor.
Table 7 summarizes the publication records. Each row gives publication counts for a 40-yr period for a single economist. The top part contains 1990-1993 counts for the 1993 young senior faculty. The bottom part gives 2000-2003 counts for 2003 faculty. The second column gives a simple publication count not adjusted for coauthorship. The other columns show the location of the publications and, as in the rest of this paper, give partial credit for coauthored papers.
The counts in the second column indicate that there has not been a decline in total number of publications. (19) Young senior faculty at Harvard are still publishing an astounding a·stound
tr.v. a·stound·ed, a·stound·ing, a·stounds
To astonish and bewilder. See Synonyms at surprise.
[From Middle English astoned, past participle of astonen, number of papers! The third and fourth columns count QJE and other top general interest publications. The aggregate decline in non-QJE general interest publications indicates that the decline in general interest publications I discussed in Fact 2 is present even in this remarkably productive group of economists.
The fifth column shows that there is also a large decline in top field journal publications in this analysis of groups at comparable career stages. A natural question is whether this is simply due to the new generation's having shifted to other field journals or to general interest journals like Review of Economics and Statistics and Journal of the European Economic Association. The answer to this appears to be no. The sixth column reports counts of articles published in other peer-reviewed economics journals. (20) The average is approximately constant. Moreover, the breakdown suggests that one outlier outlier /out·li·er/ (out´li-er) an observation so distant from the central mass of the data that it noticeably influences results.
an extremely high or low value lying beyond the range of the bulk of the data. may be obscuring a similar downward trend on these publications: economist H is responsible for most of the "other" peer-reviewed publications (and also for most of the top field journal publications). Looking at the sums of the fifth and sixth columns, we see that four of the six 1993 young senior faculty published at least 3.3 articles in non-top five peer-reviewed journals in 1990-1993 and the lowest total is 1.5. In 2000-2003, the median is 1.0.
Publications in invited journals are higher in the latter period. The final column illustrates the most dramatic change: per capita [Latin, By the heads or polls.] A term used in the Descent and Distribution of the estate of one who dies without a will. It means to share and share alike according to the number of individuals. publications in outlets that are not traditional peer-reviewed economics journals jump from 1.1 to 7.5! The majority of these "other" publications are article-like items in conference volumes or other edited volumes. Some are articles in policy-oriented journals in other fields, and some are survey-like articles in traditional economic outlets, for example, Econometric Society The Econometric Society, an International Society for the Advancement of Economic Theory in its Relation with Statistics and Mathematics was founded on December 29, 1930 at the Stalton Hotel in Cleveland, Ohio.
The sixteen founding members were: Ragnar Frisch, Charles F. World Congress volumes.
The main impression I take away from this analysis is that Harvard's young senior faculty appear to be spending an increasing fraction of their time writing articles that are not being published in peer-reviewed journals. In my summary table, Table 3, I have put + in the columns for the decline-in-peer-review and slowdown theories and a--in the column for the decline-in-top-department-quality theory. I do so because I see the data as suggesting that the Harvard faculty could publish more in peer-reviewed journals if they decided to redirect re·di·rect
tr.v. re·di·rect·ed, re·di·rect·ing, re·di·rects
To change the direction or course of.
A redirect examination.
re their efforts to (a) perform/write more of their research in a way that would make it publishable in peer-reviewed journals and/or (b) spend less time doing research and more time navigating (networking, hypertext) navigating - Finding your way around. Often used of the Internet, particularly the World-Wide Web.
A browser is a tool for navigating hypertext documents. the peer-review process.
C. Author-Level Publication Database
Another natural question to ask about the two basic facts is whether they are facts about departments or individuals: are the declines in publications by the top departments a reflection of a more widespread decline in publications by highly regarded economists, or is there something about the departments themselves that is important? In this section, I develop some evidence on this question by analyzing a database containing information on individual authors' publication records.
I collected partial publication records for all authors who published a paper in a top general interest journal in the 1980s or the 1990s. For each author-decade, the data include: (a) the number of top general interest papers published in the decade; (b) the number of top general interest papers published in the first 4 yr of the next decade; and (c) the number of top field-journal papers published in the first 4 yr of the next decade. (21) I think of the number of general interest publications in a decade as a proxy for the author's status/productivity at the end of the decade. I use the other variables to examine whether high-status/high-productivity authors are now publishing less in top journals.
The data suggest that Facts 1 and 2 are facts about top departments. Figure 2 presents a simple illustration. Authors with a nonzero non·ze·ro
Not equal to zero.
Not equal to zero. number of (coauthorship-adjusted) top general interest publications in each decade were divided into six bins on the basis of decade-specific top general interest publication counts: (0,1), [1,2), [2,3), [3,4), [4,5), and [5, [infinity infinity, in mathematics, that which is not finite. A sequence of numbers, a1, a2, a3, … , is said to "approach infinity" if the numbers eventually become arbitrarily large, i.e. ]). The top panel of the figure examines field journal publications: the squares give the mean number of 2000-2003 field journal publications for authors whose 1990-1999 general interest publications fall into each bin; and the triangles give 1990-1993 field journal publications as a function of 1980-1989 general interest publications. The data indicate that the top departments' decline in field journal publications is not attributable attributable
emanating from or pertaining to attribute.
see attributable risk (below).
attributable risk to broader decline in field journal publications by high-status/highly productive economists. The top two bins show higher field journal output in the 2000s. The means for the other bins are very similar. (22)
[FIGURE 2 OMITTED]
The bottom panel presents corresponding data on general interest publications. Fact 2 is that these were lower in the early 2000s for economists at Harvard. One could imagine that this was attributable to Harvard having a disproportionate share of the very high-status economists. Again, the figure indicates that this does not appear to be the case, at least if status is adequately proxied by prior-decade general interest publications. Means are similar for all bins except for the highest one, and the highest bin has higher output in the later decade. (23)
Table 8 presents a related regression analysis In statistics, a mathematical method of modeling the relationships among three or more variables. It is used to predict the value of one variable given the values of the others. For example, a model might estimate sales based on age and gender. . In each decade, I constructed four variables on the sample of all economists who had multiple publications in the top five general interest journals in the decade: a count of the economist's general interest publications in the decade, a count of the economist's university's publications in these journals in the decade, and counts of the economist's publications in the general interest and field journal groups in the first 4 yr of the next decade. (24)
The first two columns report coefficients from negative binomial binomial (bī'nō`mēəl), polynomial expression (see polynomial) containing two terms, for example, x+y. The binomial theorem, or binomial formula, gives the expansion of the nth power of a binomial (x+ regressions of the author's field journal publications in the first 4 yr of a decade on author- and school-level publication counts for the preceding decade. The results bolster the view that the publication decline noted in Fact 1 is a top school phenomenon rather than a top author phenomenon. The author-level variable indicates that authors with impressive publication records are not withdrawing from publishing in field journals. The school-level variable, which was insignificant in 1990-1993 is negative in 2000-2003 indicating that authors from top schools are now publishing less in field journals than are authors from less prestigious schools who have comparable publication records.
The third and fourth columns report similar regressions examining general interest publications in 1990-1993 and 2000-2003. Again, there is nothing to suggest that authors with strong publication records are withdrawing from publishing in general interest journals. The school-level variable is positive and significant for both decades. It is somewhat smaller in the second decade, but this difference is not significant.
The fifth and sixth columns examine the robustness of the above regressions to the sample selection: they reestimate the models in the second and fourth columns on the full sample of authors with at least one general interest publication in the 1990s. The coefficients on the author-level variables are similar to those reported earlier. The school-level variable is now unrelated to field journal publications.
The evidence in this section is problematic for both the slowdown and decline-in-field-journal-quality theories. Under the slowdown theory, one would have expected that the authors who were publishing the largest number of general interest papers would have been most affected by the slowdown and would have experienced the largest decline in field journal publications. In the decline-in-field-journal-quality theory, one would probably have expected that authors with the strongest publication records would be most likely to regard diminished field journals as not worth publishing in. The data are consistent with a version of the decline-in-peer-review theory in which it is being in a top department, rather than having a strong publication record, that enables an author to attract attention for his or her work without publishing it in a top journal. In my summary table, Table 3, I have put--in the slowdown and decline-in-field-journal-quality columns and a + in the decline-in-peer-review column. It should be understood that this + is only supporting one possible variant variant /var·i·ant/ (var´e-ant)
1. something that differs in some characteristic from the class to which it belongs.
2. exhibiting such variation.
adj. of the theory.
V. CITATION DATA
Citations are an obvious source of information on the dissemination of research. They are also used as a quality metric. The citation data discussed below was compiled by collecting all citations made in 1994 and 2004 by 21 journals: the 5 top general interest journals; the 13 field journals; and 3 of the "invited" journals (Journal of Economic Perspectives, Journal of Economic Literature, and the Papers and Proceedings issue of the AER.) This has several benefits relative to relying on Thomson Reuters Reuters
British cooperative news agency. Founded in 1851 by Paul Julius Reuter, it was initially concerned with commercial news but began to serve a growing newspaper clientele after the London Morning Advertiser subscribed in 1858. ISI ISI International Sensitivity Index, see there citation counts: it avoids some of the problems with intertemporal comparability caused by the proliferation proliferation /pro·lif·er·a·tion/ (pro-lif?er-a´shun) the reproduction or multiplication of similar forms, especially of cells.prolif´erativeprolif´erous
n. of journals; it provides some focus on important citations; and it allows me to construct measures of citations to unpublished as well as published works. Obviously, a number of limitations remain and new limitations are introduced. (25)
A. Journal Citation Counts
Journal-level citation counts are obviously relevant to the decline-in-field-journal-quality theory. The top panel of Table 9 reports per-article citation counts for recent articles between 1994 and 2004. More precisely, the entry for Journal X in the 1994 column is the number of times that 1994 articles in the 21 journal set cited an article in Journal X from 1984 or later, divided by the total number of articles Journal X published in 1984-1993. (26)
The bottom panel of Table 9 summarizes this data by reporting means for each journal category and also provides comparable figures that only count citations that appear in one of the top five journals. The 21-journal counts portray por·tray
tr.v. por·trayed, por·tray·ing, por·trays
1. To depict or represent pictorially; make a picture of.
2. To depict or describe in words.
3. To represent dramatically, as on the stage. the field journals as declining in influence relative to both the top general interest journals and the invited journals. This is not a very robust result, however. In the five-journal counts, the field journals are gaining slightly on the invited journals and falling only slightly farther behind the general interest journals.
There is heterogeneity het·er·o·ge·ne·i·ty
The quality or state of being heterogeneous.
the state of being heterogeneous. within each category of journals. The QJE had a huge increase in citations and became the most-cited general interest journal. REStud and AER also made substantial gains to achieve near-parity with Econometrica and JPE JPE Journal of Political Economy
JPE Jump If Parity Even
JPE Journal of Private Equity
JPE Joel Plaskett Emergency (Halifax, Nova Scotia band)
JPE Japanese Pharmaceutical Excipients
JPE Truncated JPEG file extension . All six "invited" journals show gains from 1994 to 2004. The heterogeneity here is that whereas most of the gains are relatively small, the NBER Macroeconomics Annual shows a large increase. It is more cited on a per-article basis than any general interest journal. There is no consistent trend in the field journal category: five journals gain and eight journals lose citations. The set of five field journals that gained in citations includes the two field journals that had an increase in their top five author shares across decades and two that did the next best at holding on to their top five authors.
In Table 3, I have recorded this section as providing support for the decline-in-field-journals theory because both the overall decline in field-journal citations and the relationship between which journals lost top-department authors and which journals lost citations are consistent with this theory. The strength of the support can be regarded as weak, both because the patterns themselves are not very strong and because the causation causation
Relation that holds between two temporally simultaneous or successive events when the first event (the cause) brings about the other (the effect). According to David Hume, when we say of two types of object or event that “X causes Y” (e.g. could run in the other direction.
B. Departmental Citation Counts
Departmental citation counts were tabulated in a two-step process. I first produced counts of all citations to each last name-initial pair. I then computed each department count as the sum over all last name-initial pairs that corresponded to the last name-initial of one of their faculty members. This has several obvious limitations. (27) I hope that measurement errors are largely orthogonal to the comparisons I will be making across departments and over time.
Table 10 reports average citations per faculty member for ten departments and for other authors with a recent publication in a top general interest or top field journal. The first two columns tabulate all citations made to each author in 1994 and 2004 (in the set of 21 journals and subject to the caveats above). One fact that stands out is that Harvard is doing extremely well in citations. Its citations are up by 65% from 1994 to 2004. (28) It has moved from fourth on the list to first. Authors from
the other top departments are not doing as well. The other top five departments as a whole experienced a decline in per author citations. This is particularly noteworthy because it comes at a time when reference lists are getting longer, which results in the 21 journals making more total citations in 2004 than in 1994. (29) Authors from departments 6-10 gained 15% on average. Authors from other schools gained 10%, reflecting in part that the 21 journals considered make more total citations in 2004 than in 1994.
One frequent concern that comes up in discussing citation counts is that the usual counts are dominated by citations made by obscure journals. The focus in this paper on citations in 21 journals should alleviate Alleviate
To make something easier to be endured.
Mentioned in: Kinesiology, Applied this concern, but I address it further by reporting, in the third and fourth columns, citation counts that only include citations appearing in one of the top five general interest journals. These give a similar picture: Harvard's citations are way up; citations to the other top five departments are down; citations to departments 6-10 and to other authors are up slightly. (30)
Citations obviously measure lifetime achievement and need not be closely related to recent productivity. To provide something closer to a measure of the impact of authors' recent research, the fifth and sixth columns provide tabulations that only include citations to items (published or not) cited as dating to the previous 4 yr, for example, the 1994 column reports 1994 citations to items with dates in 1990-1993. Here, Harvard is the only top five department showing a (now small) per capita increase. Departments 6-10 and other authors also show declines in this measure.
An important factor in summarizing this data is that the patterns at Harvard are different from those at the other top five departments. Hence, the data lead to conflicting conclusions. The Harvard data are more consistent with a decline in the necessity of peer review than with a decline in relative department quality. The opposite is true of the data on the other departments. In Table 3, I have indicated this by placing +/- symbols in both columns.
C. Paper-Level Citation Database
In this section, I use data on citations at the paper level to enrich the above descriptions and to address additional questions. I focus on how citations covary with the journal in which a paper is published and with the author's institution and whether there is a change over time in these relationships.
My paper-level database includes 1994 citation counts for all papers published in 1990-1993 (in one of the 23 journals studied) and 2004 citation counts for all papers published in 2000-2003. I examine how citations are related to journal- and author-characteristics using negative binomial regressions, for example,
[Cites.sub.i] ~ Poisson([[micro].sub.i])
log([[micro].sub.i]) = [[beta].sub.0] + [[beta].sub.1] [AuthorTop5School.sub.i] + [[beta].sub.2][Author6-10School.sub.i] [[beta].sub.3][FieldJoural.sub.i] [[beta].sub.4][InvitedJournal.sub.i] [[beta].sub.5][OtherCharacteristics.sub.i] + AgeDummies + [[epsilon].sub.i],
with [[epsilon].sub.i] [GAMMA The way brightness is distributed across the intensity spectrum by a monitor, printer or scanner. Depending on the device, the gamma may have a significant effect on the way colors are perceived. ] ([THETA], [THETA])-distributed random variable. This can be thought of as similar to estimating a simple regression Noun 1. simple regression - the relation between selected values of x and observed values of y (from which the most probable value of y can be predicted for any value of x)
regression toward the mean, statistical regression, regression with log([Cites.sub.i]) as the dependent variable. (31) The dependent variable Cites in the 1990s (2000s) regression regression, in psychology: see defense mechanism.
In statistics, a process for determining a line or curve that best represents the general trend of a data set. is the number of citations that each paper published in 1990-1993 (2000-2003) received in 1994 (2004). In the base model, the main explanatory ex·plan·a·to·ry
Serving or intended to explain: an explanatory paragraph.
ex·plan variables are dummy Sham; make-believe; pretended; imitation. Person who serves in place of another, or who serves until the proper person is named or available to take his place (e.g., dummy corporate directors; dummy owners of real estate). variables for the type of journal (top general interest is the omitted category) and dummies for whether the author is in a top 5 or 6-10 ranked economics department. Three paper characteristics (in addition to year dummies) are included as control variables: the log of the order in which the paper appears in its issue; the length of the article; and the number of authors.
Coefficient coefficient /co·ef·fi·cient/ (ko?ah-fish´int)
1. an expression of the change or effect produced by variation in certain factors, or of the ratio between two different quantities.
2. estimates and standard errors for the base model are presented in the first two columns of Table 11 with t-statistics in parentheses. The estimated coefficients on the control variables bring out several interesting and potentially relevant facts. First, the order in which a paper appears in its journal issue is a significant predictor of citations. (32) This indicates that editors are able to predict which articles are likely to be more influential and/or that more readers look at articles that appear earlier in a journal issue. Second, the coefficient on the NumAuthor variable indicates that papers with more authors are more widely cited. (33) One possible interpretation of this result is that citations reflect how extensively authors "market" a paper as well as the paper's inherent quality. The coefficients on the age dummies indicate that knowledge of papers diffuses sufficiently quickly so that 3-yrold and 4-yr-old papers are cited at about the same rate.
AuTop5School is the fraction of a paper's authors who are faculty members in a top five economics department. Papers by authors in the top departments are more widely cited in each decade. This could be attributed to authors in top departments' having an advantage in marketing papers or to differences in average quality (that are not fully reflected in how journals order papers within an issue). I find a similar effect for department 6-10 authors in the early 1990s, but not in the 2000s.
Field Journal and Invited Journal are dummy variables for papers appearing in field and invited journals. Papers in both types of journals are less cited on average than papers appearing in top general interest journals. The coefficient estimates of about -0.9 indicate that papers in field journals receive approximately 60% fewer citations than papers in top general interest journals. This difference appears to be fairly stable over time (again not providing much support for the decline-in-field-journal-quality theory). Invited journals are also well behind top general interest journals but appear to be gaining somewhat.
The regressions in the third and fourth columns of Table 11 add interactions between the journal (Top 5, Field and Invited) and author affiliation (Top 5, 6-10, Other) classifications. The coefficient on AuTop5School now measures the extra citations that accrue To increase; to augment; to come to by way of increase; to be added as an increase, profit, or damage. Acquired; falling due; made or executed; matured; occurred; received; vested; was created; was incurred. to authors from top five departments when publishing in the top general interest journals. Papers by authors from top departments were substantially more widely cited than other papers in top general interest journals in the 1990s, but this effect has declined in the last decade and is no longer statistically significant. This could reflect a decrease in dissemination advantages or a decrease in relative quality.
A comparison between the coefficients on the Field Journal x Department Category interactions indicates that papers by authors in top five departments receive many more citations than other papers in the same field journal. (34) This is consistent with the hypothesis that authors from top schools are better able to gain attention for their work without publishing it in top general interest journals. (35) Note, however, that the estimates on the Invited Journal interactions do not follow this pattern.
One aspect of these findings that is a little puzzling puz·zle
v. puz·zled, puz·zling, puz·zles
1. To baffle or confuse mentally by presenting or being a difficult problem or matter.
2. is that they suggest that citations to papers by economists in top departments are now not very sensitive to where the paper is published (the causal effect of publishing the same paper in a field journal must be less than [e.sup.-027] if the general interest papers in our sample are of higher quality than the field journal papers). This must be reconciled with the finding that authors from top departments are publishing fewer papers in these journals. Two possible lines of argument are that top five authors may be primarily publishing in top general interest journals to promote their fields or maintain their reputations, and that the citation penalty from publishing in nonjournal outlets may also be getting smaller.
A striking result of the previous section was that Harvard is doing relatively well in citations in a period when it is doing relatively poorly in top-journal publications. This naturally raises the question of whether this is due to gains in per-article citations outweighing the drop in top-journal publications, or whether it is due to Harvard's garnering more citations on papers that are not in top journals. The fifth and sixth columns of Table 11 address this question by adding dummies for each top-ten school. The regressions also include (unreported) journal dummies and interactions between top-department and journal-class dummies. Hence, the coefficients on the school dummies reflect the citations accruing to papers the school published in top general interest journals relative to other papers in the same journal, and citations accruing to papers the school published in field and invited journals relative to papers in those journals by members of other top departments. The school dummies from these regressions are reported separately in Table 12.
The results indicate that Harvard's strong recent citation performance is not due to citations to its top-journal publications: the point estimate on the Harvard dummy is smaller in the later period than in the earlier period and indicates that Harvard is not gaining relative to MIT, Princeton, and Chicago. The standard errors are such that the cross-coefficient comparisons are not statistically significant, but this does not really matter for the above conclusion--the sample here is the lull set of papers in the 23 journals in 2000-2003, which was the period for which Harvard was shown to be doing well in the final column of Table 10. I conclude that Harvard's relatively strong citation record must be attributable to its receiving many citations for papers that are not in top journals. (36)
The fifth and sixth columns also provide some evidence that results noted earlier are robust by showing that they hold across departments and do not go away when a full set of journal dummies is included. For example, the regressions again indicate that top departments are receiving many citations for their field journal publications and show that the pattern of getting fewer citations for top general interest publications appears to be consistent across schools.
One conclusion of this section seems fairly clear: the data seem inconsistent Reciprocally contradictory or repugnant.
Things are said to be inconsistent when they are contrary to each other to the extent that one implies the negation of the other. with the notion that authors in top departments are shunning field journals because these journals are getting worse and no longer provide sufficient dissemination. The other conclusions are less clear. Several observations are consistent with the idea that authors at top schools may increasingly be able to attract attention without publishing in top journals. The significance of the number of authors and the author's institution in the citation regressions provides evidence that nonjournal dissemination has always been important. Both the fact that Harvard's strong recent citation performance appears to be due to citations to papers not published in top journals and the fact that the general interest-field journal citation gap is narrowing for authors at top schools suggest that the importance of publishing in the best journals is declining over time. I put a +/in the decline-in-peer-review necessity column, however, because the results on relative citations of general interest and field journal publications is awkward to reconcile with top departments' concentrating their efforts on publishing in general interest journals. The evidence is also mixed on the decline-in-top-department-quality theory. The citation declines on general interest publications could be taken to suggest a quality decline, but the opposite results on citations to papers in field journal have the opposite implication implication
In logic, a relation that holds between two propositions when they are linked as antecedent and consequent of a true conditional proposition. Logicians distinguish two main types of implication, material and strict. .
I started this paper by pointing out two trends: economists in several highly regarded departments are publishing fewer papers in the top field journals; and Harvard's economics department is also publishing fewer papers in the top general interest journals.
Several pieces of evidence bolster the view that one factor contributing to these trends is that the role of journals in disseminating research has been reduced. One is that the citation benefit to publishing in a top general interest journal now appears to be fairly small for top-department authors. Another is that Harvard authors appear to be quite successful in garnering citations to papers that are not published in top journals. The fact that the publication declines appear to be a top-department phenomenon (as opposed to a prolific-author phenomenon) suggests that a top-department affiliation may be an important determinant of an author's ability to sidestep side·step
v. side·stepped, side·step·ping, side·steps
1. To step aside: sidestepped to make way for the runner.
2. the traditional journal system.
Other potential explanations for the trends also appear to be relevant. The slowdown of the publication process continued through the 1990s. It is natural that this would lead authors to cut back on the number of papers they subject to peer review and that the best papers would be given highest priority. The fact that top-department authors continue to publish in special issues of field journals (and that we see many publications in invited journals and nonjournals) suggests that the arduousness ar·du·ous
1. Demanding great effort or labor; difficult: "the arduous work of preparing a Dictionary of the English Language" Thomas Macaulay.
2. of the publication process is playing a role in deterring submissions.
I should also note that although the analyses do not provide much support the other explanations--the decline in the importance of field journals and the decline in the quality of the departments--considered they still may be part of the story. The analyses are not designed to refute re·fute
tr.v. re·fut·ed, re·fut·ing, re·futes
1. To prove to be false or erroneous; overthrow by argument or proof: refute testimony.
2. explanations. For example, it is only for the five young senior faculty at Harvard that I can say definitively that there was not a shift from the studied field journals to other peer-reviewed economics journals.
The "trends" discussed in this paper refer to changes between the first 4 yr of the 1990s and the first 4 yr of the 2000s. It is natural to wonder whether they have continued or reversed since that time. To this end, I collected additional publication counts for the year 2008 relating to relating to relate prep → concernant
relating to relate prep → bezüglich +gen, mit Bezug auf +acc the two main facts that motivated mo·ti·vate
tr.v. mo·ti·vat·ed, mo·ti·vat·ing, mo·ti·vates
To provide with an incentive; move to action; impel.
mo this paper. A quick summary of these data are that it appears that the highlighted trends have not deepened since 2003, but may have broadened. First, one phrasing of the primary trend for field journals was that the number of papers that the top seven departments published in the 13 field journals dropped from an average of 4.3 papers per department per year in 1990-1993 to 3.4 papers per department per year in 2000-2003. The year 2008 looks a lot like the early 2000s for these departments: they averaged 3.5 papers per department. The possible "broadening" of the trend concerns the other three "top-ten" departments, which had not seen a decline in their field journal publications between the early 1990s and the early 2000s. These departments, which had averaged five papers per department per year in the early 2000s, averaged just 2.4 papers per department in 2008. One year, of course, is a limited time period and any changes would need to be interpreted in light of other changes that have occurred in the profession since 2003, which includes the launches of Theoretical Economics and the new AEA journals and Economic Inquiry's no revision option. The second motivating fact was that Harvard's publications in the top general interest journals dropped from 12.2 papers per year in the early 1990s to 8.0 papers per year in the early 2000s. There is no evidence of a further decline at Harvard: Harvard economists published 8.75 papers in these journals in 2008. A potentially interesting observation here is that this decline may be extending to other departments: the other top five departments, which had averaged 8.2 papers per department per year in the early 2000s, averaged just 5.1 papers per department in 2008; eight of other nine departments were below their 2000-2003 average in 2008; and five were down by more than one-third. Again, however, one year is a short period and one would want to consider various factors including the shrinkage Shrinkage
The amount by which inventory on hand is shorter than the amount of inventory recorded.
The missing inventory could be due to theft, damage, or book keeping errors. of Econometrica before drawing any conclusions.
The changes that have occurred over the past decade are modest in magnitude. Economists at top departments are still spending a great deal of effort publishing in top peer-reviewed journals and publishing many papers there. One could imagine, however, that much larger changes will be seen in the near future. Technologies for disseminating papers will continue to improve. More top economists may realize that the publication hassles they have been enduring are not necessary. The peer-review process may also be subject to unraveling: as more top economists withdraw from the process, the signal that publication in a given journal provides is devalued de·val·ue also de·val·u·ate
v. de·val·ued also de·valu·at·ed, de·val·u·ing also de·val·u·at·ing, de·val·ues also de·val·u·ates
1. To lessen or cancel the value of. , and this may lead to further withdrawals. Even a partial unraveling could have a significant impact on the course of economic research. For example, if only the top general interest journals maintain their stature stature /sta·ture/ (stach´ur) the height or tallness of a person standing.stat´ural
The height of a person.
the height of an animal in the standing position. , then more economists may concentrate on "general interest" research and decline to make the kinds of incremental contributions to sophisticated literatures that appeal relatively more to those who are experts on a topic. Indeed, another potential line of explanation for the facts presented here is that top departments may be focusing on projects that are of greater "general interest" but of less interest to specialists in any particular field.
One could imagine that new institutions may arise and perform many of the same functions as the current peer-review system more efficiently. Given how central peer-review has been to academic research over the past century, however, the thought that the current system might collapse before any successor is clearly established is troubling.
Swedish biochemist and physician. He shared a 1982 Nobel Prize for research into the chemical structure of prostaglandins. , and C. T. Bergstrom. Differences in Impact Factor across Fields and over Time. University of California, Santa Barbara History
The predecessor to UCSB, Santa Barbara State College, focused on teacher training, industrial arts, home economics, and foreign languages. Intense lobbying by an interest group in the City of Santa Barbara led by Thomas Storke and Pearl Chase persuaded the State Department of Economics, Working Paper 2008-4-23, 2008.
Azar, O. H. "The Review Process in Economics: Is It Too Fast?" Southern Economic Journal, 72(2), 2005, 482-91.
--."The Slowdown in First-Response Times of Economics Journals: Can It Be Beneficial?" Economic Inquiry, 45(1), 2007, 179-87.
Bergstrom, T. C. "Free Labor the labor of freemen, as distinguished from that of slaves.
See also: Free for Costly Journals." Journal of Economic Perspectives, 15, 2001, 183-98.
Colander, D. "Research on the Economics Profession." Journal of Economic Perspectives, 3(Autumn), 1989, 137-48.
Davis, P. M., and M. J. Fromerth. "Does the ArXiv Lead to Higher Citations and Reduced Publisher Downloads for Mathematics Articles?" Scientometrics Scientometrics is the science of measuring and analysing science. In practice, scientometrics is often done using bibliometrics that is measurement of (scientific) publications.
Modern scientometrics is mostly based on the work of Derek J. de Solla Price and Eugene Garfield. , 71(2), 2007, 203-15.
Ellison, G. "Evolving Standards for Academic Publishing: A q-r Theory." Journal of Political Economy, 110(5), 2002a, 994-1034.
--. "The Slowdown of the Economics Publishing Process." Journal of Political Economy, 110(5), 2002b, 947 -93.
Gans, J. Publishing Economics: An Analysis of the Academic Journal Market in Economics. London London, city, Canada
London, city (1991 pop. 303,165), SE Ont., Canada, on the Thames River. The site was chosen in 1792 by Governor Simcoe to be the capital of Upper Canada, but York was made capital instead. London was settled in 1826. : Edward Elgar Sir Edward William Elgar, 1st Baronet, OM, GCVO (2 June 1857 – 23 February 1934) was an English Romantic composer. Several of his first major orchestral works, including the Enigma Variations and the Pomp and Circumstance Marches, were greeted with acclaim. , 2000.
Greene, W. H. Econometric e·con·o·met·rics
n. (used with a sing. verb)
Application of mathematical and statistical techniques to economics in the study of problems, the analysis of data, and the development and testing of theories and models. Analysis. Upper Saddle River Saddle River may refer to:
In 1913, law professor Dr. , 1997.
Hausman Haussmann, Hausmann, Hausman are surnames that may refer to: Hausmann
Johnson, D. "Getting Noticed in Economics: The Determinants of Academic Citations." American Economist, 41(1), 1997, 43-50.
Kim, E. H., A. Morse, and L. Zingales. "Are Elite Universities Losing Their Competitive Edge?" Journal of Financial Economics, 93(3), 2009, 353-81.
Laband, D. N. "Article Popularity." Economic Inquiry, 24, 1986, 173-80.
Laband, D. N., and R. D. Tollison. "Intellectual Collaboration Working together on a project. See collaborative software. ." Journal of Political Economy, 108(3), 2000, 632-62.
McCabe, M. J., A. Nevo, and D. L. Rubinfeld. "Academic Journal Pricing and the Demand of Libraries." American Economic Review, 95(2), 2005, 447-52.
McCabe, M. J., and C. M. Snyder. "A Model of Academic Journal Quality with Applications to Open-Access Journals." American Economic Review, 95(2), 2005, 453-8.
Medoff, M. H. "Collaboration and the Quality of Economics Research." Labour Economics, 10, 2003, 597-608.
Morbius, M., and T. Rosenblat. "Getting Closer or Drifting Apart." Quarterly Journal of Economics, 119(3), 2004, 971-1009.
Oyer, P. "Initial Labor Market labor market A place where labor is exchanged for wages; an LM is defined by geography, education and technical expertise, occupation, licensure or certification requirements, and job experience Conditions and Long-Term Outcomes for Economists." Journal of Economic Perspectives, 20(Summer), 2006, 143-60.
Stigler, G. J., and C. Friedland. "The Citation Practices of Doctorates in Economics." Journal of Political Economy, 83, 1975, 477-508.
(1.) Among the new institutions in the economics profession are the working paper archives like RePEc RePEc Research Papers in Economics (USA) and SSRN SSRN Social Science Research Network , new electronic-only journals like Economics Bulletin, Theoretical Economics, and the BEPress journals, and NAJ NAJ Nurses Association of Jamaica
NAJ Network Attached Jukebox (system) Economics, which aims to provide peer review without publication. Economics is also influenced by general tools, including Google Scholar This article or section contains information about computer software currently in development.
The content may change as the software development progresses. . Electronic archives have achieved greater success in several other disciplines.
(2.) Kim, Morse, and Zingales (2009) argue for the latter hypothesis and provide empirical evidence.
(3.) Colander (1989) and Gans (2000) contain nice surveys of the literature at different points in time.
(4.) The publication counts I present are always for regular faculty in the economics departments at these universities. They do not include publications by graduate students or visitors to these departments, nor publications by faculty members whose primary affiliation is in some unit other than the economics department. Because of limitations in Econlit, the data do not include the identity or affiliation for all authors other than the first when papers have four or more authors.
(5.) The ranking used to choose the ten schools is that in footnote Text that appears at the bottom of a page that adds explanation. It is often used to give credit to the source of information. When accumulated and printed at the end of a document, they are called "endnotes." 29 of Ellison (2002b). it is based on counts of publications in top five journals in 1990-1997. Obviously, many other rankings of departments are available and could have been used to motivate selecting a somewhat different to a different set of departments for study. My list includes the top nine from the NRC's 1995 ranking. To reduce the risk of offending anyone who thinks that their department should have been included, I have chosen to withhold with·hold
v. with·held , with·hold·ing, with·holds
1. To keep in check; restrain.
2. To refrain from giving, granting, or permitting. See Synonyms at keep.
3. the identity of one department, which I label School Z. Curious readers can find it in footnote 29 of Ellison (2002b).
(6.) The general interest journals are the same journals studied in Ellison (2002b). They seem to be widely regarded as being the top general journals. The selection of the field journals reflects two motivations: a desire to include the top field journals in major fields; and a desire to piggyback piggyback
1. A broker trading in his or her personal account after trading in the same security for a customer. The broker may believe the customer has access to privileged information that will cause the transaction to be profitable.
2. on data that had been collected for use in Ellison (2002b). The procedure for selecting journals was as follows. First, 1 selected the nine field journals from Table 12 of Ellison (2002b). Next, I added the Journal of Finance and Journal of Labor Economics The Journal of Labor Economics, published by the University of Chicago Press presents international research examining issues affecting the economy as well as social and private behavior. , which I perceived to be regarded as the top journals in major fields. Finally, I used data from Journal Citation Reports Journal Citation Reports (JCR) is an annual publication by the Institute of Scientific Information, a division of Thomson Scientific. It provides information about academic journals in the sciences and social sciences. to look for other highly cited journals. Specifically, for a large sample of journals, I computed the number of times that the journal was cited in general interest journals in a recent year and divided this by the number of articles published in 2001 to obtain a per article measure. Games and Economic Behavior Games and Economic Behavior (GEB) is a journal of game theory published by Elsevier. First published in 1989, it is considered to be the leading journal of game theory and one of the top journals in economics. and Journal of Law, Economics, and Organization were both in the top six in this measure and were added to the sample. Obviously, however, many other selections of 13 field journals could have been made.
(7.) The counts omit o·mit
tr.v. o·mit·ted, o·mit·ting, o·mits
1. To fail to include or mention; leave out: omit a word.
a. To pass over; neglect.
b. papers in special issues of the journals and papers that are three or fewer pages in length.
(8.) Coauthorship rates have increased over the period. Despite this, an unweighted count that gives each author full credit for a paper still shows a decline: unweighted counts are 136 in 1990-1993 and 126 in 2000-2003.
(9.) The counts omit special articles like presidential addresses and also omit all papers in the Papers and Proceedings issues of the American Economic Review.
(10.) Counts giving full credit for papers whether coauthored or not give a similar picture. Harvard's unweighted publication count declines from 86 to 56. The other top five departments increase from 184 to 222.
(11.) Azar (2007) reports that working papers working papers
Legal documents certifying the right to employment of a minor or alien.
Noun 1. working papers grew from 3% of the citations in the AER and Econometrica in 1960 to 14% in 2002. Given that the majority of citations are now to papers that are more than 10 yr old, working papers account for a substantial fraction of the citations to recent works.
(12.) Harvard's hires in this period include Philippe Aghion, John Campbell John Campbell is the name of: British political figures
His work focuses on the use of incentives, particularly the design of incentive mechanisms to encourage the development of vaccines for use in , Ariel Ariel, in astronomy
Ariel (âr`ēəl), in astronomy, one of the moons, or natural satellites, of Uranus.
Ariel, in the Bible
Ariel (ā`rēĕl) Pakes, Ken Rogoff, Jim Stock, and Jeremy Jeremy (jĕr`ĭmē), English form of Jeremiah. The
Epistle of Jeremy is a title given to the sixth chapter of Baruch. Stein Stein , William Howard 1911-1980.
American biochemist. He shared a 1972 Nobel Prize for pioneering studies of ribonuclease. .
(13.) Azar (2005, 2007) points out first responses have also slowed and first-response times should have a substantial impact on submission decisions because most initial submissions are rejected.
(14.) These are unweighted means across journals.
(15.) See, for example, Bergstrom (2001) and McCabe, Nevo, and Rubinfeld (2005).
(16.) The submit-accept time data is from Table I of Ellison (2002a). In three cases, (JIntE, ,JLE JLE Journal of Lutheran Ethics
JLE Jubilee Line Extension (London Underground)
JLE Justice League Europe
JLE Justice League Elite (forum)
JLE Jump If Less Than or Equal to
JLE Jewish Learning Exchange , and JF) data on 1990 are missing in Ellison (2002a) and the graph instead uses one half of the 1980 1999 difference as the x-variable.
(17.) The large decline in the number of published articles in part reflects the discontinuance Cessation; ending; giving up. The discontinuance of a lawsuit, also known as a dismissal or a non-suit, is the voluntary or involuntary termination of an action.
DISCONTINUANCE, pleading. A chasm or interruption in the pleading.
2. of the journal's microeconomics microeconomics
Study of the economic behaviour of individual consumers, firms, and industries and the distribution of total production and income among them. It considers individuals both as suppliers of land, labour, and capital and as the ultimate consumers of the final series.
(18.) This includes cases where a journal publishes a set of "special" papers and some regular papers in the same issue. To maintain consistency across time, the Carnegie-Rochester Conference Series is treated as a separate journal throughout the period and not counted as part of the Journal of Monetary Economics.
(19.) This is defined roughly as all items on the authors' vitae that are not very short (< four pages), comment-like, or published in the popular press (or other outlets that do not publish academic research). The 2003 group is also well ahead in coauthorship-adjusted publications.
(20.) This is defined roughly as all articles published in journals listed in Econlit other than those in my top 5, 13 field, and invited sets.
(21.) The data on general interest publications do not include short papers and special papers such as presidential addresses. The field-journal counts do not include articles in special issues.
(22.) None of the differences are statistically significant. In the lower bins, this is because the differences are very small (standard errors on each estimate are approximately 0.02). In the two highest bins, this is because the bins contain few economists: 48 and 51 in the earlier decade and 35 and 22 in the later decade. The relative paucity pau·ci·ty
1. Smallness of number; fewness.
2. Scarcity; dearth: a paucity of natural resources. of economists in the later decade could also affect the interpretation of the gap if it were significant.
(23.) Here, the difference in means is significant in the highest bin.
(24.) All counts are weighted for coauthorship. The school-level variable is set to the maximum count for all of an author's affiliations for authors who have different affiliations on different papers. Unlike in previous analyses, this variable is defined at the university level, and I make no attempt to separate economics department faculty from the many other authors with the same university affiliation. The restriction of the sample to economists having multiple general interest publications reflects a data limitation for the earlier decade: Econlit does not contain affiliation data prior to 1989. I obtained affiliation data from Markus Mobius, who collected it for papers beyond an author's first in a decade for use in Mobius and Rosenblat (2004). The restriction of the sample in the second decade is designed to make the regressions comparable, but some potential differences remain: the maximum over schools is potentially taken over a larger set in the second decade and school-level counts are higher because they are not missing for first publications. To minimize the latter difference, the school counts are normalized to have a maximum of one in each decade.
(25.) Most prominently, the raw data from which citations are tabulated only includes the last name and initials of the first author, I deal with this differently in different parts of the analysis, but many of the "counts" reported below should in no way be thought of as aggregations of accurate counts of individuals' citations.
(26.) The raw data do not distinguish between regular AER articles and those in the Papers and Proceedings issues, nor do they distinguish between citations to articles in the Carnegie-Rochester series and articles in the Journal of Monetary Economics after the former's incorporation into the latter. In each case, I apportion ap·por·tion
tr.v. ap·por·tioned, ap·por·tion·ing, ap·por·tions
To divide and assign according to a plan; allot: "The tendency persists to apportion blame as suits the circumstances" all references between the two components by using the relative frequencies for articles that can be definitively matched to an article in one of the two components.
(27.) Three of the main ways in which this calculation departs from the idea are: authors are getting credit for citations made to other economists who share their last name-initial; authors are getting no credit for coauthored papers on which they are not the first author; and the departmental faculty lists only include faculty members who published a paper in one of the 5 general interest or 13 field journals in a 4-yr period and therefore omit citations to faculty members who do not meet this criterion.
(28.) A bit less than one-third of the per-author growth is due to the decrease in the denominator denominator
the bottom line of a fraction; the base population on which population rates such as birth and death rates are calculated.
denominator . Part of this could be an artifact A distortion in an image or sound caused by a limitation or malfunction in the hardware or software. Artifacts may or may not be easily detectable. Under intense inspection, one might find artifacts all the time, but a few pixels out of balance or a few milliseconds of abnormal sound of omitting nonpublishing authors from the calculation.
(29.) See Ellison (2002a) and Althouse et al. (2008) for more on the growth of reference lists in various disciplines. The former contains data for a few top general interest economics journals showing increases ranging from 65% to 139% between 1978 and 1998. The latter finds an average increase of 23% between 1994 and 2004 in a large sample of economics journals.
(30.) It may be interesting to note that here again, Yale and Berkeley look like the top five and Pennsylvania Pennsylvania (pĕnsəlvā`nyə), one of the Middle Atlantic states of the United States. It is bordered by New Jersey, across the Delaware River (E), Delaware (SE), Maryland (S), West Virginia (SW), Ohio (W), and Lake Erie and New York , Northwestern, and School Z look somewhat different.
(31.) See Hausman, Hall, and Griliches (1984) and section 19.9.4 of Greene (1997) for more on this and other models for count data.
(32.) The order variable is one for the lead article, two for the second article, etc.
(33.) Laband (1986) and Johnson (1997) find a similar relationship in other datasets. Medoff (2003) does not.
(34.) To make this comparison, one also needs to add in the AuTop5School coefficient.
(35.) Part of the gain in citations could also be due to a selection effect: the decline in field journal publications could be due to top-department authors only publishing their best field-journal papers in those outlets.
(36.) These unpublished papers, of course, may include papers that will eventually be published in top journals.
GLENN ELLISON, I would like to thank the National Science Foundation (SES-0550897), the Institute for Advanced Study, and the Toulouse Toulouse (tlz`), city (1990 pop. 365,933), capital of Haute-Garonne dept., S France, on the Garonne River. Network for Information Technology for their support. Tom Chang Chang (chăng) or Yangtze (yăng`sē`, yäng`dzŭ`), Mandarin Chang Jiang, longest river of China and of Asia, c.3,880 mi (6,245 km) long, rising in the Tibetan highlands, SW Qinghai prov. and Brandon Brandon, city, Canada
Brandon, city (1991 pop. 38,567), SW Man., Canada, on the Assiniboine River. The business center of the wheat-raising area of SW Manitoba, Brandon has an extensive trade in farm products and machinery. Lehr provided valuable research assistance. I thank Markus Mobius for providing data. Susan Athey Susan Carleton Athey, born on November, 1970 is an American economist. She is currently a professor of economics at Harvard University and the first female winner of the John Bates Clark Medal.
She was born in Boston and grew up in Rockville, Maryland. , Ofer Ofer (Hebrew: עופר) is a moshav located south of Haifa, Israel in the Carmel Mountains and is a part of the Hof HaCarmel Regional Council. Azar, Edward Glaeser Edward L. Glaeser (born May 1, 1967) is an economist at Harvard University. He was educated at The Collegiate School in New York City before obtaining his B.A. in economics from Princeton University and his PhD in economics from the University of Chicago. , Drew Fudenberg, Andrei Shleifer Andrei Shleifer (born February 20, 1961) is a prominent academic economist. He was born in Russia and emigrated to Rochester, NY as a teenager. He then studied economics, obtaining his Ph.D. at MIT in 1986. , and several anonymous referees provided helpful comments. Greg GREG Great Egg Harbor National Scenic and Recreational River (US National Park Service) Mankiw and Phil Davis
Ellison: Department of Economics, MIT, 50 Memorial Drive, E52-380A, Cambridge Cambridge, city, Canada
Cambridge (kām`brĭj), city (1991 pop. 92,772), S Ont., Canada, on the Grand River, NW of Hamilton. It was formed in 1973 with the amalgamation of Galt, Hespeler, and Preston, all founded in the early 19th cent. , MA 02142-1347. Phone 617-253-8702, Fax 617-253-1330, E-mail email@example.com See .edu.
(networking) edu - ("education") The top-level domain for educational establishments in the USA (and some other countries). E.g. "mit.edu". The UK equivalent is "ac.uk".
TABLE 1 Publications by Members of Ten Highly Ranked Departments in 13 Field Journals 1990-1993 "Top" Dept. "Top" Dept. Total Journal/school Count as % Pubs. Coauthorship-weighted publication counts All 13 Field/Departments 1-5 86.4 4.0 2,148 All 13 Field/Departments 6-10 88.8 4.1 2,148 Breakdown by school Harvard 20.8 1.0 2,148 Chicago 8.7 0.4 2,148 MIT 21.0 1.0 2,148 Stanford 12.2 0.6 2,148 Princeton 23.8 1.1 2,148 Yale 14.2 0.7 2,148 Berkeley 20.4 1.0 2,148 Pennsylvania 18.5 0.9 2,148 Northwestern 14.7 0.7 2,148 School Z 21.0 1.0 2,148 Breakdown by journal for top five departments Gaines and Economic Behavior 1.5 1.7 88 Journal of Development Economics 3.3 1.7 193 Journal of Econometrics 7.5 4.5 167 Journal of Economic Theory 18.2 6.7 271 Journal of Finance 3.9 1.3 293 Journal of International 7.0 4.3 162 Economics Journal of Labor Economics 4.5 5.6 80 Journal of Law and Economics 5.0 7.2 69 Journal of Law, Ec., and 2.7 3.3 80 Organization Journal of Monetary Economics 9.0 5.0 181 Journal of Public Economics 7.0 3.1 225 Journal of Urban Economics 3.5 1.9 183 RAND Journal of Economics 13.3 8.5 156 Breakdown for Northwestern, Penn & School Z Games and Economic Behavior 0.5 0.6 88 Journal of Economic Theory 14.8 5.5 271 Journal of Monetary Economics 6.3 3.5 181 Ten others 32.5 2.0 1,608 2000-2003 "Top" Dept. "Top" Dept. Total Journal/school Count as % Pubs. Coauthorship-weighted publication counts All 13 Field/Departments 1-5 71.2 2.7 2,643 All 13 Field/Departments 6-10 83.9 3.2 2,643 Breakdown by school Harvard 18.1 0.7 2,643 Chicago 3.5 0.1 2,643 MIT 21.3 0.8 2,643 Stanford 14.6 0.6 2,643 Princeton 13.7 0.5 2,643 Yale 12.7 0.5 2,643 Berkeley 11.8 0.4 2,643 Pennsylvania 22.5 0.9 2,643 Northwestern 16.2 0.6 2,643 School Z 20.8 0.8 2,643 Breakdown by journal for top five departments Gaines and Economic Behavior 2.8 1.2 237 Journal of Development Economics t.5 0.6 241 Journal of Econometrics 6.3 2.9 220 Journal of Economic Theory 10.4 3.0 344 Journal of Finance 9.1 2.9 310 Journal of International 2.5 1.3 197 Economics Journal of Labor Economics 7.2 5.6 128 Journal of Law and Economics 2.5 2.8 90 Journal of Law, Ec., and 2.0 2.5 79 Organization Journal of Monetary Economics 3.0 1.6 190 Journal of Public Economics 11.5 4.1 281 Journal of Urban Economics 3.3 1.6 203 RAND Journal of Economics 9.0 7.3 123 Breakdown for Northwestern, Penn & School Z Games and Economic Behavior 8.0 3.4 237 Journal of Economic Theory 17.9 5.2 344 Journal of Monetary Economics 11.2 5.9 190 Ten others 22.3 1.2 1,872 TABLE 2 Publications by Members of Six Highly Ranked Departments in Five Top General Interest Journals 1990-1993 "Top" Dept. "Top" Dept. Total Journal/school Count as % Pubs. Coauthorship-weighted publication counts All 5/Departments 1-5 167.8 13.4 1,248 All 5/Departments 6-10 126.4 10.1 1,248 Breakdown by school Harvard 49.0 3.9 1,248 Chicago 21.3 1.7 1,248 MIT 36.3 2.9 1,248 Stanford 12.2 1.0 1,248 Princeton 49.1 3.9 1,248 Yale 25.3 2.0 1,248 Berkeley 22.6 1.8 1,248 Pennsylvania 31.7 2.5 1,248 Northwestern 29.0 2.3 1,248 School Z 17.8 1.4 1,248 Breakdown by journal: all departments American Economic Review 67.3 18.0 375 Econometrica 63.4 26.0 244 Journal of Political Economy 48.6 21.5 226 Quarterly Journal of Economics 59.5 27.9 213 Review of Economic Studies 55.4 29.2 190 Breakdown by journal: Harvard American Economic Review 10.8 2.9 375 Econometrica 8.7 3.6 244 Journal of Political Economy 5.5 2.4 226 Quarterly Journal of Economics 14.8 7.0 213 Review of Economic Studies 9.2 4.8 190 2000-2003 "Top" Dept. "Top" Dept. Total Journal/school Count as % Pubs. Coauthorship-weighted publication counts All 5/Departments 1-5 164.3 14.4 1,141 All 5/Departments 6-10 111.1 9.7 1,141 Breakdown by school Harvard 32.1 2.8 1,141 Chicago 22.5 2.0 1,141 MIT 38.5 3.4 1,141 Stanford 24.9 2.2 1,141 Princeton 46.3 4.1 1,141 Yale 24.3 2.1 1,141 Berkeley 22.7 2.0 1,141 Pennsylvania 21.8 1.9 1,141 Northwestern 22.7 2.0 1,141 School Z 19.6 1.7 1,141 Breakdown by journal: all departments American Economic Review 57.1 15.1 377 Econometrica 68.8 26.3 261 Journal of Political Economy 46.3 24.3 190 Quarterly Journal of Economics 64.4 39.0 165 Review of Economic Studies 38.9 26.3 148 Breakdown by journal: Harvard American Economic Review 8.1 2.1 377 Econometrica 4.0 1.5 261 Journal of Political Economy 2.5 1.3 190 Quarterly Journal of Economics 14.9 9.0 165 Review of Economic Studies 2.6 1.7 148 TABLE 3 Summary of Fit of Theories to Evidence Theories Decline in Top Slowdown of Publication Dept. Quality Process Facts 1 and 2 +/- + More pub. counts - + Author CV data - + Author pub. data - Jrnl. citation counts Dept. citation counts +/- Paper citation data +/- Theories Decline in Fld. Decrease in Peer Review Jrnl. Quality Necessity Facts 1 and 2 + + More pub. counts Author CV data + Author pub. data - + Jrnl. citation counts + Dept. citation counts +/ Paper citation data - +/- TABLE 4 Publication Counts for "Invited" Journals 1990-1993 "Top" Dept. "Top" Dept. Total Journal Count as % Pubs. Top five departments AEA Papers & Proceedings 55.8 17.1 326 Brookings Papers 34.8 34.8 100 Carnegie-Rochester Conf. Series 5.5 9.5 58 Journal of Economic Literature 15.0 20.0 75 Journal of Economic Perspectives 23.2 11.4 203 NBER Macroeconomics Annual 8.0 34.8 23 Departments 6-10 AEA Papers & Proceedings 41.2 12.6 326 Brookings Papers 19.1 19.1 100 Carnegie-Rochester Con. Series 4.8 8.3 58 Journal of Economic Literature 9.5 12.7 75 Journal of Economic Perspectives 20.3 10.0 203 NBER Macroeconomics Annual 0.5 2.2 23 2000-2003 "Top" Dept. "Top" Dept. Total Journal Count as % Pubs. Top five departments AEA Papers & Proceedings 63.8 18.8 340 Brookings Papers 11.0 20.8 53 Carnegie-Rochester Conf. Series 5.6 11.4 49 Journal of Economic Literature 12.5 15.8 79 Journal of Economic Perspectives 31.3 17.2 182 NBER Macroeconomics Annual 8.0 33.3 24 Departments 6-10 AEA Papers & Proceedings 35.3 10.4 340 Brookings Papers 6.3 11.9 53 Carnegie-Rochester Con. Series 3.5 7.1 49 Journal of Economic Literature 7.7 9.7 79 Journal of Economic Perspectives 16.5 9.1 182 NBER Macroeconomics Annual 3.0 12.5 24 TABLE 5 Publication Counts for Special Issues of Field Journals and Invited Journals 1990-1993 Set of Departments Dept. Count Dept. as % Total Pubs. Departments 1-5 33.3 8.2 408 Departments 6-10 36.8 9.0 408 2000-2003 Set of Departments Dept. Count Dept. as % Total Pubs. Departments 1-5 39.9 8.6 464 Departments 6-10 29.2 6.3 464 TABLE 6 Publications by Faculty of Three Business Schools 1990-1993 Journal/school Dept. Count Dept. as % Total Pubs. Coauthorship-weighted publication counts Top five general interest journals Chicago GSB 29.8 2.4 1,248 Northwestern Kellogg 34.5 2.8 1,248 Stanford GSB 19.7 1.6 1,248 13 field 13 field 13 field journals journals journals Chicago GSB 24.2 1.1 1,248 Northwestern Kellogg 44.7 2.1 1,248 Stanford GSB 18.7 0.9 1,248 2000-2003 Journal/school Dept. Count Dept. as % Total Pubs. Coauthorship-weighted publication counts Top five general interest journals Chicago GSB 39.3 3.4 1,141 Northwestern Kellogg 10.6 0.9 1,141 Stanford GSB 11.8 1.0 1,141 13 field 13 field 13 field journals journals journals Chicago GSB 27.2 1.0 1,141 Northwestern Kellogg 31.0 1.2 1,141 Stanford GSB 17.5 0.7 1,141 TABLE 7 Publications by Young Senior Faculty at Harvard Other 13 Fld Other Invt'd Author Total QJE Top Five Jrnls Ref'd Jrnls Other 1990-1993 data A 18 0.5 2.3 0.0 3.8 0.0 1.7 B 16 0.0 2.8 3.8 0.0 0.5 1.0 C 19 1.3 0.5 1.0 1.0 1.5 3.6 D 10 0.3 0.3 0.0 3.3 1.5 0.0 E 28 1.2 0.5 2.8 2.8 3.3 0.0 F 6 0.0 1.5 1.0 0.5 0.5 0.0 Average 16 0.6 1.3 1.4 1.9 1.2 1.1 2000-2003 data G 31 0.0 0.0 0.5 1.1 3.3 10.6 H 41 1.1 1.0 1.8 6.4 1.2 10.2 1 13 2.0 1.0 0.0 1.0 0.0 9.0 1 13 0.0 0.9 0.0 1.0 2.3 5.0 K 14 1.3 0.5 0.0 0.3 1.5 2.6 Average 22 0.9 0.7 0.5 2.0 1.7 7.5 TABLE 8 Publications as a Function of Author and School Characteristics Dep Var: Pub. Counts for Journals/Time Period 13 Field Journals Top Five General 90-93 00-03 90-93 00-03 AuthorTop5Pubs 0.09 0.21 0.19 0.29 in last decade (2.6) (6.1) (7.5) (11.6) SchoolTop5Pubs 0.15 -0.44 0.84 0.46 in last decade (0.8) (2.3) (5.4) (2.9) Constant -1.13 -1.19 -1.51 -1.55 (9.5) (11.2) (14.0) (15.3) Pseudo [R.sup.2] 0.01 0.02 0.07 0.09 N 767 755 767 755 Dep Var: Pub. Counts for Journals/Time Period 13 Field Top Five 00-03 00-03 AuthorTop5Pubs 0.23 0.35 in last decade (8.2) (17.4) SchoolTop5Pubs 0.03 0.82 in last decade (0.2) (6.3) Constant -1.49 -2.08 (-25.1) (30.6) Pseudo [R.sup.2] 0.02 0.13 N 2,050 2,050 t-statistics in parentheses. TABLE 9 Per-Article Citation Counts for Recent Articles in Various Journals: 1994 and 2004 Cites/Art Journal 1994 2004 Top five journals QJE 0.79 1.75 Econometrrca 1.28 1.06 REStud 0.68 0.98 JPE 1.02 0.88 AER 0.63 0.87 Invited Journals NBER Macro Ann. 1.33 1.94 JEL 0.75 0.88 Brookings 0.63 0.62 JEP 0.33 0.39 AEA P&P 0.25 0.32 Carnegie-Rochester 0.20 0.22 Cites/Art Journal 1994 2004 Field journals J Monetary Ec 0.69 0.59 J Finance 0.41 0.56 J Intern'l Ec 0.33 0.48 RAND J Ec 0.66 0.45 J Labor Ec 0.32 0.40 Games Ec Behav 0.62 0.37 J Econometrics 0.36 0.33 J Public Ec 0.28 0.33 J Econ Theory 0.43 0.31 JLEO 0.34 0.30 J Law and Ec 0.31 0.27 J Urban Ec 0.17 0.27 J Development Ec 0.09 0.20 21 Journal Cites Top Five Cites Journal Set 1994 2004 1994 2004 Top five 0.88 1.11 0.32 0.39 Field 13 0.39 0.37 0.07 0.08 Invited 0.58 0.73 0.17 0.17 TABLE 10 Departmental Citation Counts: Average Citations per Faculty Member Average Citations per Author Citing Journal/Cited Paper Set: 21 Jrnls/Any Top Five/Any Econ. Dept. 1994 2004 1994 2004 Harvard 18.9 31.2 6.5 10.1 Chicago 36.1 30.8 11.8 9.9 MIT 24.7 27.9 8.5 9.1 Stanford 15.6 13.9 5.9 3.6 Princeton 19.8 18.4 6.9 5.5 Yale 13.4 12.5 4.6 3.9 Berkeley 16.4 17.5 4.2 3.8 Pennsylvania 7.6 8.2 2.3 2.8 Northwestern 9.3 15.4 3.0 5.0 School Z 5.9 7.4 1.9 2.5 Other 3.0 3.3 0.7 0.8 Harvard 18.9 31.2 6.5 10.1 Depts. 2-5 22.6 21.6 7.8 6.6 Depts. 6-10 10.5 12.0 3.2 3.5 Other 3.0 3.3 0.7 0.8 Average Citations per Author Citing Journal/ Cited Paper Set: 21/Recent Number of Authors Econ. Dept. 1994 2004 1994 2004 Harvard 7.0 7.1 49 43 Chicago 8.5 5.5 19 23 MIT 8.1 7.6 29 36 Stanford 4.4 2.9 28 36 Princeton 7.8 5.8 36 44 Yale 4.5 3.1 35 29 Berkeley 5.6 4.1 29 36 Pennsylvania 2.8 2.2 36 29 Northwestern 3.8 4.5 29 25 School Z 1.7 2.3 29 39 Other 1.2 1.1 3,285 4,282 Harvard 7.0 7.1 49 43 Depts. 2-5 7.1 5.3 110 137 Depts. 6-10 3.6 3.2 156 157 Other 1.2 1.1 3,285 4,408 TABLE 11 Paper-Level Citation Regressions Dependent Variable: Citations in 1994 or 2004 1990-1993 2000-2003 1990-1993 Log(Order) -0.16 -0.17 -0.16 (4.5) (5.5) (4.4) Pages 0.03 0.02 0.03 (10.5) (9.9) (10.6) NumAuthor 0.20 0.14 0.19 (4.6) (4.1) (4.5) Age 3 0.02 -0.04 0.02 (0.3) (0.6) (0.2) Age 2 -0.14 -0.31 -0.15 (1.8) (4.4) (1.8) Age 1 -0.77 -0.75 -0.77 (8.7) (9.7) (8.7) Constant -0.86 -0.56 -0.87 (6.0) (4.4) (6.0) AuTop5School 0.61 0.48 0.55 (6.4) (5.2) (4.2) Au6-10School 0.41 0.10 0.57 (3.7) (0.8) (3.5) FieldJournal -0.87 -0.94 (13.6) (16.0) InvitedJournal -0.74 -0.50 (8.0) (5.9) FldJrnl x Top5Sch -0.76 (3.6) FldJrnl x 6-10Sch -1.00 (4.2) FldJrnl x Other -0.87 (12.0) InvJrnl x Top5Sch -0.57 (2.7) InvJrnl x 6-10Sch -1.26 (4.4) InvJrnl x Other -0.69 (6.3) Ten school dummies No No No Journal dummies No No No Pseudo [R.sup.2] 0.08 0.07 0.08 Number of obs. 4,580 4,970 4,580 Dependent Variable: Citations in 1994 or 2004 2000-2003 1990-1993 2000-2003 Log(Order) -0.17 -0.21 -0.25 (5.5) (5.3) (7.3) Pages 0.02 0.04 0.03 (10.0) (12.2) (9.8) NumAuthor 0.14 0.19 0.12 (4.0) (4.5) (3.4) Age 3 -0.05 0.03 -0.03 (0.7) (0.3) (0.5) Age 2 -0.31 -0.17 -0.29 (4.4) (2.2) (4.2) Age 1 -0.76 -0.77 -0.73 (9.7) (8.9) (9.5) Constant -0.52 -0.92 -0.22 (4.0) (5.5) (1.5) AuTop5School 0.25 (1.9) Au6-10School 0.09 (0.5) FieldJournal InvitedJournal FldJrnl x Top5Sch -0.29 0.02 0.78 (1.4) (0.1) (3.7) FldJrnl x 6-10Sch -0.83 -0.20 0.25 (3.2) (0.8) (0.9) FldJrnl x Other -1.02 (15.5) InvJrnl x Top5Sch -0.41 0.32 0.09 (2.0) (1.4) (0.4) InvJrnl x 6-10Sch -0.87 -0.38 -0.53 (2.7) (1.2) (1.6) InvJrnl x Other -0.48 (4.7) Ten school dummies No Yes Yes Journal dummies No Yes Yes Pseudo [R.sup.2] 0.07 0.10 0.09 Number of obs. 4,970 4,580 4,970 t-statistics in parentheses. TABLE 12 School Dummies from Paper-Level Citation Regression School Dummies from Citation Regressions 1990-1993 2000-2003 Harvard 0.68 (3.6) 0.18 (0.9) Chicago 0.65 (2.6) 0.37 (1.5) MIT 0.54 (2.5) 0.22 (1.1) Stanford 0.42 (1.4) -0.46 (1.8) Princeton 0.37 (1.9) 0.37 (1.9) Yale 0.59 (2.3) -0.33 (1.1) Berkeley 0.63 (2.5) 0.37 (1.5) Pennsylvania 0.35 (1.4) 0.14 (0.5) Northwestern 0.67 (2.6) 0.37 (1.4) School Z 0.48 (1.5) -0.04 (0.1) t-statistics in parentheses.