Citation performance of behaviorally oriented journals.
Key words: behavioral psychology, citation analysis, journals
Content analyses of the behavioral literature have become increasingly popular in recent years. The content of behavioral journals has been examined for authorship trends (e.g., Mathews, 1997), assessment practices (e.g., Carr, Austin, Britton, Kellum, & Bailey, 1999), and general research characteristics (e.g., Northup, Vollmer, & Serrett, 1993), among other features. In addition, several articles have investigated the citation practices of specific journals (e.g., Laties & Mace, 1993). For example, Poling, Alling, and Fuqua (1994) evaluated self-and cross-citations between Journal of Applied Behavior Analysis (JABA) and Journal of the Experimental Analysis of Behavior (JEAB) and found relatively high self-citation rates within each journal and low cross-citation rates between them. Critchfield (2002) used citation measures to document relations among and between certain clusters of journals (e.g., basic and applied behavioral journals). The author reported, among other findings, that JABA's recent influence on other applied behavioral journals was relatively minor. Recently, Carr and Britton (2003) provided citation frequencies and impact factors for six applied behavioral journals over a 20-year period and found that the journals generally maintained their impact on the literature. The authors also reported additional citation measures (i.e., impact factor rank, self-citation rate, immediacy index, cited half-life) for nine applied behavioral journals for the 2000 publication year.
The purpose of the present article is to extend the Carr and Britton (2003) article by providing recent citation data for a larger and more diverse array of behaviorally oriented journals. The intended benefit of this effort is to summarize data that are widely available, but time-consuming to collate, such that the behavioral community has ready access to multiple quantitative measures for specific journals. These data reveal how frequently articles from specific journals are cited, how a journal's citation performance is ranked compared to other journals, the proportion of a journal's citations that are self-citations, how rapidly a journal's articles are cited, and whether a journal's citations are from its recently published articles (compared to ones from earlier years).
Journals were primarily selected from the PsycSCAN: Behavior Analysis & Therapy coverage list of behavioral journals. PsycSCAN: Behavior Analysis & Therapy was published by the American Psychological Association (APA) from 1995 to 2001 and biannually reprinted the bibliographic information and abstracts of articles published in the journals on its coverage list. The coverage list was developed by individuals selected by the Executive Committee of APA Division 25 (Behavior Analysis) and members of the APA Publications and Communications Board. Two additional behaviorally oriented journals (The Behavior Analyst Today, Cognitive and Behavioral Practice) were added to the 38 found on the PsycSCAN coverage list because they began publication after the coverage list was developed. The 40 journals that comprised our final list can be found in Tables 1 and 2.
Journal Citation Database
All citation data were obtained from Journal Citation Reports--Social Sciences Edition (JCR; Institute for Scientific Information, 2002). The JCR database is comprised of approximately 1,700 journals from all of the social sciences. According to Testa (2004), journals are selected for the JCR database via a peer-review process that considers (a) the consistency with which a journal is published, (b) whether a journal follows international publication conventions (e.g., informative titles, descriptive abstracts), (c) whether a journal employs peer review, (d) the novelty of a journal's content compared to journals that are already indexed, (e) the international diversity of a journal's contributions, and (f) a journal's citation performance based on several measures (e.g., citation rate, impact factor, self-citation rate) that are described below.
Journal Citation Measures
The following measures were obtained from the 2002 edition of JCR for each of our target journals that were included in its database.
Times cited (citation frequency). This measure represents the number of times a journal's articles were cited by all journals in the JCR database in a given year. Although this measure provides an indication of a journal's impact on the scientific literature, it does not take into account the number of articles published by a journal each year, which can easily influence the measure.
Citation impact factor. The impact factor is a citation measure that reflects the influence of a journal's recently published articles and accounts for the number of articles published each year. The impact factor is calculated by dividing [the number of times articles published by a journal during the two previous years were cited during the current year] by [the total number of articles published in the journal during the two previous years]. For example, to calculate the impact factor for a journal for the year 2002, the number of times its articles from 2000 and 2001 were cited during 2002 would be divided by the total number of articles published during 2000 and 2001. A journal with an impact factor of less than 1.0 is generally characterized as "low impact."
Impact factor rank. JCR ranks the impact factors of its journals, which are categorized into scholarly domains (e.g., biological psychology, educational psychology). A journal can be ranked in multiple domains depending on how its content was classified by the Institute for Scientific Information. The ranks of each of our target journals were recorded and a percentile score was calculated to represent the percentage of journals ranked below them in their respective categories. For example, if a journal has a percentile score of 80, this indicates that 80% of the other journals ranked in the same domain have lower impact factors.
Self-citation rate (self-cited). This measure represents the percentage of times a journal's articles were cited within the journal relative to citations found in other journals (i.e., # of self citations / # of times cited). For example, a journal would have a self-citation rate of 20% if 300 of the 1,500 citations of the journal in a given year were from the journal itself. Journals with high self-citation rates are generally characterized as "isolated."
Immediacy index. The immediacy index is an estimate of the latency of a journal's impact on the scientific community. The immediacy index is calculated by dividing the number of citations to articles published in the current year by the total number of articles published that year. For example, a journal's immediacy index would be .25 if there were 25 citations to its articles that were published in 2002 and it published a total of 100 articles in 2002.
Cited half-life. This measure represents the number of publication years before the current year that account for half of the citations received in the current year. For example, if a journal was cited 500 times in 2002, and 250 of those were from articles published between 1998 and 2002, its cited half-life measure would be 5 (years). The cited half-life measure provides an indication of the impact of a journal's contemporary articles compared to ones published earlier.
RESULTS AND DISCUSSION
Table 1 displays six citation measures for the 2002 publication year for 27 journals that were indexed by JCR. The 13 journals from our target list that were not indexed by JCR are shown in Table 2. Although the raw data in Table 1 are available for independent interpretation, several findings are worth mentioning. Nine (33.3%) journals had more than 1,000 citations and 5 (18.5%) of these had more than 2,000 citations, including two of the behavior-analytic flagship journals, JABA and JEAB. Fourteen (51.9%) journals had impact factors greater than 1.0 (see Figure 1). Twelve (44.4%) of the journals were ranked in the top halves of their respective domains based on their impact factors. Four (14.8%) journals were ranked in the top quartile of their respective domains: Applied Animal Behavior Science (79th percentile, Agriculture, Dairy & Animal Science), Behavioral and Brain Sciences (93rd percentile, Biological Psychology), Behaviour Research and Therapy (88th percentile, Clinical Psychology), and Experimental and Clinical Psychopharmacology (ECP; 78th percentile, Clinical Psychology). The flagship behavior-analytic journals The Behavior Analyst and JABA were ranked at the 39th and 54th percentiles, respectively, in the Clinical Psychology domain and JEAB was ranked at the 40th percentile in Biological Psychology and at the 64th percentile in Experimental Psychology. Interestingly, JCR categorized the 27 journals in Table 1 into 14 different content-area domains. Those domains are: agriculture, applied psychology, behavioral sciences, biological psychology, clinical psychology, dairy and animal science, ethics, experimental psychology, family studies, management, multidisciplinary psychology, neurosciences, pharmacology and pharmacy, and psychiatry.
[FIGURE 1 OMITTED]
According to Testa (2004), approximately 80% of the journals indexed in JCR have self-citation rates less than 20%. Similarly, Table 1 shows that 19 (70.4%) of the behaviorally oriented journals indexed in JCR have self-citation rates less than 20%. The three highest self-citation rates from Behavior and Philosophy (58.8%), Integrative Physiological and Behavioral Science (62.8%), and Journal of Organizational Management (78.9%) might possibly be influenced by the journals' low citation frequencies (34, 129, and 190, respectively). The flagship behavior-analytic journals JABA, JEAB, and The Behavior Analyst had self-citation rates of 17.6%, 27.9%, and 22.9%, respectively. Combined, these three journals publish approximately two thirds of all content among journals devoted exclusively to behavior analysis (Shabani, Carr, Petursdottir, Esch, & Gillett, 2004). Thus, these relatively high self-citation rates indicate that a significant portion of the behavior-analytic literature is rather "isolated," at least based on this single citation measure. These data, taken together with those reported by Critchfield (2002), indicate that behavior analysis, although thriving in a number of areas, might exert relatively little influence on scholarship outside of the discipline. Critchfield proposed several remedies to this problem, including publishing behavior-analytic work in non-behavioral journals and publishing on topics traditionally not associated with behavior analysis. However, it is unknown whether resources exist to mount these efforts to the degree necessary for non-behavioral researchers to come in greater contact with our literature.
Immediacy-index data indicate that, with the exception of ECP (0.98), none of the journals was very likely to have articles cited during their publication year. Finally, cited half-life figures ranged from 3.9 years (Applied Psychophysiology and Biofeedback) to over 10 years (e.g., JABA). Seven (25.9%) journals had cited half-life measures over 10 years indicating that the majority of the articles cited during 2002 were published before 1992.
The data presented in this article should be evaluated in the context of several cautions. First, the use of citation frequency and impact factor as indicators of journal impact has been questioned (Boor, 1982). A journal's impact on a discipline's non-research activities (e.g., application) is clearly not represented by these measures. In addition, it is possible for an article to influence other scientific work without being cited. Nonetheless, a recent survey of authors showed that a journal's impact factor was an important variable in determining to which journals manuscripts are submitted (Swan, 1999). Second, JCR does not index every journal. Table 2 shows that 13 (32.5%) of the journals on our target list were not indexed by JCR and, therefore, cannot be compared using citation measures to the journals that are indexed in the database. Third, not all of the journals on our target list exclusively publish behaviorally oriented articles (e.g., Behavioral and Brain Sciences); thus their citation data are a reflection of this diversity of content.
The journals listed in Tables 1 and 2 encompass 40 publication outlets for behaviorally oriented scholarly contributions. We hope the citation data provided herein are useful (as one metric) in helping authors evaluate these journals.
Boor, M. (1982). The citation impact factor: Another dubious index of journal quality. American Psychologist, 37, 975-977.
Carr, J. E., Austin, J. L., Britton, L. N., Kellum, K. K., & Bailey, J. S. (1999). An assessment of social validity trends in applied behavior analysis. Behavioral Interventions, 14, 223-231.
Carr, J. E., & Britton, L. N. (2003). Citation trends of applied journals in behavioral psychology: 1981-2000. Journal of Applied Behavior Analysis, 36, 113-117.
Critchfield, T. S. (2002). Evaluating the function of applied behavior analysis: A bibliometric analysis. Journal of Applied Behavior Analysis, 35, 423-426.
Institute for Scientific Information. (2002). Journal Citation Reports --Social Sciences Edition: A bibliometric analysis of social science journals in the ISI database. Philadelphia: Author.
Laties, V. G., & Mace, F. C. (1993). Taking stock: The first 25 years of the Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 26, 513-525.
Mathews, R. M. (1997). Editors as authors: Publication trends of articles authored by JABA editors. Journal of Applied Behavior Analysis, 30, 717-721.
Northup, J., Vollmer, T. R., & Serrett, K. (1993). Publication trends in 25 years of the Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 26, 527-537.
Poling, A., Alling, K., & Fuqua, R. W. (1994). Self- and cross-citations in the Journal of Applied Behavior Analysis and the Journal of the Experimental Analysis of Behavior: 1983-1992. Journal of Applied Behavior Analysis, 27, 729-731.
Shabani, D. B., Carr, J. E., Petursdottir, A. I., Esch, B. E., & Gillett, J. N. (2004). Scholarly productivity in behavior analysis: The most prolific authors and institutions from 1992 to 2001. The Behavior Analyst Today, 5, 235-243.
Swan, A. (1999). What authors want: The ALPSP research study on the motivations and concerns of contributors to learned journals. Retrieved November 11, 2004 from http://www.isinet.com/essays/ selectionofmaterialforcoverage/199701.html/.
Testa, J. (2004). The ISI[R] database: The journal selection process. Retrieved November 11, 2004 from http://www.isinet.com/essays/ selectionofmaterialforcoverage/199701.html/.
Address correspondence to James E. Carr, Ph.D. Department of Psychology Western Michigan University 1903 W. Michigan Ave. Kalamazoo, MI 49008-5439 . Tel. (269) 387-4925; Fax (269) 3874550; e-mail: email@example.com
JAMES E. CARR AND KELISE K. STEWART
WESTERN MICHIGAN UNIVERSITY
Table 1. JCR citation data for 27 behaviorally oriented journals based on the 2002 publication year. Founding Times Impact Journals Year Cited Factor Animal Learning and Behavior (3) 1973 1148 1.05 Applied Animal Behaviour Science (4) 1975 2323 1.20 Applied Psychophysiology and Biofeedback 1976 103 0.86 Behavior Analyst 1978 140 0.83 Behavior and Philosophy 1972 34 0.59 Behavior Modification 1977 463 0.79 Behavior Therapy 1970 1717 1.28 Behavioral and Brain Sciences 1978 3451 8.73 Behavioral Interventions 1986 62 0.24 Behaviour Change 1984 121 0.33 Behaviour Research and Therapy 1963 4358 2.19 Behavioural Pharmacology (4) 1989 1481 1.94 Behavioural Processes 1977 774 0.88 Child and Family Behavior Therapy 1979 118 0.23 Cognitive and Behavioral Practice 1994 127 0.33 Experimental and Clinical Psychopharmacology 1993 766 1.65 Integrative Physiological and Behavioral Science 1966 129 0.29 Journal of Applied Behavior Analysis 1968 2015 1.11 Journal of Behavior Therapy and Experimental Psychiatry 1970 657 0.50 Journal of Experimental Psychology: Animal Behavior Processes 1975 1540 1.96 Journal of Organizational Behavior Management 1977 190 1.27 Journal of Psychopathology and Behavioral Assessment 1979 344 0.62 Journal of the Experimental Analysis of Behavior 1958 2111 1.58 Learning and Motivation 1970 799 1.14 Quarterly Journal of Experimental Psychology: Comparative and Physiological Psychology 1949 536 1.26 Psychological Record 1937 470 1.01 Verhaltenstherapie 1991 99 0.37 Journals Impact Factor Rank (1) (% below) Animal Learning and Behavior (3) 12 b (20), 43 e (35) Applied Animal Behaviour Science (4) 9 (79) ag Applied Psychophysiology and Biofeedback 48 (41) Behavior Analyst 50 (39) Behavior and Philosophy 57 (44) m, 14 (50) et Behavior Modification 52 (37) Behavior Therapy 31 (62) Behavioral and Brain Sciences 1 (93) b Behavioral Interventions 76 (7) Behaviour Change 71 (13) Behaviour Research and Therapy 10 (88) Behavioural Pharmacology (4) 20 (49) bs, 103 (47) n, 67 (64) ph Behavioural Processes 13 (31) b Child and Family Behavior Therapy 77 (6), 27 (18) f Cognitive and Behavioral Practice 70 (15) Experimental and Clinical Psychopharmacology 18 (78), 7 (53) b Integrative Physiological and Behavioral Science 15 (0) b Journal of Applied Behavior Analysis 38 (54) Journal of Behavior Therapy and Experimental Psychiatry 65 (21), 60 (23) p Journal of Experimental Psychology: Animal Behavior Processes 6 (60) b, 18 (73) e Journal of Organizational Behavior Management 15 (70) a, 18 (72) ma Journal of Psychopathology and Behavioral Assessment 63 (23) Journal of the Experimental Analysis of Behavior 9 (40) b, 24 (64) e Learning and Motivation 11 (27) b, 38 (42) e Quarterly Journal of Experimental Psychology: Comparative and Physiological Psychology 10 (33) b, 35 (47) e Psychological Record 30 (71) m Verhaltenstherapie 67 (18) Self- Citation Immediacy Journals Rate Index Animal Learning and Behavior (3) 10.9% 0.06 Applied Animal Behaviour Science (4) 36.3% 0.39 Applied Psychophysiology and Biofeedback 19.4% 0.27 Behavior Analyst 22.9% 0.00 Behavior and Philosophy 58.8% no data Behavior Modification 6.9% 0.43 Behavior Therapy 4.3% 0.15 Behavioral and Brain Sciences 2.3% no data Behavioral Interventions 14.5% 0.05 Behaviour Change 3.3% 0.00 Behaviour Research and Therapy 11.7% 0.41 Behavioural Pharmacology (4) 5.5% 0.25 Behavioural Processes 7.1% 0.11 Child and Family Behavior Therapy 6.8% 0.00 Cognitive and Behavioral Practice 7.9% no data Experimental and Clinical Psychopharmacology 11.6% 0.98 Integrative Physiological and Behavioral Science 62.8% 0.00 Journal of Applied Behavior Analysis 17.6% 0.28 Journal of Behavior Therapy and Experimental Psychiatry 11.0% 0.22 Journal of Experimental Psychology: Animal Behavior Processes 11.0% 0.50 Journal of Organizational Behavior Management 78.9% no data Journal of Psychopathology and Behavioral Assessment 3.5% 0.00 Journal of the Experimental Analysis of Behavior 27.9% 0.14 Learning and Motivation 8.5% 0.24 Quarterly Journal of Experimental Psychology: Comparative and Physiological Psychology 11.8% 0.37 Psychological Record 28.1% 0.21 Verhaltenstherapie 27.3% 0.33 (1) Journals were ranked with others in the domain of "Clinical Psychology" except when noted as follows: a = applied psychology; ag = agriculture, dairy & animal science; b = biological psychology; bs = behavioral sciences; e = experimental psychology; et = ethics; f = family studies; m = multidisciplinary psychology; ma = management; n = neurosciences; p = psychiatry; ph = pharmacology & pharmacy. (2) Cited half-life data are not available for journals that receive fewer than 100 citations in the target year. (3) Retitled Learning and Behavior in 2003. (4) Data for these journals were obtained from Journal Citation Reports--Science Edition. Table 2. Behaviorally oriented journals that were not indexed in the ISI database. Journal Founding Year Acta Comportamentalia 1992 Analysis of Verbal Behavior 1982 Behavior Analyst Today 1999 Behavior and Social Issues 1991 Behavioural and Cognitive Psychotherapy 1978 Cognitive Behaviour Therapy 1972 Education and Treatment of Children 1977 Japanese Journal of Behavior Analysis 1987 Japanese Journal of Behavior Therapy 1976 Journal of Behavioral Education 1991 Journal of Rational-Emotive and Cognitive Behavior Therapy 1983 Revista Mexicana de Analisis de la Conducta 1975 Science et Comportement 1970