Printer Friendly

An update of the vox populi approach to academic journal rankings: 2011 in review.

Introduction

Journal rankings and academic quality continues to attract attention from social science academics, education management and policy makers (Lee, 2011). Publishing in "A" journals have become synonymous for academic quality, reflecting the research value of academics, and representing institutional excellence. Fortunately (or unfortunately), a number of "quality" journal lists exist (e.g., Scopus, SSCI, UTD, RAE/REF, ABDC) which have been used to influence academic hiring, tenure, promotion decisions, and individual evaluation (Saunders and Wong, 2011).

This study is the second in a planned series of triennial longitudinal studies examining the ranking of journals by academics around the world, including the personal and career impact on academics of publishing in ranked journals. The current paper only presents the journal ranking data for discussion. As in the previous study (Fam et al., 2011), the research attempts to understand the effect of journal ranking from the 'contributor' perspective, using the voxpopuli approach; that any judgment based on the intelligence of the masses will be free of passion and uninfluenced by rhetoric (Galton, 1907). More recently, research on the "wisdom of crowds" has found public opinion to be an effective predictive tool because of the capacity to synthesise large amounts of information and improve their judgement (Hastie and Kameda, 2005; Mannes, 2009; Soll and Larrick, 2009; Surowiecki, 2004). This approach attempts to address the direction suggested by Steward and Lewis (2010) that "... efforts aimed at creating new perspectives on appraising the quality of journals in Marketing should be encouraged".

Educational institutions continue to use publishing metrics to measure their influence, progress and research merit. There are a number of ostensibly objective methods used to rank or rate academic journals; however, there is no one 'correct' ranking methodology which supersedes all others. It is to the benefit of the Marketing discipline that alternative perspectives are considered and validated (Steward and Lewis, 2010).

The study contributes to the ongoing discourse on business research quality assessment and journal ranking by examining marketing academic publication ranking across a representative academic spectrum (e.g., from lecturers to professors), not solely based on leading business schools or the academic elite (Theoharakis and Hirst, 2002; Van Fleet et al., 2000). In addition the study considers the relative impact of a number of journal ranking criteria, the journals in which academics actually publish, and differences between regions. The vox populi approach, considering views from a wide range of academics from around the world provides a more balanced view of journal quality. Finally, the study compares the results against two other generally accepted impact factor based journal ranking lists.

Methodology

Academics were asked to freely recall and nominate up to ten journals. This unaided "top-of-the-mind" method is used extensively in measuring advertisement effectiveness and brand recall as it accesses schema and long term memory without cues (Bagozzi and Silk, 1983; Finn, 1992; Krugman, 1986; Shapiro and Krishnan, 2001; Stapel, 1998; Till and Baack, 2005). In the current case, this method was used to assess academics awareness of 'A" grade journals.

Sampling

To implement the vox populi sampling approach, this study developed a sampling frame of academics by scanning marketing, tourism, and international business departmental web sites of universities across the five continents. In total, 8,355 potential respondents were identified from Shanghai Jiao Tong University Ranking 2010 List, Times Supplement University Ranking 2010 List (excluding academics not listed on the Shanghai Jiao Tong List), ANZMAC Conference Directory of Academics, and a further 300 universities located in South Africa, Asia/South Pacific, Australia, New Zealand, USA, Middle East, South America and Europe that were not present on any of the above lists. Respondents included academics from all levels (lecturers, senior lecturers, assistant professors, associate professors, professors, and chair professors). Two weeks following the initial email invitation, a follow-up email was sent.

Survey Instrument

The survey instrument consisted of three sections. The first section asked respondents to recall up to ten A-grade journals, and to indicate which of these journals respondents had published in since 2006. In addition the respondents were asked to list any other journals they had published in. The second section addressed career, life-balance, family and health considerations with respect to publishing, while the third section collected demographic information.

Result

Of the 8,355 potential respondents contacted by email, 825 returned out-of office auto-generated messages, 982 had "undeliverable" e-mails (e.g., invalid e-mail addresses), 87 others declined to participate and 4,641 provided no response to either email. In total 1,820 usable responses were collected giving an overall response rate of 21.8%. Following data cleaning and verification, the final sample size was 1,005 cases (12.0%). This number of respondents was a significant (186%) increase from the previous study (Fam et al., 2011). The respondents were full-time academics from marketing, tourism or international business.

The file was examined visually for appropriate journal names, standard journal names (from the journal site) were adopted and the data reviewed and journal names changed to reflect the standard names. Formulas were created to sort journals and the different ranking methods were calculated.

The cleaned and verified data was then input into SPSS, where CFA and EFA were completed for each of the constructs.

Survey data Analysis

Table 1 indicates that the single highest numbers of respondents were mid-career, aged 35 to 44 (22%), while 10% (101) were in their early career. The majority of the respondents 34.4% (aged 45 to 64) were well established in their academic career. There were 471 males (46.9%) and 210 females (20.9%), 324 respondents did not report their gender.

The majority of respondents are from the United States of America (31.7%), followed by the United Kingdom (9.4%) Australia (7.9%), Canada (4.3%) and New Zealand (3.7%), see Table 2. This compares favourably with the previous survey, although 285 (28.3%) respondents did not provide their country of residence, with 222 (41.3%) respondents from the United States and Canada, 142 (26.3%) from UK and Europe, 114 (21.2%) from Australia and New Zealand, and 60 (11.2%) from Asia and Africa.

Journal Ranking

The six A* journals included Journal of Marketing with the highest number of unaided recalls, 714 (9.9%), followed by Journal of Marketing Research with 659 (9.2%) unaided recalls, Journal of Consumer Research with 598 (8.3%), Marketing Science with 470 (6.5%), Journal of Academy of Marketing Science with 389 (5.4%), and the Journal of Retailing with 282 (3.9%).

Overall, the academics who responded indicated that they had published 1,444 articles in 501 journals. Of these articles, 688 (47.6%) were published in the top ten journals with 93 (6.4%) papers published in the Journal of Marketing, 82 (5.7%) papers in the Journal of Marketing Research, 86 (6.0%) papers in the Journal of Consumer Research, 45 (3.1%) papers in Marketing Science, 74 (5.1%) papers in the Journal of Academy of Marketing Science and 37 (2.6%) papers in the Journal of Retailing. A total of 417 papers were published in the six top-ranked journals, see Table 3.

Regionally, US academics reported the highest number of publications in the Journal of Marketing (43, Canada with 6), and the highest number of A* publications overall with 202, (Canada had 27 A* publications).

Table 4 shows that although the top ten rankings are similar, there are some significant regional differences. Outside North America international and European related journals are ranked higher, especially by the UK respondents. The Journal of Service Research is ranked highest by Australia and New Zealand, which may represent an academic focus on service in the area, while the UK respondents rank European Journal of Marketing, Journal of Marketing Management, and Industrial Marketing Management altogether higher than most other regions. Acta Commercii is a an obvious outlier, it is a South African management journal ranked within the top ten by those respondents designated in the rest of the world, due to the small number of respondents (15) overall.

To explore whether the large number of US respondents biased the journal rankings, the data was analysed without the US data (n=400), see Appendix D for details of the top 100 journals. There were no significant differences in the top six rankings without the US data. However, it is evident that the non-US data shows a more international flavour, with European Journal of Marketing and International Journal of Research in Marketing moving to 7th and 8th place and an overall increase in the number of international journals below the top 30, including more eclectic journals. The US and non-US data appear similar through the top 29 rankings; however the US data includes more classically (quantitative and US) focused journals.

Ranking criteria

In addition to the journal ranking exercise, each respondent was also asked to indicate the relative importance (weighting) of seven factors in ranking journals, see Table 5. In order to ensure relative weighting was considered, each respondent provided a weighting for each of the seven factors which had to total to 100%.

The results showed that overall impact factor provided the greatest impact on ranking, with a weighting of 24.6, but was not the only criteria considered important. The use of perceptive studies to rank the journals and contribution to knowledge were also major contributors to journal ranking.

MAG Score

MAG Scholar is the abbreviation for the Marketing in Asia Group (www.magscholar.com) which initiated and supports the ongoing study. The study designed a formula to capture the relative standing of these journals. Since each journal has its own features and merits, the first unaided recalled journal was allocated more weight than the second, third, fourth, and so on, until the tenth position. The sum of each journal's value was labelled the MAG score, and this score was used to rank the journal relative to the others, see Appendix B for example.

MAG score = [[summation].sub.j](([R.sub.ij]/[summation][T.sub.j]/[Rank.sub.j])

Note: [R.sub.ij] is the number of unaided recalls for the ith journal (i = 1 - 632) with jth rank (j = 1,2, 3, ..., 10), and T is the total number of unaided recalls for all journals with rank j. [Rank.sub.j] represents the rank of the journal.

In addition, a MAG index was computed where a value of "100" was attributed to the number 1 journal. The indices for the remaining journals were calculated based on their respective MAG scores relative to the number 1 journal. Table 6 contains the top 50 MAG journals based on the number of journal recalls; the MAG scores combined with the MAG index was used to guide the rank separations. For comparative purposes, the table also shows the 2009 MAG journal ranking. The complete list of journal rankings can be found at www.magscholar.com.

Validation

To further validate the MAG score, correlation analysis of the top 100 MAG score journals was undertaken against the SSCI and Scopus indices (Fam et al., 2011; Steward and Lewis, 2010), see Appendix C. In total 89 of the top 100 MAG Scholar journals are included in the SCOPUS index, while 66 are included in the SSCI database. As shown in Table 7, the correlation between the MAG scholar ranked journals and the Scopus, r(87) = .40,p < .01, and SSCI, r(64) = .34,p < .05, rankings are positive and significant. This indicates that the MAG Scholar list is broadly similar to both the Scopus and SSCI indices.

Conclusion

In an educational environment that considers performance on the basis of objective measures, publishing in highly ranked journals are an important indicator of perceived quality. The key questions are not; are the rankings valid or without bias?, but "are there other valid measures of journals rankings?" and "do the rankings indicate quality?" (Lee, 2011). This study used the vox populi approach in order to introduce an element of 'crowd sourcing' from which to enhance and augment the spectrum of academically ranked journals in the marketing domain (Mannes, 2009). As Steward and Lewis (2010) suggest, Marketing journals are ultimately appraised by members of the Marketing research community, and investigating a variety of validated ranking measures will only "... strengthen the scientific integrity of the Marketing discipline ..."

The top six journals, regardless of region, are: Journal of Marketing, Journal of Marketing Research, Journal of Consumer Research, Marketing Science, Journal of Academy of Marketing Science, and the Journal of Retailing. These results are similar to other studies identified in the literature (Steward and Lewis, 2010). The contribution from this study is the use of a wide spectrum of academics across multiple countries to gather the ranking data; thus the ranking provides an international perspective by design. In addition, the high correlation results with Scopus and SSCI provides evidence that the MAG rankings are reliable when compared against other ranking indices (Steward and Lewis).

The results also demonstrate that the six top ranked journals remain the most highly ranked even when the US data is removed. This implies a uniform recognition of quality and a consistency of quality perception internationally. More interestingly is the divergence of journal rankings, below the top six. The data set without US data indicates an emphasis on international marketing journals, while the data set including US data rank general management journals higher.

Although there are strong correlations between the MAG method, Scopus and SSCI, there are large differences in philosophy and in statistical variance. Both Scopus and SSCI are actuarial counting of citation rates, and are based on the logical inference that the more often a journal's articles are cited, the more prestigious that journal is. In contrast, the vox populi method used here may have captured the broader range of the uses to which published articles are used - such as teaching, consultancy and contributions to public issues. A case could be made that a focus on research citations too narrowly represents the usefulness of research. A sole focus on high citation rates risks becoming a self-fulfilling prophecy, where publication in top journals is pursued as an outcome in itself rather than as a marker for practical usefulness or genuine contribution to knowledge. As universities are often subject to publish or perish cultures, and are also sometimes accused of failing to do research that is relevant, a broader means of valuing journals might help nudge university life in positive directions.

The current study showed that impact factors are critical, but account for less than 25% of a journal's perceived ranking. The other six factors vary widely in the extent to which they are valued, and probably still only account for some of the factors (such as its topics) that make a journal useful or not. The diversity of ways that a journal can be useful, and the diversity of academics, suggest that a MAG scholar approach fits Surowiecki's (2004) four important elements for outlining a wise crowd: (1) diversity of opinion, (2) independence, (identified though the range of tertiary institutions and anonymous design) (3) decentralization, and (4) aggregation.

The publication results support the findings of Saunders and Wong (2011), that UK academics (and to a lesser extent UK influenced academics such as found in Australia and New Zealand) are falling behind in A* journal publications compared to the USA, Asia and Europe. As a percentage of respondents UK academics have a 6.4% publication rate in A* journals, while Australia and New Zealand have a15.5% publication rate, Asia has a 40.0% rate and Europe has a 45.1% publication rate. Given that the top six journals remain relatively stable across regions and over time this should be a concern to UK, Australian and New Zealand researchers and institutions.

Future research should investigate regional differences more fully, including examining the influence of career duration, age and family-life balance. The determination of journal quality, outside of citation and impact continues to come under scrutiny (Lee, 2011), it is therefore important that additional research explore quality criteria from both a wider academic perspective, and a business impact perspective.

Limitations

The sample consisted of 1,005 self-selected respondents, which may not be representative of all academics, however the sample has increased significantly from the 2009 study, where 538 academics responded. Surveys in which respondents are self-selected will contain an element of bias, especially when the data collected have potential personal and career impact; the results from this survey are no different. However, utilising the vox populi approach and continuing to conduct the survey every three years should build a substantial longitudinal data set.

One concern commonly voiced is that importance of emerging journals and attempting to rank them (Fam et al., 2011). A second concern is finding an acceptable method to capture the dynamic nature of journal rankings, as marketing priorities and activities move forward (Steward and Lewis, 2010).

Appendix A: List of countries included in each region

North America (NA, 362) includes USA (319) and Canada (43).

UK (94).

ANZ (116) includes Australia (79) and New Zealand (37).

Asia (50) includes Bangladesh (1), China (9), Hong Kong (15), India (5), Indonesia (1), Japan (2), Malaysia (9), Republic of Korea (1), Singapore (2), Thailand (3), and Vietnam (2).

Europe (82) includes Austria (9), Belgium (1), Denmark (5), Finland (3), France (14), Germany (3), Greece (2), Hungary (1), Ireland (4), Netherlands (14), Norway (9), Poland (1), Portugal (2), Spain (1), Sweden (10), Switzerland (1), and Turkey (2).

Rest of the world (ROW, 15) includes Brazil (1), Egypt (1), Israel (2), Saudi Arabia (2), South Africa (7), United Republic of Tanzania (1), and the United Arab Emirates (1).

Appendix B: Example of the MAG score calculation

Journal MAG score = [[summation].sub.j](([R.sub.ij]/[summation][T.sub.j]))/[Rank.sub.j])

Note: = [R.sub.ij] is the number of unaided recalls for the ith journal (i = 1 - 632) with jth rank (j = 1, 2, 3, ..., 10), and T is the total number of unaided recalls for all journals with rank j. [Rank.sub.j] represents the rank of the journal.

Example: MAG score for Journal of Marketing

(426/993/1) + (114/989/2) + (75/983/3) + ... (1/236/10) = 0.536
Appendix C: MAG scholar journal ranking compared to Scopus and SSCI

MAG        Journal                               Scopus   Scopus
                                                 IF       Rank

Rk   Cat

1    A *   Journal of Marketing                  4.71     13
2    A *   Journal of Marketing Research         3.60     15
3    A *   Journal of Consumer Research          2.68     7
4    A *   Marketing Science                     2.03     61
5    A *   Journal of Academy of Marketing       2.48     74
             Science
6    A *   Journal of Retailing                  5.06     45
7    A     Academy of Management Journal         6.02     11
8    A     Journal of International              3.91     48
             Business Studies
9    A     Journal of Consumer Psychology        4.06     Z
10   A     Journal of Business Research          1.51     128
11   A     Management Science                    2.52     20
12   A     Strategic Management Journal          4.20     30
13   A     European Journal of Marketing         1.29     180
14   A     Academy of Management Review          7.90     4
15   A     International Journal of              2.21     56
             Research in Marketing
16   A     Journal of Advertising                1.36     121
17   A     Annals of Tourism Research            1.40     134
18   A     Journal of Service Research           2.92     57
19   A     Administrative Science Quarterly      3.51     2
20   A     Industrial Marketing Management       1.88     136
21   A     Journal of Advertising Research       0.70     221
22   A     Tourism Management                    2.53     90
23   A     Journal of Public Policy and          1.85     47
             Marketing
24   A     Journal of Marketing Management       N/A      X
25   B     Marketing Letters                     0.91     95
26   B     Journal of International Marketing    2.18     190
27   B     Harvard Business Review               1.70     33
28   B     Psychology & Marketing                N/A      X
29   B     Journal of Travel Research            1.73     Z
30   B     Journal of Product Innovation         2.09     73
             Management
31   B     Journal of Business Ethics            1.18     245
32   B     Organization Science                  3.32     46
33   B     Entrepreneurship, Theory and          N/A      X
             Practice
34   B     Journal of Personality and            4.82     Z
             Social Psychology
35   B     Journal of Business Venturing         2.92     52
36   B     Journal of Operations Management      4.60     24
37   B     Journal of Business Logistics         N/A      X
38   B     Journal of Applied Psychology         4.33     Z
39   B     Journal of Consumer Behavior          N/A      X
40   B     Journal of Personal Selling and       1.54     182
             Sales Management
41   B     Journal of Macromarketing             1.20     214
42   B     Journal of Management                 4.87     22
43   B     Journal of Management Studies         3.13     39
44   B     Organization Studies                  2.47     70
45   B     American Economic Review              2.65     Z
46   B     Journal of World Business             2.79     137
47   B     Journal of Services Marketing         1.02     218
48   B     Journal of Sustainable Tourism        1.08     162
49   B     Quantitative Marketing and            1.19     119
             Economics
50   B     Research Policy                       N/A      X
51   B     Management International Review       1.83     224
52   B     Psychological Science                 4.87     Z
53   B     Consumption, Markets & Culture        N/A      X
54   B     MIS Quarterly                         5.65     12
55   B     Organizational Behavior and Human     2.22     6
             Decision Processes
56   B     Journal of Interactive Marketing      3.91     69
57   B     International Marketing Review        1.46     194
58   B     Journal of Finance                    4.07     5
59   B     International Journal of              1.27     166
             Market Research
60   B     Marketing Theory                      0.00     672
61   B     British Journal of Management         1.53     110
62   B     Journal of Supply Chain Management    2.35     113
63   B     Environment and Planning              2.04     Z
64   B     International Business Review         1.48     230
65   B     Journal of Marketing Theory           1.31     248
             and Practice
66   B     Transportation Research               2.28     79
67   B     Journal of Business and               1.03     309
             Industrial Marketing
68   B     Decision Sciences                     2.81     54
69   B     Journal of Travel and Tourism         0.64     393
             Marketing
70   B     Sloan Management Review               1.24     122
71   B     Journal of Consumer Affairs           2.06     72
72   B     International Journal of Physical     N/A      X
             Distribution & Logistics
             Management
73   B     Organization                          1.47     160
74   B     Tourism Analysis                      N/A      X
75   B     Journal of International              2.08     172
             Management
76   B     Journal of Small Business             1.38     215
             Management
77   B     Human Resource Management Journal     1.35     124
78   B     Operations Research                   1.70     Z
79   B     Information Systems Research          2.21     Z
80   B     Journal of Marketing Education        0.72     238
81   B     Public Opinion Quarterly              1.48     Z
82   B     California Management Review          2.60     71
83   B     Journal of Political Economy          4.00     Z
84   B       International Review of Retail
             Distribution and Consumer           N/A      X
             Research
85   B     Industrial and Corporate Change       1.92     Z
86   B     Journal of Experimental Social        2.24     Z
             Psychology
87   B     Quarterly Journal of Economics        5.78     Z
88   B     Current Issues in Tourism             0.70     340
89   B     Business History Review               0.36     516
90   B     Science                               24.19    Z
91   B     International Journal of              1.40     159
             Advertising
92   B     Journal of Consumer Culture           1.46     Z
93   B     IEEE Transactions on Engineering      1.90     91
             Management
94   B     American Journal of Sociology         3.49     Z
95   B     Small Business Economics              1.44     Z
96   B     Journal of Experimental Psychology    4.67     Z
97   B     Accounting, Organizations and         2.58     105
             Society
98   B     Journal of Financial Economics        4.43     10
99   B     European Journal of Operational       2.51     Z

             Research
100  B     Human Relations                       1.84     84

MAG         Journal                               SSCI   SSCI   MAG
                                                  IF     Rank   Score

Rk    Cat

1     A *   Journal of Marketing                  3.78   12     0.536
2     A *   Journal of Marketing Research         3.1    17     0.330
3     A *   Journal of Consumer Research          3.02   18     0.272
4     A *   Marketing Science                     2.19   35     0.161
5     A *   Journal of Academy of Marketing       1.58   61     0.109
              Science
6     A *   Journal of Retailing                  4.57   4      0.080
7     A     Academy of Management Journal         6.48   2      0.067
8     A     Journal of International              3.77   13     0.059
              Business Studies
9     A     Journal of Consumer Psychology        5.35   3      0.057
10    A     Journal of Business Research          1.29   74     0.051
11    A     Management Science                    2.23   33     0.050
12    A     Strategic Management Journal          4.46   6      0.049
13    A     European Journal of Marketing         0.76   115    0.045
14    A     Academy of Management Review          7.87   1      0.044
15    A     International Journal of              1.87   48     0.040
              Research in Marketing
16    A     Journal of Advertising                1.17   83     0.037
17    A     Annals of Tourism Research            1.17   Z      0.027
18    A     Journal of Service Research           1.67   53     0.026
19    A     Administrative Science Quarterly      3.84   11     0.025
20    A     Industrial Marketing Management       1.33   73     0.023
21    A     Journal of Advertising Research       0.8    111    0.022
22    A     Tourism Management                    1.88   46     0.021
23    A     Journal of Public Policy and          N/A    X      0.020
              Marketing
24    A     Journal of Marketing Management       N/A    X      0.019
25    B     Marketing Letters                     0.56   130    0.017
26    B     Journal of International Marketing    1.59   59     0.016
27    B     Harvard Business Review               1.66   55     0.016
28    B     Psychology & Marketing                1.34   72     0.016
29    B     Journal of Travel Research            N/A    X      0.015
30    B     Journal of Product Innovation         1.52   63     0.015
              Management
31    B     Journal of Business Ethics            1.09   89     0.015
32    B     Organization Science                  3.13   15     0.014
33    B     Entrepreneurship, Theory and          N/A    X      0.014
              Practice
34    B     Journal of Personality and            4.73   Z      0.013
              Social Psychology
35    B     Journal of Business Venturing         2.26   31     0.013
36    B     Journal of Operations Management      3.24   14     0.012
37    B     Journal of Business Logistics         N/A    X      0.012
38    B     Journal of Applied Psychology         3.84   Z      0.011
39    B     Journal of Consumer Behavior          N/A    X      0.010
40    B     Journal of Personal Selling and       N/A    X      0.010
              Sales Management
41    B     Journal of Macromarketing             N/A    X      0.010
42    B     Journal of Management                 4.43   7      0.010
43    B     Journal of Management Studies         2.81   19     0.010
44    B     Organization Studies                  2.12   38     0.009
45    B     American Economic Review              2.53   Z      0.009
46    B     Journal of World Business             2.63   20     0.008
47    B     Journal of Services Marketing         N/A    X      0.008
48    B     Journal of Sustainable Tourism        N/A    X      0.008
49    B     Quantitative Marketing and            N/A    X      0.008
              Economics
50    B     Research Policy                       2.26   30     0.007
51    B     Management International Review       N/A    X      0.007
52    B     Psychological Science                 5.09   Z      0.006
53    B     Consumption, Markets & Culture        N/A    X      0.006
54    B     MIS Quarterly                         4.49   5      0.006
55    B     Organizational Behavior and Human     2.55   23     0.006
              Decision Processes
56    B     Journal of Interactive Marketing      2.6    21     0.006
57    B     International Marketing Review        1.16   84     0.006
58    B     Journal of Finance                    3.76   Z      0.006
59    B     International Journal of              0.99   96     0.006
              Market Research
60    B     Marketing Theory                      N/A    X      0.006
61    B     British Journal of Management         1.45   66     0.005
62    B     Journal of Supply Chain Management    N/A    X      0.005
63    B     Environment and Planning              N/A    X      0.005
64    B     International Business Review         1.06   92     0.005
65    B     Journal of Marketing Theory           N/A    X      0.004
              and Practice
66    B     Transportation Research               N/A    X      0.004
67    B     Journal of Business and               N/A    X      0.004
              Industrial Marketing
68    B     Decision Sciences                     2.38   26     0.004
69    B     Journal of Travel and Tourism         N/A    X      0.004
              Marketing
70    B     Sloan Management Review               N/A    X      0.004
71    B     Journal of Consumer Affairs           2.18   36     0.004
72    B     International Journal of Physical     N/A    X      0.004
              Distribution & Logistics
              Management
73    B     Organization                          1.35   70     0.003
74    B     Tourism Analysis                      N/A    X      0.003
75    B     Journal of International              1.85   49     0.003
              Management
76    B     Journal of Small Business             1.09   89     0.003
              Management
77    B     Human Resource Management Journal     N/A    X      0.003
78    B     Operations Research                   1.58   62     0.003
79    B     Information Systems Research          1.79   50     0.003
80    B     Journal of Marketing Education        N/A    X      0.003
81    B     Public Opinion Quarterly              N/A    X      0.003
82    B     California Management Review          1.98   44     0.003
83    B     Journal of Political Economy          3.84   Z      0.003
84    B       International Review of Retail
              Distribution and Consumer           N/A    X      0.003
              Research
85    B     Industrial and Corporate Change       1.51   64     0.003
86    B     Journal of Experimental Social        2.24   Z      0.003
              Psychology
87    B     Quarterly Journal of Economics        5.65   Z      0.003
88    B     Current Issues in Tourism             N/A    X      0.003
89    B     Business History Review               0.35   144    0.003
90    B     Science                               N/A    X      0.002
91    B     International Journal of              1.09   88     0.002
              Advertising
92    B     Journal of Consumer Culture           N/A    X      0.002
93    B     IEEE Transactions on Engineering      1.25   75     0.002
              Management
94    B     American Journal of Sociology         3.48   Z      0.002
95    B     Small Business Economics              1.38   69     0.002
96    B     Journal of Experimental Psychology    N/A    X      0.002
97    B     Accounting, Organizations and         N/A    X      0.002
              Society
98    B     Journal of Financial Economics        4.02   Z      0.002
99    B     European Journal of Operational       N/A    X      0.002
              Research

100   B     Human Relations                       1.64   56     0.002

Notes: SSCI: Business, Management; Scopus: Business, Management and
Accounting. Z: Not in Business, Management or Business, Management
and Accounting category X: Not in SSCI or Scopus database

Appendix D: MAG scholar journal ranking comparing with and
without US data

      Journal with US data                                    Total

1     Journal of Marketing                                    714
2     Journal of Marketing Research                           659
3     Journal of Consumer Research                            598
4     Marketing Science                                       470
5     Journal of Academy of Marketing Science                 389
6     Journal of Retailing                                    282
7     Journal of Consumer Psychology                          184
8     Journal of Business Research                            166
9     Management Science                                      148
10    International Journal of Research in Marketing          148
11    Academy of Management Journal                           139
12    Journal of International Business Studies               137
13    European Journal of Marketing                           130
14    Strategic Management Journal                            119
15    Journal of Advertising                                  118
16    Academy of Management Review                            99
17    Journal of Service Research                             79
18    Industrial Marketing Management                         71
19    Administrative Science Quarterly                        67
20    Journal of Advertising Research                         66
21    Journal of Public Policy and Marketing                  63
22    Journal of Marketing Management                         58
23    Journal of International Marketing                      56
24    Marketing Letters                                       56
25    Annals of Tourism Research                              49
26    Psychology & Marketing                                  48
27    Harvard Business Review                                 47
28    Organization Science                                    46
29    Tourism Management                                      41
30    Journal of Product Innovation Management                40
31    Journal of Personality and Social Psychology            40
32    Journal of Business Ethics                              39
33    Journal of Management                                   35
34    Journal of Consumer Behavior                            34
35    Journal of Travel Research                              31
36    Entrepreneurship, Theory and Practice                   30
37    Journal of Personal Selling and Sales Management        30
38    Journal of Management Studies                           30
39    Journal of Business Venturing                           29
40    Journal of Applied Psychology                           29
41    Quantitative Marketing and Economics                    29
42    Journal of Macromarketing                               28
43    Journal of Services Marketing                           26
44    Journal of World Business                               24
45    Organization Studies                                    22
46    Organizational Behavior and Human Decision Processes    22
47    Journal of Operations Management                        21
48    Journal of Sustainable Tourism                          21
49    Psychological Science                                   21
50    International Marketing Review                          21
51    Research Policy                                         19
52    Management International Review                         19
53    Journal of Interactive Marketing                        19
54    Journal of Business Logistics                           17
55    Marketing Theory                                        17
56    British Journal of Management                           17
57    Consumption, Markets & Culture                          16
58    International Business Review                           16
59    American Economic Review                                15
60    International Journal of Market Research                15
61    Environment and Planning                                15
62    Sloan Management Review                                 15
63    MIS Quarterly                                           13
64    Decision Sciences                                       13
65    Journal of Supply Chain Management                      12
66    Journal of Marketing Theory and Practice                12
67    Journal of Business and Industrial Marketing            12
68    Journal of Travel and Tourism Marketing                 12
69    Tourism Analysis                                        12
70    Journal of International Management                     12

71    Journal of Small Business Management                    10
72    Journal of Marketing Education                          10
73    California Management Review                            9
74    Journal of Political Economy                            9
75    Industrial and Corporate Change                         9
76    Journal of Experimental Social Psychology               9
77    Journal of Finance                                      8
78    International Journal of Physical Distribution          8
        & Logistics Management
79    Operations Research                                     8
80    Current Issues in Tourism                               8
81    Quarterly Journal of Economics                          8
82    Journal of Consumer Culture                             8
83    Small Business Economics                                8
84    Science                                                 8
85    International Journal of Hospitality Management         8
86    Journal of Experimental Psychology                      8
87    Human Relations                                         8
88    Journal of Organizational Behavior                      8
89    Journal of Consumer Affairs                             7
90    Human Resource Management Journal                       7

91    Information Systems Research                            7
92    International Journal of Advertising                    7

93    International Journal of Logistics Management           7
94    Tourism Geographies                                     7
95    Transportation Research                                 6
96    Journal of Marketing Communications                     6
97    Accounting, Organizations and Society                   6
98    Journal of Consumer Marketing                           6
99    Journal of Strategic Marketing                          6
100   International Journal of Human Resource Management      6

      Journal without US data                                Total

1     Journal of Marketing                                   265
2     Journal of Marketing Research                          232
3     Journal of Consumer Research                           208
4     Marketing Science                                      160
5     Journal of Academy of Marketing Science                155
6     Journal of Retailing                                   115
7     European Journal of Marketing                          91
8     International Journal of Research in Marketing         91
9     Journal of Business Research                           79
10    Journal of International Business Studies              64
11    Academy of Management Journal                          62
12    Journal of Consumer Psychology                         52
13    Industrial Marketing Management                        45
14    Academy of Management Review                           43
15    Journal of Marketing Management                        42
16    Journal of Service Research                            41
17    Management Science                                     37
18    Journal of Advertising                                 36
19    Annals of Tourism Research                             35
20    Strategic Management Journal                           33
21    Administrative Science Quarterly                       32
22    Journal of International Marketing                     27
23    Tourism Management                                     26
24    Organization Science                                   25
25    Marketing Letters                                      24
26    Journal of Product Innovation Management               23
27    Psychology & Marketing                                 22
28    Journal of Advertising Research                        21
29    Journal of Business Ethics                             18
30    Journal of Management Studies                          17
31    Journal of Travel Research                             17
32    Journal of Consumer Behavior                           15
33    Journal of Sustainable Tourism                         15
34    Organization Studies                                   15
35    Harvard Business Review                                14
36    International Marketing Review                         13
37    Journal of Business Venturing                          13
38    Journal of Management                                  13
39    Journal of World Business                              13
40    Entrepreneurship, Theory and Practice                  12
41    Journal of Public Policy and Marketing                 12
42    British Journal of Management                          11
43    Environment and Planning                               11
44    Journal of Macromarketing                              11
45    Marketing Theory                                       11
46    Consumption, Markets & Culture                         10
47    Journal of Applied Psychology                          10
48    International Business Review                          9
49    Journal of Interactive Marketing                       9
50    International Journal of Market Research               8
51    Journal of Consumer Marketing                          8
52    Advances in Consumer Research                          5
53    Current Issues in Tourism                              5
54    Industrial and Corporate Change                        5
55    Journal of Advertising and Marketing Communications    5
56    Journal of Business and Industrial Marketing           5
57    American Economic Review                               4
58    ASQ                                                    4
59    California Management Review                           4
60    Human Relations                                        4
61    International Journal of Hospitality Management        4
62    Accounting, Organizations and Society                  3
63    Australian Journal of Management                       3
64    Economic Journal                                       3
65    European Journal of Operational Research               3
66    Information Systems Research                           3
67    International Journal of Advertising                   3
68    International Journal of Human Resource Management     3
69    International Journal of Tourism Research              3
70    International Review of Retail Distribution and        3
        Consumer Research
71    International Small Business Journal                   3
72    Journal of Consumer Culture                            3
73    Academy of Marketing Science Quarterly                 2
74    Acta Commercii                                         2
75    Asia Pacific Journal of Marketing                      2
76    Asia Pacific Journal of Marketing and Logistics        2
77    Asia Pacific Journal of Tourism Research               2
78    Asian Journal of Business Research                     2

79    Asian Survey                                           2
80    Business Ethics Quarterly                              2
81    Business Horizons                                      2
82    Decision Sciences                                      2
83    Department of Management Studies                       2
84    Entrepreneurship and Regional Development              2
85    Environmental Politics                                 2
86    European Economic Review                               2
87    European Management Journal                            2
88    Human Resource Management Journal                      2
89    International Journal of Bank Marketing                2
90    International Journal of Contemporary                  2
        Hospitality Management
91    International Journal of Cross Cultural Management     2
92    International Journal of Retail &                      2
        Distribution Management
93    Journal of Brand Management                            2
94    Journal of Business                                    2
95    Journal of Business Accounting and Finance             2
96    Journal of Business Logistics                          2
97    Journal of Business to Business Marketing              2
98    Journal of Common Market Studies                       2
99    Journal of Contemporary Management                     2
100   Journal of Economic Geography                          2

Note: The highlighted listings indicate journals either ranked
outside the top 50, or not ranked within the 100 journal listed,
(e.g., Journal of Personality and Social Psychology is ranked #31
with US data included, but is not ranked in the top 100 when US
data is not included).


Acknowledgement:

The authors would like to thank the National Natural Science Foundation of China (70902011) and Victoria University of Wellington FCA Small Grant (114945) for financial support.

References

Bagozzi, R. P. & Silk, A. J. (1983), "Recall, recognition, and the measurement of memory for print advertisement". Marketing Science, vol. 2, no. 2, pp. 95.

Fam, K.-S., Shukla, P., Sinha, A., Parackel, M. & Chai, J. C. Y. (2011), "Rankings in the Eyes of the Beholder: A Vox Populi Approach to Academic Journal Ranking". Asian Journal of Business Research, vol. 1, no. 1, pp. 1-17.

Finn, A. (1992), "Recall, Recognition and the Measurement of Memory for Print Advertisements: A Reassessment". Marketing Science, vol. 11, no. 1, pp. 95-95.

Galton, F. (1907), "Vox populi". Nature, vol. 75, no. 450-451.

Hastie, R. & Kameda, T. (2005), "The Robust Beauty of Majority Rules in Group Decisions". Psychological Review, vol. 112, no. 2, pp. 494-508.

Krugman, H. E. (1986), "Low Recall and High Recognition of Advertising". Journal of Advertising Research, vol. 26, no. 1, pp. 79-79.

Lee, N. (2011), "Reflections on assessing academic quality in marketing, and the UK REF". European Journal of Marketing, vol. 45, no. 4, pp. 477-483.

Mannes, A. E. (2009), "Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision". Management Science, vol. 55, no. 8, pp. 1267-1279.

Saunders, J. & Wong, V. (2011), "Manoeuvring towards research decline". European Journal of Marketing, vol. 45, no. 4, pp. 484-512.

Shapiro, S. & Krishnan, H. S. (2001), "Memory-based measures for assessing advertising effects: A comparison of explicit and implicit memory effects". Journal of Advertising, vol. 30, no. 3, pp. 1-13.

Soil, J. B. & Larrick, R. P. (2009), "Strategies for revising judgment: How (and how well) people use others' opinions". Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 35, no. 3, pp. 780-805.

Stapel, J. (1998), "Recall and Recognition: A Very Close Relationship". Journal of Advertising Research, vol. 38, no. 4, pp. 41-45.

Steward, M. D. & Lewis, B. R. (2010), "A Comprehensive Analysis of Marketing Journal Rankings". Journal of Marketing Education, vol. 32, no. 1, pp. 75-92.

Surowiecki, J. (2004), The Wisdom of Crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations, Doubleday, New York, NY.

Theoharakis, V. & Hirst, A. (2002), "Perceptual differences of marketing journals: A worldwide perspective". Marketing Letters, vol. 13, no. 4, pp. 389-402.

Till, B. D. & Baack, D. W. (2005), "Recall and persuasion". Journal of Advertising, vol. 34, no. 3, pp. 47-57.

Van Fleet, D. D., McWilliams, A. & Siegel, D. S. (2000), "A Theoretical and Empirical Analysis of Journal Rankings: The Case of Formal Lists". Journal of Management, vol. 26, no. 5, pp. 839-861.

James E. Richard

Victoria University of Wellington

Kim-Shyan Fam

Victoria University of Wellington

Geoff Plimmer

Victoria University of Wellington

Stephan Gerschewski

Hankuk University of Foreign Studies
Table 1: Age demographics

Age          Frequency   Percent

25 to 34     101         10.0
35 to 44     221         22.0
45 to 54     173         17.2
55 to 64     173         17.2
65 to 70     35          3.5
71 or over   16          1.6
Missing      286         28.5
             1005        100

Table 2: Respondents by country

Country                       Frequency   Percent

United States of America      319         31.7
United Kingdom                94          9.4
Australia                     79          7.9
Canada                        43          4.3
New Zealand                   37          3.7
Hong Kong                     15          1.5
France                        14          1.4
Netherlands                   14          1.4
Sweden                        10          1.0
Austria                       9           0.9
China                         9           0.9
Malaysia                      9           0.9
Norway                        9           0.9
South Africa                  7           0.7
Denmark                       5           0.5
India                         5           0.5
Ireland                       4           0.4
Finland                       3           0.3
Germany                       3           0.3
Thailand                      3           0.3
Greece                        2           0.2
Israel                        2           0.2
Japan                         2           0.2
Portugal                      2           0.2
Saudi Arabia                  2           0.2
Singapore                     2           0.2
Turkey                        2           0.2
Viet Nam                      2           0.2
Bangladesh                    1           0.1
Belgium                       1           0.1
Brazil                        1           0.1
Egypt                         1           0.1
Hungary                       1           0.1
Indonesia                     1           0.1
Poland                        1           0.1
Republic of Korea             1           0.1
Spain                         1           0.1
Switzerland                   1           0.1
United Arab Emirates          1           0.1
United Republic of Tanzania   1           0.1
Missing                       286         28.5
Total                         1005        100.0

Table 3: A * Publications by region

A * Journals   Overall   NA    UK   ANZ   Asia   Europe   ROW

Journal of       93      49    2     2     3       11      1
  Marketing
Journal of       82      40    1     2     3       8       0
  Marketing
  Research
Journal of       86      55    0     2     3       4       1
  Consumer
  Research
Marketing        45      21    1     4     2       6       0
  Science
Journal of       74      44    1     6     7       4       0
  Academy of
  Marketing
  Science
Journal of       37      20    1     2     2       4       0
  Retailing

Total            417     229   6    18     20      37      2

Note: See Appendix A for the list of countries included
in each region. ROW = Rest of the World.

Table 4: Top ten journals as ranked by regions

                                       Journal Rank by Region

Journal                           NA   ANZ   UK   Eur   Asia   ROW

Journal of Marketing              1     1    1     1     1      1
Journal of Marketing Research     2     2    2     2     1      2
Journal of Consumer Research      3     2    3     3     3      3
Marketing Science                 4     7    6     4     5      3
Journal of Academy of             5     4    4     6     4      5
  Marketing Science
Journal of Retailing              6     8    5     7     6     10
Journal of Consumer Psychology    7    29    14    9     8     14
Journal of Advertising            8    11    26   20     22    14
Management Science                8    31    37    9     14    14
Journal of Business Research      10    6    10   19     10     8
Academy of Management Journal     11   23    12    8     10    14
Journal of International          11   16    10   11     6     14
  Business Studies
International Journal of          15    9    12    5     8      6
  Research in Marketing
Journal of Service Research       17   10    26   14     22    14
Industrial Marketing              26   13    9    26     17     8
  Management
European Journal of Marketing     31    5    7    18     10     6
Journal of Marketing              52   11    8    34     --    14
  Management
Acta Commercii                    --   --    --   --     --    10

Note: the relative position of journals outside of other region's
top ten are shown in small italicised font.

Table 5: Journal ranking criteria as
perceived by respondents

Criteria                        Weight   Rank

Impact factor                   24.64    1
Perceptive studies              18.84    2
Contribution to knowledge       18.48    3
Contribution to career          10.87    4
Rejection rate                  9.59     5
Useful for research students    8.90     6
Esteem factors                  8.68     7

Table 6: Top 50 Marketing journals and MAG scores

Rank                                       2011

2011   2009   Journal                      Total     MAG     MAG
                                           Recalls   Score   Index

1      1      Journal of Marketing         714       0.536   100.00
2      2      Journal of Marketing         659       0.330   61.50
                Research
3      3      Journal of Consumer          598       0.272   50.74
                Research
4      4      Marketing Science            470       0.161   29.94
5      5      Journal of Academy of        389       0.109   20.35
                Marketing Science
6      6      Journal of Retailing         282       0.080   14.88
7      14     Academy of Management        139       0.067   12.47
                Journal
8      7      Journal of International     137       0.059   10.98
                Business Studies
9      16     Journal of Consumer          184       0.057   10.68
                Psychology
10     9      Journal of Business          166       0.051   9.50
                Research
11     11     Management Science           148       0.050   9.25
12     23     Strategic Management         119       0.049   9.11
                Journal
13     8      European Journal of          130       0.045   8.33
                Marketing
14     15     Academy of Management        99        0.044   8.12
                Review
15     10     International Journal of     148       0.040   7.48
                Research in Marketing
16     13     Journal of Advertising       118       0.037   6.83
17     12     Annals of Tourism Research   49        0.027   5.01
18     26     Administrative Science       67        0.026   4.92
                Quarterly
19     27     Journal of Service           79        0.025   4.67
                Research
20     25     Industrial Marketing         71        0.023   4.34
                Management
21     21     Journal of Advertising       66        0.022   4.18
                Research
22     18     Tourism Management           41        0.021   3.98
23     33     Journal of Public Policy     63        0.020   3.65
                and Marketing
24     28     Journal of Marketing         58        0.019   3.56
                Management
25     20     Marketing Letters            56        0.017   3.13
26     30     Journal of International     56        0.016   3.03
                Marketing
27     24     Harvard Business Review      47        0.016   3.00
28     41     Psychology & Marketing       48        0.016   2.98
29     29     Journal of Travel            31        0.015   2.87
                Research
30     36     Journal of Product           40        0.015   2.74
                Innovation Management
31     40     Journal of Business          39        0.015   2.72
                Ethics
32     46     Organization Science         46        0.014   2.61
33     109    Entrepreneurship,            30        0.014   2.51
                Theory and Practice
34     45     Journal of Personality       40        0.013   2.40
                and Social Psychology
35     95     Journal of Business          29        0.013   2.40
                Venturing
36     128    Journal of Operations        21        0.012   2.23
                Management
37     61     Journal of Business          17        0.012   2.14
                Logistics
38     84     Journal of Applied           29        0.011   1.97
                Psychology
39     60     Journal of Consumer          34        0.010   1.87
                Behavior
40     62     Journal of Personal          30        0.010   1.86
                Selling and Sales
                Management
41     101    Journal of                   28        0.010   1.85
                Macromarketing
42     87     Journal of Management        35        0.010   1.79
43     88     Journal of Management        30        0.010   1.77
                Studies
44     113    Organization Studies         22        0.009   1.71
45     77     American Economic            15        0.009   1.61
                Review
46     79     Journal of World             24        0.008   1.56
                Business
47     22     Journal of Services          26        0.008   1.47
                Marketing
48     90     Journal of Sustainable       21        0.008   1.46
                Tourism
49     89     Quantitative Marketing       29        0.008   1.39
                and Economics
50     NR     Research Policy              19        0.007   1.37

Rank                                       2011

2011   2009   Journal                      [DELTA]   Cat

1      1      Journal of Marketing         206.43    A *
2      2      Journal of Marketing         57.72     A *
                Research
3      3      Journal of Consumer          111.51    A *
                Research
4      4      Marketing Science            51.40     A *
5      5      Journal of Academy of        29.35     A *
                Marketing Science
6      6      Journal of Retailing         12.91     A *
7      14     Academy of Management        8.00      A
                Journal
8      7      Journal of International     1.61      A
                Business Studies
9      16     Journal of Consumer          6.35      A
                Psychology
10     9      Journal of Business          1.32      A
                Research
11     11     Management Science           4.93      A
12     23     Strategic Management         4.20      A
                Journal
13     8      European Journal of          1.15      A
                Marketing
14     15     Academy of Management        3.43      A
                Review
15     10     International Journal of     0.22      A
                Research in Marketing
16     13     Journal of Advertising       9.79      A
17     12     Annals of Tourism Research   0.49      A
18     26     Administrative Science       1.34      A
                Quarterly
19     27     Journal of Service           1.74      A
                Research
20     25     Industrial Marketing         0.84      A
                Management
21     21     Journal of Advertising       1.09      A
                Research
22     18     Tourism Management           1.76      A
23     33     Journal of Public Policy     0.49      A
                and Marketing
24     28     Journal of Marketing         2.29      A
                Management
25     20     Marketing Letters            0.57      B
26     30     Journal of International     0.16      B
                Marketing
27     24     Harvard Business Review      0.11      B
28     41     Psychology & Marketing       0.57      B
29     29     Journal of Travel            0.70      B
                Research
30     36     Journal of Product           0.13      B
                Innovation Management
31     40     Journal of Business          0.58      B
                Ethics
32     46     Organization Science         0.53      B
33     109    Entrepreneurship,            0.59      B
                Theory and Practice
34     45     Journal of Personality       0.00      B
                and Social Psychology
35     95     Journal of Business          0.90      B
                Venturing
36     128    Journal of Operations        0.46      B
                Management
37     61     Journal of Business          0.92      B
                Logistics
38     84     Journal of Applied           0.57      B
                Psychology
39     60     Journal of Consumer          0.02      B
                Behavior
40     62     Journal of Personal          0.07      B
                Selling and Sales
                Management
41     101    Journal of                   0.32      B
                Macromarketing
42     87     Journal of Management        0.10      B
43     88     Journal of Management        0.32      B
                Studies
44     113    Organization Studies         0.18      B
45     77     American Economic            0.30      B
                Review
46     79     Journal of World             0.46      B
                Business
47     22     Journal of Services          0.05      B
                Marketing
48     90     Journal of Sustainable       0.38      B
                Tourism
49     89     Quantitative Marketing       0.12      B
                and Economics
50     NR     Research Policy              0.60      B

Note: [DELTA] = difference between JM MAG score multiplied
by 1000. NR = not ranked

Table 7: Correlation Matrix:
MAG, Scopus and SSCI ranking

         MAG      Scopus   SSCI

MAG      1
Scopus   .40 **   1
SSCI     .34 *    .81 **   1

**. Correlation is significant
at the 0.01 level (2-tailed).

*. Correlation is significant
at the 0.05 level (2-tailed).
COPYRIGHT 2012 Asia Business Research Corporation Ltd.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2012 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Richard, James E.; Fam, Kim-Shyan; Plimmer, Geoff; Gerschewski, Stephan
Publication:Asian Journal of Business Research
Geographic Code:1USA
Date:Jan 1, 2012
Words:8394
Previous Article:Occupational hazards in the workplace: a case of an electronic company in Sama Jaya, Kuching, Sarawak, Malaysia.
Next Article:An empirical research on the effect of low-carbon knowledge of the urban residents on their low-carbonized energy consumption behavior.
Topics:

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |