Abstract
Business ethics journals have appeared on a few ranked lists that are specific to this niche discipline. As with more traditional academic disciplines, these rankings are used for academic rewards such as faculty tenure and promotion, along with department and school ratings. Journal ranking has been subject to considerable criticism even as its administrative use persists. Among the criticisms are that journal quality is a poor proxy for article quality, citation rate is an imperfect reflection of article influence, and bias may be introduced into rankings by visibility characteristics such as journal age, size, circulation, and experience of the rater with a journal. This research note studies the effect of journal age and size on the rankings of business ethics journals compiled by Beets, Lewis, and Brower, by Albrecht, Thompson, Hoopes, and Rodrigo, and by Serenko and Bontis. Significant correlation was found for journal age with the administratively derived Beets et al. ranking. No significant correlation was found for size in any ranking study. Results were not significant for the Albrecht et al. and the Serenko and Bontis rankings representing the perspectives from surveys of active researchers or citation analysis. Perhaps sometimes a journal’s reputation precedes it, as perception of journal quality may be biased by journal visibility, either because it has been published and available for a number of years, or because it is well known and likely to be cited.
Quality research articles open up new areas of research and move the profession forward. They resolve problems and improve decision making. Because it is not possible to measure quality directly, scholars and administrators have resorted to journal rankings that are accessible quantitative measures as proxies of article quality. Numerous studies have produced journal rankings. However, it is by no means clear how closely related journal rank is to article quality. Results of journal-ranking studies are sometimes inconsistent. As a result, there have also been studies critical of journal ranking. Age has been reported as one possible bias (Ellis & Durden, 1991; Serenko & Dohan, 2011).
The present research investigates age as well as size, both variables which may cause possible bias, for business ethics journals. Potential bias of size also seems an obvious question for business ethics journals, because of the size dominance of the Journal of Business Ethics. A significant correlation with age was found for the administratively derived rank of Beets, Lewis, and Brower (2016). Relationships between age and journal rank were neither significant for the Albrecht, Thompson, Hoopes, and Rodrigo’s (2010) rank based on surveys of active researchers nor for the Serenko and Bontis’s (2009) rank based on citation analysis. No significant correlations were found between size and journal rank.
Journal-Ranking Literature
There have been studies in various business-related fields whose objective was to develop a list of ranked journals. Most journal-ranking studies have been discipline specific: in accounting, Chow, Haddad, Singh, and Wu (2008) and Howard and Nikolai (1983); in economics, Liebowitz and Palmer (1984); in finance, Borde, Cheney, and Madura (1999) and Coe and Weinstock (1983); in management, Clark and Wright (2007) and Coe and Weinstock (1984); in marketing, Luke and Doke (1987) and Dibb and Simkin (2005); and in artificial intelligence Serenko (2010) and Serenko and Dohan (2011). Some journal-ranking studies have aggregated disciplines: Swanson (2004) and Herron and Hall (2004).
In general, the disciplines follow traditional academic lines; niche fields or new research areas are not well represented. In fact, one of the criticisms of journal-ranking studies is that the lists they produce can be misleading because they are not entirely representative (see, for example, Ellis & Durden, 1991; MacRoberts & MacRoberts, 1989; Ritzberger, 2008). Journals in business ethics, the focus of this research note, a niche and relatively new field, typically do not appear on ranked lists of journals for traditional business disciplines. The lone exception is the Journal of Business Ethics that appears on the Financial Times FT45 list. There have been several studies covering journals in the specific business ethics area. These ranking studies may be of particular interest to scholars of business ethics, but they are off the radar of administrators whose business schools do not have separate departments in business ethics.
Studies that have addressed the niche of business ethics journals researched either citations or opinion surveys. The Serenko and Bontis (2009) and Paul (2004) studies are based on data collected for citation rates. The paucity of impact factor data for these relatively young business ethics journals makes it difficult to create citation-based rankings. The Wicks and Derry (1996) and Albrecht et al. (2010) 1 studies are based on data that reflect reputation based on opinion surveys. The Beets et al. (2016) study is based on lists that were aggregated administratively.
Wicks and Derry (1996) used five criteria for opinion-based rankings of a group of business ethics journals and a selected group of management journals. They surveyed members of the Society for Business Ethics, which sponsors Business Ethics Quarterly. The Wicks and Derry study was one of the earliest attempts to assess how the niche business ethics literature compares with the larger field of management in general.
Albrecht et al. (2010) surveyed a global group of business ethics scholars. Their survey provided a list of 25 business ethics journals and asked respondents to identify their top four (or suggest alternative journal titles); then, respondents were asked to rank their top four. These responses were weighted by number of votes to create a list of the top 10 business ethics journals. Then, they compared the list with a group of mainstream management journals that publish business ethics research, concluding that many business ethics researchers prefer to publish in the niche business ethics journals rather than the more general management journals.
Beets et al. (2016) studied internally developed journal lists that were used by Association to Advance Collegiate Schools of Business (AACSB) business schools to evaluate faculty and selected from those lists 24 business-ethics-centric (BEC) journals. The BEC journals were ranked based on composite scores reflecting both how highly the AACSB schools placed each journal on its internally developed list and on how frequently each journal was included across schools. Beets et al. (2016) thus focused on the practical application of journal ranking; it is heavily influenced by administrative perspective because the list was generated through an accumulation of administrative decisions.
Among those studies that use citation data, Serenko and Bontis (2009) developed a list of 20 business ethics journals and ranked them based on citation impact data collected from Google Scholar. Paul (2004) studied the citation rates among the top three journals in the fields of business and society/business ethics. Included journals were Business & Society, Business Ethics Quarterly, and the Journal of Business Ethics. Cahn (2011) used Google Scholar to develop citation data. Citation-based rankings reflect the perspective of authors of business ethics research because citations are generated by authors’ research.
There is considerable overlap in the lists of business ethics journals produced by the studies described above, but there is no complete agreement on rankings or included journals. The lack of agreement suggests that at best, journal ranking is an inexact science. At worst, it may be creating the appearance of quantification that is, in fact, of minimal value.
Business Ethics Journal Stakeholders
Integral to any field are the publications by way of which internal stakeholders communicate with each other as well as with outside stakeholders. For academic scholars, key publications are refereed journals. There are sets of journals for each well-recognized discipline with varying publication objectives and varying levels of reflected prestige. So important are journals that, following Scott and Lane (2000), the identity of a discipline depends on a set of journals with perceived legitimacy. Scholars read these journals to be current in their field, and they submit manuscripts to these journals to self-identify as members of the discipline, and authors published in these journals have perceived legitimacy. Faculty scholars are evaluated by tenure/promotion committees on whether their work is published and in which journals it is published. Academic institutions are evaluated in part by the perceived legitimacy of their faculty and publications. Outside, stakeholders such as business and government organizations will give more credence to the ideas and advice of those internal stakeholder scholars whose publications are well regarded. Journal editors are among the most influential stakeholders on discipline identity (Sidorova, Evangelopoulos, Valacich, & Ramakrishnan, 2008, p. 469).
As time passes, and the discipline matures, the number of journals in the discipline increases, sometimes even splitting into sub-disciplines. In a way, this expansion is supply of publication outlets balancing demand by manuscripts seeking publication.
Serenko (2013, p. 3) suggested that maturing disciplines are sociometrically studied by both internal and external stakeholders. Sociometric measures for ranking journals are one feature of discipline identity. These ranked lists are then used to evaluate the quality of research and faculty in the field, or when faculty research is aggregated, to evaluate the quality of their schools. In academia, discipline-specific journals are an important aspect of identity. For scholar stakeholders, their legitimacy is reflected in the appraisal of their publications and the recognition of those journals in which they publish. In the business ethics field, discipline and/or scholar legitimacy is reflected in the accord given to business ethics journals and to the scholars whose work appears in business ethics journals.
History of Business Ethics Journals
Business & Society was the first journal dedicated to scholarship in the business ethics field, beginning publication in 1960. Prior to that time, business ethics scholars published in general business journals. About a decade later, Business and Society Review, Journal of Law Medicine & Ethics, and Business & Professional Ethics Journal followed with first volumes in 1972, 1973, and 1981, respectively. The Journal of Business Ethics, the journal which has come to dominate the list of business ethics journals, began publication in 1982. Currently, it is the only business ethics-specific journal on a general business journal list, the Financial Times FT45. In the following several decades, another 27 journals began publication; a few of them also ceased publication, with International Journal of Value-Based Management and Teaching Business Ethics merging into the Journal of Business Ethics in 2004. Business Ethics Quarterly, also often considered one of the top journals in the business ethics field, started publication in 1991. 2 Figure 1 shows the timeline of journals in business ethics.
Timeline of business ethics journals.
The entire set of publications in business ethics covers a relatively recent time span. A resultant shortcoming is that there is not a robust set of impact factors. There are, however, ranked lists based on reputation.
Criticism of Journal Ranking
The underlying assumption has been that higher ranked journals reflect inclusion of higher quality articles. However, Smith (2004) found both Type I errors—“top” articles as indicated by subsequent citations were published in non-top journals—and Type II errors—“non-top” articles were published in top journals (p. 148). Chow, Haddad, Singh, and Wu (2007) found “substantial classification errors from using publication in a top-three journal as a proxy for an article’s contribution” (p. 411). Singh, Haddad, and Chow (2007) found that “there were substantial classification errors from using journal ranking as a proxy for quality” (p. 319). Peters and Ceci (1982, p. 187) selected 12 already published research articles from 12 highly regarded and widely read journals, substituted fictitious names and institutions for the originals, and formally resubmitted the manuscripts to the same journals. Three were detected as having already been published; of the remaining nine manuscripts, eight were rejected, indicating inconsistency in the review process over time. Starbuck found that there is much overlap in different prestige strata: “Every category of journals publishes articles with a very wide range of true values, so average true values for journals actually provide no useful information about any single article” (Starbuck, 2005, p. 15). “Evaluating articles based primarily on which journals published them is more likely than not to yield incorrect assessments of articles’ values” (Starbuck, 2005, p. 17).
Both researchers and administrators may introduce bias into ranked lists of perceived journal quality (Van Fleet, McWilliams, & Siegel, 2000). Examining citation bias, MacRoberts and MacRoberts (1989) found that “text-bibliography discrepancies ranged from a failure to cite basic assumptions and background knowledge to referencing works that were not influential” (p. 343). For the articles they studied, only about 30% of the material that was influenced by the work of others was actually cited, indicating that citations may not be a good measure of influence (MacRoberts & MacRoberts, 1989, p. 343). They found that biased citing was common, with “no correlation between the frequency of use and the frequency of citation” finding some works “disproportionately used and disproportionately cited” (MacRoberts & MacRoberts, 1989, p. 343), with 38% of citations to secondary sources (MacRoberts & MacRoberts, 1989, p. 344). There is also the problem of self-citation. Citation rates vary as well by discipline and country of publication (MacRoberts & MacRoberts, 1989, p. 343). The herding concept may also be at work, meaning that where an article has been cited by one, it is then also cited by others.
Ellis and Durden (1991) found that for economics journals, the past reputation of a journal influences its current ranking, creating a bias toward older, more established journals. Serenko and Dohan (2011, p. 635) found that the impact of journal longevity depends on the ranking method. Cahn (2011) explored the hypothesis that journal age may be a factor affecting journal reputation, and consequently its rank, using a representative but not exhaustive sample. It appeared that a journal’s age is a factor in its reputation. Journal size also appeared to be a factor in journal reputation. It may be that scholars send manuscripts to journals with name recognition, the older journals, resulting in higher selectivity and better articles in older journals. Or, it may be that, in surveys, the journals with the longest history are perceived as best because of name recognition.
Method
The objective of this study is to investigate two variables that earlier researchers have suggested may affect journal rankings although they have little connection with article quality. In particular, the correlations of journal age and size with published rankings are studied for business ethics journals. As discussed above, both journal age and journal size have been hypothesized as possible biases in journal rankings. Correlation with rank would demonstrate that the bias effectively skews the rankings. Lack of correlation would indicate that the visibility of age and size does not significantly affect the rankings.
Because journal rank is used as a proxy measure, it is important to understand how well it represents the quality of scholarship that it is intended to reflect. Authors submitting articles to older, larger journals would benefit from a bias in favor of such journals; if accepted for publication, those articles would benefit from a halo effect even before being cited by other scholars. University administrators, granting agencies, and other stakeholders who use publications to measure faculty quality should be aware of the extent to which journal rank, which is only a proxy measure of quality, may be biased by factors that do not directly measure article quality.
This research note studies business ethics journals to examine the effect of a journal’s age and size on its ranking or reputation. A list of business ethics journals was compiled using the journals studied earlier by Beets et al. (2016), Albrecht et al. (2010), and Serenko and Bontis (2009) along with a keyword search. That list is shown in Table 1.
Business Ethics Journals in Alphabetical Order by Title With Year Founded.
Data were collected on journal age and the number of articles published by each journal from 2010 to 2012. Rankings by Beets et al. (2016), Albrecht et al. (2010), and Serenko and Bontis (2009) were noted for those journals covered by their respective studies. Research on Professional Responsibility and Ethics in Accounting and Business Ethics were omitted from the sample. The former is published as a book series, and the latter as a magazine. Due to inconsistency with the other academic, refereed journals, they were omitted. The sample was also limited by data availability. Journals for which sufficient data were available are shown in Table 2.
Business Ethics Journal Metrics.
Journal age is measured by the number of years of continuous publication. For most journals, such as the oldest BEC journal Business & Society, this age measure is straightforward. For some journals, there were discontinuities like a change of name or merger with another journal or a change in editorial policy. For example, the Journal of Business Ethics had several journals merge in and also includes a number of special issues. Business and Society Review changed the type of article in 1996 from essays on topics of business ethics concern to research articles. Years with scholarly articles were treated as continuous years of publication without regard to mergers of journals.
Journal size was measured as total number of articles per year. Comment articles were excluded from the count of total number of articles. The intent was to capture the structure of available publication outlets for scholarly articles analogous to the way market structure is measured by the number and relative size of firms in an industry. With one journal, the Journal of Business Ethics, much larger than the others, the market for business ethics articles appears to be an oligopoly. As an illustration of how journal size has changed over time, Figure 2 shows the number of articles per year for Business & Society, Journal of Business Ethics, and Business Ethics Quarterly for the entire time span of their publication. The time span chosen for the analysis averaged the number of articles per year over the years 2010-2012, roughly comparable with the timing of the most recent of the business ethics journal rankings: Beets et al. (2016). Averaging over 3 years helps overcome the possibility that any one year may not be representative, as might happen if there were a special topic issue. The relatively compact average avoids the possibility that changes over long spans of time in a journal’s publication policy will blur its relative size to other journals; relative size blurring might be an issue especially for younger journals.
Journal size timeline.
Linear regression was used to test for the relationship of each ranking with age and size. Ordinarily models estimated using ranked data are estimated using discrete choice method (see Green, 1993, pp. 832-833). Discrete choice models are nonlinear in parameters. They require intensive search techniques to find the estimated parameter coefficients. Tests of parameter significance rely on asymptotic estimates (Green, 1993, p. 779). Because this study has a small data sample, discrete choice methods are not workable. The issue then becomes whether to not use the data or whether they still provide some insights. This analysis assumes that the intervals between any two successive rankings can be treated as comparable intervals. Although not strictly correct, as a result of this assumption, linear regression can be used to extract relationships from the data set used in this study.
Results
For all three rankings of business ethics journals, regression results showed correlations in the expected direction: Older journals had higher rankings, and journals with more articles per year had higher rankings. Because high rank is indicated by lower numbers (highest rank is 1), these correlations are negative. However, the only significant correlation found in this study is for the Beets et al. (2016) ranking, dominated by the age variable (see Table 3). The other two rankings were not significantly correlated with age or size. Beets et al.’s (2016) is a survey-based ranking derived from lists used administratively by AACSB schools for faculty evaluation and tenure/promotion decisions. Based on these results, this administratively derived ranking may be biased in favor of older, more well-known journals. This finding confirms the conclusion of earlier researchers (Chow et al., 2007; Singh et al., 2007; Smith, 2004; Starbuck, 2005) that using journal rank as a proxy for article quality may introduce the possibility of error into the evaluation process. In turn, error may be introduced into evaluation of faculty quality by judging scholarship on the basis of ranking of the publishing journals.
Regression of Journal Rank on Age and Size.
Because the Journal of Business Ethics has so many more articles per year than any of the other business ethics journals, it may be biasing the relationships observed above. Regressions of journal rank on age and size were also done without the Journal of Business Ethics (see Table 4). Results were similar. Age is still significant in the Beets et al. (2016) regression; otherwise, all other variables are insignificant. Typically, removal of a data point is justified when the data point is an anomaly that distorts the relationship between the dependent and independent variables. One could argue that the Journal of Business Ethics should be included in the sample to get a more realistic picture of the journal-ranking relationships.
Regression of Journal Rank on Age and Size Without Journal of Business Ethics.
Unfortunately, the small sample size lowers statistical power. Business ethics is a relatively young growing field so the sample size is necessarily small, and consequently, there is a lack of variation in the data. As time passes, the data set will become richer in information, and a follow-up study would estimate relationships more precisely.
Discussion
For the business ethics journals examined in this study, the most significant relationship of the three ranking schemes examined was with the Beets et al. (2016) ranking. Given the mixed results, it is of interest to note how the ranking that is significantly correlated with journal age is different from the rankings that are not significantly correlated. Beets et al.’s is the ranking that was based on administrative systems used for evaluating quality of scholarly output at AACSB schools; this ranking, therefore, reflects administrative perspective, whereas the other rankings are more closely tied to faculty and authors. Age was correlated with the administratively derived journal rank. Perhaps a journal’s reputation precedes it, as perception of journal quality may be biased by how well known that journal is from having been in the public eye, either because it has been published and available for a number of years or because it is well known and likely to be cited.
The other two journal rankings also showed higher rank for older, larger journals, although the correlation was not statistically significant. The Albrecht et al. (2010) and Serenko and Bontis (2009) studies, respectively, are based on a survey of faculty perception and on citations. These measures are derived more closely from input of authors rather than administrators.
The bias found in some of the results of the present study corroborates results of earlier studies that visibility characteristics of journals can sometimes result in higher rank. Authors such as Chow et al. (2007), Smith (2004), and Starbuck (2005) advocated against using journal rank as a measure of author quality, in part, because of the possibility of such bias.
An inference derived from the inconsistency of results from regressions with the three differently sourced rankings of business ethics journals is the conclusion that significant differences may occur in rankings depending on how they were created. Scholars working on journal rankings should be particularly careful about the data they use to build rankings, as data based on perceptions may reflect bias including name recognition of older, more established journals. Perception bias investigated here is a small part of the complex story of journal quality, article quality, and scholar quality.
As journal ranking is, at best, an imperfect reflection of journal quality, considerable caution is recommended for using journal rank to evaluate author quality. In a quote attributed to Albert Einstein, “Not everything that counts can be counted, and not everything that can be counted counts.” 3
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.