How Media Scholars’ Attributes Affect Their Ratings of Journalism and Mass Communication Quarterly *

Abstract

This article examines how media scholars’ attributes affect ratings of Journalism and Mass Communication Quarterly (JMCQ), based on the 2014 JMCQ readership survey. It compares the impacts of these attributes on five different types of subjective journal ratings. For instance, importance of journal impact factor in the respondent’s institution only affects the rating of the journal’s standing in the field. Attributes such as use and knowledge of the journal, research recognition received in the field, doctoral institution affiliation, and ethnicity consistently predict the rating of the journal’s standing in the field, overall ratings, and relevance to the respondents; but other attributes predict the ratings of the journal in serving the association members well and author-friendliness.

Keywords

Impact Factor Journal Rating Journal Ranking Media Scholars Journalism and Mass Communication Quarterly

Research reputation is the most important criterion in evaluating a faculty’s performance in research in tenure and promotion decision in universities. Apart from hiring external reviewers to evaluate a faculty member’s research publications, administrators often consult quantitative journal ranking measures, which show the reputation of the journals the faculty publishes in, such as its acceptance rate, its subjective ratings by scholars in the field, and its impact factor. ¹ Consequently, authors are steered toward publishing in those journals that have favorable quantitative measures.

The field of communication is no exception. The National Communication Association’s report for the Council of Communication Associations, titled The Impact Factors, Journal Quality and Communication Journals, revealed the doubling of communication journals included in Social Science Citation Index from 36 to 72 between 1997 and 2012.² There are many more communication journals not included in the index. With the proliferation of academic journals, the importance of these quantitative measures increases as administrators rely on them to distinguish the quality and significance of the research articles of the faculty members. As these quantitative measures are tied to the career advancement of researchers, it is important to understand the nature of these measures and the factors influencing these measures. The recent JMCQ readership survey results can provide some insights on factors influencing media scholars’ rating of JMCQ.

In general, quantitative journal ranking measures can be divided into two types:

Subjective rating measures by surveying scholars or peers in the subject field and

Ranking by computation such as a count of the number of citations of the journal’s articles (e.g., journal impact factors) or probability of acceptance (article acceptance rates). The former is subjective and highly dependent on who the raters are in the survey. The latter is considered “objective” as people believe numbers don’t lie as no human judgment is involved.³ Haensly, Hodges & Davenport’s study showed that a journal with higher citation rates (impact factors) and survey-based rankings (ratings) is likely to have lower acceptance rates in the field of economics and finance. ⁴ Another study comparing the correlation between subjective ratings and citation ranking found high correlations between the two measures in biomedical sciences and social sciences, but not in physical sciences.⁵

Many new journals and commercial journal publishers, in particular, promote the use of the citation ranking method because they can compete with established journals for authors through the number of citations and libraries that subscribe to journals based on their “impact factor.” The NCA report showed that Council of Communication Association- (CCA) sponsored journals using humanistic methods are lower in journal impact factors than journals using social scientific methods and newer journals.⁶ The validity of citation ranking varies by field and approaches.

Journal reputation is the perception of the journal brand by scholars in the field. It may or may not take into account the citation ranking. The perceptions of a journal’s prestige, quality and leadership in the field constitute its reputation. How scholars form the perception for a journal’s reputation can help the academe understand the factors influencing journal ratings and reveal advantages and limitations of journal ratings as a measure of journal reputation. This article uses Journalism and Mass Communication Quarterly (JMCQ) as a case study to illustrate the predictors of subjective rating of a journal by media scholars, taking into account the importance of journal impact factor in the researcher’s institution as a potential predictor.

Evolution of Academic Journal Ratings

The idea of Journal Impact Factor (JIF) was originally proposed by Eugene Garfield in 1955 to assess the quality of journals.⁷ JIF is calculated for each year, using the number of citations of a journal’s content in the preceding two years divided by the number of citable materials published by that same journal.⁸ Usually, scholars cite previous research literature for the following purposes: motivating new research, giving credit for ideas, offering detailed information about methodology, highlighting or correcting preexisting errors, and establishing the legitimacy of a new claim.⁹ Citable materials include original research articles, literature reviews, technical notes, but do not include letters to the editor, book reviews and editorials.¹⁰ As early as in the 1970s, Myers¹¹ conducted a comprehensive study of psychology journals and found that citation counts are a reliable and valid measure of research achievements among psychologists. Now it has become a widely accepted index to measure the frequency of journal citation and compare them among different journals. The adoption of JIF makes evaluation of researchers’ standing easy, which creates pressure for scholars who want to get promotion to publish more articles in journals with higher JIF.¹² Apart from JIF, there are several other variations of citation-based journal rankings such as SCImago Journal Rank, Google Scholar’s h5-index, which is the number of times the articles have been cited in the last five calendar years, and Microsoft Academic’s “field rating.”¹³

Do we still need subjective ratings?

Journal Impact Factor has become a useful index of journal ranking due to several reasons. First, an evaluation system based on impact factor has been acclaimed by many scholars because it is concrete as opposed to being anecdotal.¹⁴ Second, the creation of journal impact factor as a standardized measure makes it convenient to assess and compare the quality of different journals. As a result, impact factor is widely used by governments and funding agencies, especially outside the United States.¹⁵ Feeley & Moon’s study showed that the journal impact factor is a reliable indicator of impact for communication journals.¹⁶ But Smith argued that there are always limitations in relying on one single approach to determine the quality of a journal. ¹⁷ As he pointed out, impact factor can be misleading in some circumstances since it is based on the measures of citations. There are three limitations of evaluating journals based on citations. First, authors have a tendency of citing themselves to boost their rankings in the evaluation system intentionally. Second, citation rates do not give information about whether a citation is for correction purposes or not. In a correctional citation, the author might point out the shortcomings of the cited research, which might show the cited article was of poor quality, rather than a top article. Third, citation rates can also be influenced by the popularity of a research topic. Since a hot topic has a certain time frame, a journal with more articles related to a hot topic, especially if they are published at the early stage of the topic’s popularity time frame, is likely to have a higher citation rate. Bornmann, Marx, Gasparyan, and Kitas also argued that one serious limitation of JIF is the uncertainty over what “citable” items should be counted in the formula because they are debatable and there were even negotiations between journal publishers and Thomson Reuters, the publisher of Journal Citation Reports.¹⁸

Rejection rate/acceptance rate is also problematic because such information may not be available for some journals. It also can be a reflection of unclear editorial policy leading to many inappropriate and low quality submissions, which result in high desk rejection rate.¹⁹

Christenson and Sigelman compared the subjective ratings of political science and sociology journals to Social Science Citation Index (SSCI) journal impact.²⁰ They concluded that many journals’ prestige does not match the journal impact in the social science research community. They argued that reputational measures of journal quality reflect persistent stereotypes. However, Dometrius argued that journal prestige is such a complex concept that its subtle nuances can be better captured by subjective evaluations than by counting citations. ²¹

Subjective rating by peers in the field is subject to all the common problems faced by self-reported responses in such surveys. It is much more costly to conduct a representative sample of scholars in the field on a regular basis than computation methods such as citation ranking and rejection rates. However, the ability to use multiple criteria in evaluating a journal and the human holistic judgment of a journal by individuals are some important advantages of subjective ratings. One may also argue that subjective ratings may affect the scholar more since the scholarly community is quite confined and peer recognition is important in the daily life of an academic. But more is needed to examine the factors influencing the subjective rating of a journal.

Demographic characteristics in journal assessment

A study of the readership of Public Opinion Quarterly investigated how demographic variables affect readership and assessment of the journal. ²² Age did not affect the ratings of the journal although the youngest and oldest age groups read more articles than the two middle-aged groups. Research interest affected the journal evaluation as the study found that political scientists and sociologists gave the most favorable assessment of the journal. Membership did not explain the ratings of the journal either. McLean, Blais, Garand, and Giles conducted a comparative survey study of journal ratings among political scientists in three countries – Canada, UK and USA. ²³ The results showed scholars have a tendency to rate journals from their home countries as more reputable than those from foreign countries: the top journal with higher impact in the UK than the USA were all edited from the UK; the top three journals with higher impact in Canada than the USA were edited from Canada. This was due to their inclusion of familiarity in the rating. The ‘impact’ score used in that study was calculated by the formula used by Garand: Impact = mean rating + (familiarity * mean rating.). ²⁴

Familiarity with JMCQ

In Garand’s impact rating measurement, familiarity with the journal is viewed as a component of the rating. ²⁵ In our opinion, familiarity with the journals consists of three dimensions: knowledge, use and readership. Someone who uses the journal for research, teaching and service (review), knowledgeable about the journal and frequently read the same journal, should rate it higher than those who don’t. In our study, however, we argue that familiarity with a journal is a predictor rather than the rating itself and assess how much it affects the rating instead of counting it in the rating.

Credentials of the Media Scholars

Editors of quality academic journals strive to recruit the best authors in the field to publish their works in their journals. Highly productive researchers are visible in the field and are more well-known than those who rarely publish. In addition, recognition in the field in terms of research awards, top papers, research grants, etc. also are important indicators of reputation in the field for the individual rater. With their higher expectation of the standards of the articles, the higher credentials of the raters with higher research recognition and productivity may lead to a lower rating of a journal.

Surveillance of the Field

When subjective ratings are used, one assumes the raters will not just be familiar with the journal being rated, but with the field as a whole. Surveillance of the field through membership in more academic professional associations will increase the rater’s exposure to other journals and the field in general. Hence we expect that membership in other communication associations will affect their ratings of JMCQ.

In addition to being a member of different associations, a researcher can utilize the Internet and social media to monitor trends in the field. Through subscription to journal publishers’ e-mail alerts, researchers can obtain the latest article listings with abstracts in these e-mail alerts. In addition, many professional social media such as ResearchGate and Academia.edu encourage the sharing of online links to articles, or even allow users to post and share full length articles. These affect the scholar’s familiarity with the field as a whole and help in keeping them up to date with current research trends. Such surveillance of the field may affect the judgment of a journal.

Manuscript Acceptance in the Journal

Koulack and Keselman’s study of a flagship psychology journal in 1975 suggested that journal ratings are not correlated with either submissions, rejections or subscriptions. ²⁶ However, in a recent study on the authors’ ratings of JMCQ, acceptance by JMCQ was revealed as an important factor in the evaluation of the journal.²⁷ This discrepancy may be caused by different fields and the rating measure. But more importantly, now there are many more journals than in the 70s. Academic journal publishing is no longer a publisher’s market, but also an author’s market. Authors have many more outlets to choose than before and the pressure to publish is much higher because of the increase in accountability in higher education trends around the world. So, manuscript acceptance is a positive step in their careers and to reassure themselves and their administrators of the quality of their work, they may rate journals that accepted their work as better than those that did not accept their work. Hence, we included acceptance by the journal as one of the predictors of the journal’s rating.

Importance of Impact Factor in Evaluating Research by Administration

Due to the prevalent use of journal impact factor in higher education worldwide, it is expected that the perceived importance of journal impact factor by scholars’ administration in the evaluation of their research may affect the rating of a journal. Since JMCQ and some other older CCA-sponsored journals have a lower impact factor than some newer journals, this study examines whether the importance given to impact factor by the respondents’ institution may negatively affect their rating of JMCQ.

Research Questions

There are two research questions in the study to examine the predictors of JMCQ rating.

RQ1. To what extent do the respondents’ demographic characteristics, research credentials, familiarity with the journal, book review readership, surveillance of the field, previous manuscript acceptance at the journal, and importance of impact factor in the respondent’s institution, predict their rating of JMCQ’s standing in the field?

RQ2. Do the predictors differ in different types of journal ratings?

Method

The study is a part of the JMCQ readership survey which was distributed online from October 1 to November 1, 2014. Both AEJMC members and media scholars who are not AEJMC members were sent invitations to participate in the study. As an incentive to encourage participation, a 30% Sage book discount was offered to participants and the participant could also use the survey as an opportunity to volunteer to review for JMCQ as either book reviewer or journal article reviewer after submitting the survey. The survey was anonymous to encourage candid comments on the journal. AEJMC members and non-members were given different versions of the survey with mostly identical questions. The difference between the two versions of the survey was that AEJMC members enjoyed exclusive benefits such as full online access of JMCQ and e-mail notification by AEJMC, hence they received some specific questions on those benefits. Members were also asked how long they had been an AEJMC member.

AEJMC members were invited to participate in the survey via AEJMC official e-mails three times (two in the first three days and the last one week before November 1). Non-AEJMC members were reached through various e-mail listservs accessible to the researchers such as Association of Internet Researchers, Chinese Communication Association, Korean Communication Association, and posting on scholarly social media such as LinkedIn’s Mass Communication Scholars Group, Sage communication scholar Twitter and LinkedIn pages, and National Communication Association Mass Communication Division Twitter Page. In addition, JMCQ authors/reviewers were e-mailed the survey invitation twice. The response rate of AEJMC members was 24.5%. Response rate of non-AEJMC member media scholars could not be computed as no population number is available and many listservs and the postings used by the researchers had overlaps between AEJMC and non-members unknown to the researchers. However, we can safely assume the response rates were higher among AEJMC members because a much larger proportion of respondents were AEJMC members.

Measures

Ratings of JMCQ

Because journal rating is only a part of the readership survey, we simplified the rating items to a three-point scale from low, medium and high on each journal rating item and with an option of “don’t know” to facilitate valid response and minimize respondent fatigue. Those who marked “don’t know” were not included in the rating analysis. Five types of ratings were examined based on the different utility values of each type of rating.

a) Overall ratings (sum of attributes)

There were a total of nine attributes in the calculation of overall ratings as the sum of the items: 1) Prestige of the journal, 2) Quality of the articles, 3) Relevance to the respondent’s research, 4) Leadership in the field of communication, 5) Ideal publication outlet for the respondent’s research, 6) Useful for the respondent’s graduate teaching, 7) Useful for the respondent’s undergraduate teaching, 8) Ease of access, 9) Serve the members of the association well. This overall rating was a combination of both individual utility value to the respondents and general perceived reputation of the journal. The Cronbach’s Alpha for this 9-item scale is .84.

b) Standing in the field

To measure the media scholars’ rating of the journal’s standing in the field, we took out three items from the nine attributes: 1) Prestige of the journals, 2) Quality of the articles, 3) Leadership in the field of communication. The three attributes formed the reputation/standing in the field of the journal. The Cronbach’s Alpha for this 3-item scale is .81.

c) Relevance to the research of the respondents

Respondents were asked to rate on two items: 1) Relevance to respondent’s research, 2) Ideal publication outlet for respondent’s research. The Cronbach’s Alpha for the 2-item scale is .77.

d) Services to association members

This item is only useful for society/association based journals in which journal subscription is part of the membership benefit. But as JMCQ is a flagship journal of AEJMC, it was a useful evaluation criterion for the association. This is a one-item rating only.

e) Author-friendliness

This is a three-item rating based on the rater’s perception of the journal from the perspective of an author: 1) Quality of the reviews, 2) Fast turnaround of editorial decision, 3) Support service to authors. The Cronbach Alpha of this rating scale is .77.

Attributes of Media Scholars (Predictors)

a. Importance of impact factor in research evaluation

Respondents were asked the importance of impact factor in their own institution in evaluating their research from unimportant (1) to very important (5).

b. Demographic characteristics

Demographic characteristics were all coded as dummy variables, include ethnicity (Caucasian or non-Caucasian), JMCQ membership (member or non-member), gender, and type of institution (doctoral or non-doctoral).²⁸

c. Credentials of the respondent

The credentials of the respondent consisted of three components: 1) Academic rank from the lowest of graduate student (1), instructor/visiting faculty/post-doc, assistant professor or equivalent, associate professor or equivalent, to the highest of full professor or equivalent/retired faculty (5). 2) Research productivity was measured by the self-reported number of refereed journal articles the respondent published in their entire career. 3) Research recognition was a sum of the presence of five items: Recipient of a) competitive external research grants, b) book award or journal article award, c) top paper award in national or international conference, d) fellowship, and e) research award in major academic association.

d. Familiarity with JMCQ

The media scholars’ familiarity with JMCQ was determined by three indicators in the study: 1) JMCQ use, 2) knowledge of JMCQ, and 3) frequency of print issue readership. JMCQ use is the sum of the respondent’s use JMCQ for research, teaching and service (review for JMCQ) ranging from 0 to 3. Knowledge of JMCQ is based on whether the respondent knew six facts about JMCQ — average editorial turnaround time, number of peer reviews for manuscripts, circulation size, the fact that the journal is the oldest mass communication journal, acceptance style change from Chicago to APA, and JMCQ’s OnlineFirst, which is the online publication of forthcoming print articles of the journal — ranging from 0 to 6 (0 no knowledge, 6 highest knowledge). For print issue readers, we asked how often they read the journal — from on demand only to reading almost every issue. The score ranged from 1 to 3.

e. Surveillance of the field

In this study, surveillance of the field consisted of three indicators: Major communication association membership, subscription to journal e-alerts, and professional social media use. Major communication association membership was the sum of the respondent’s memberships in major national and international communication associations such as Broadcast Education Association (BEA), International Communication Association (ICA), National Communication Association (NCA) and International Association of Mass Communication Researchers (IAMCR). Subscription to journal e-mail alerts was based on the question whether the respondent subscribes to any academic journal publisher’s content alert or OnlineFirst alert (yes/no). Professional social media use was measured by the sum of the use of online social media for professionals such as Linkedin, ResearchGate and Academia.edu.

f. Acceptance in journal

Acceptance in journal was based on the question whether the respondent is the author of any article(s) published in JMCQ.

g. Book review readership

JMCQ is one of the few journals that still offer a substantive number of book reviews in an issue. The study also looks at whether book review readership influenced the rating of the journal. Book review readership was based on the question whether the respondents read book reviews (yes/no).

To avoid the creation of bias in the ratings with regard to knowledge of JMCQ facts, the rating questions were asked much earlier than the recognition of the JMCQ facts questions. JMCQ knowledge questions were placed toward the end of the questionnaire before the demographic questions.

Results

A total of 1,142 media scholars responded to the survey, of which 827 were AEJMC members and 315 were non-AEJMC member media scholars. To avoid the bias of respondents highly involved in the journal with vested interests, respondents who were current or former editors or associate editors of JMCQ and the AEJMC publications committee members were excluded from the data analysis. In addition, respondents who did not answer or marked “don’t know” in the rating questions were not included in the analysis.

Table 1 is a comparison of the importance of each predictor on each type of rating. All five regression models predicting the different ratings of JMCQ were statistically significant (p < .01). Multicollinearity diagnostics were run and none of the predictors in the model had tolerance values of concern. The chosen set of predictors performed best for overall journal rating (r square =.24), explaining 24% of the variance in the overall rating of JMCQ. They also performed almost similarly well in predicting the ratings of JMCQ’s standing in the field (r square = .21). JMCQ’s standing in the field was predicted by many more media scholar attributes than other ratings. Only 5 out of the 16 predictors did not significantly predict the journal’s standing.

Table 1.

Predictors of Different Journal Rating Measures.

	JMCQ Standing in the FieldBeta^a	JMCQ Overall RatingBeta^a	JMCQ RelevanceBeta^a	Serving membersBeta^a	Author-Friendliness RatingBeta^a
1. Importance of impact factor	−.10**	0.00	0.00	.04	.04
Demographic Attributes
2. Ethnicity (Caucasian = =1, non-Caucasian=0)	−.08*	−.19***	−.17**	−.12*	−.05
3. Gender (male =1, female = 0)	−.07	−.001	−.03	.05	.02
4. Institution with doctoral programs in communication (1=doctoral,0=non-doctoral)	−.12**	−.19**	−.09*	−.15	−.05
5. Membership in AEJMC	−.08*	−.08	−.09*	−.09*	−.12*
Credential Attributes
6. Academic Rank	−.02	−.14*	−.20**	−.01	−.05
7. Number of refereed journal article publications (research productivity)	−.07	.03	.05	−.09*	−.06
8. Research recognition	−.12**	−.14**	−.14**	−.08	−.03
Familiarity with JMCQ (9-11)
9. JMCQ Use	.11**	.13**	.25***	.18**	.08
10. JMCQ Knowledge	.13**	.10*	.09*	.14**	.21**
11. Frequency of reading print JMCQ	−.04	−.02	.05	.03	−.03
Surveillance of the Field (12-14)
12. Number of major professional communication membership	−.15***	−.08	−.06	−.05	−.12*
13. Subscribe to email alert	−.15***	−.07	−.03	−.04	.08
14. Professional social media use	.10**	.06	.05	.04	.08
15. Acceptance in JMCQ	−.02	0.00	.18**	−.01	.16**
16. Reading book review	.16***	.07	0.02	.10*	.02
R Square	.21	.24	.20	.11	.13
N	626	432	658	547	332
F	10.15***	8.38***	10.03***	4.18***	2.93***

p < .05, ** p < .01, *** p < .001.

Standardized regression coefficients.

Attributes Predicting JMCQ’s Standing in the Field

Our results showed that the importance of journal impact factor in the respondent’s institution was indeed one statistically significant factor (beta=-.10, p < .01) affecting the standing of JMCQ in the field. But it was not significant in predicting any other types of rating measures of the journal. In addition to journal impact factor, there were other more important factors in predicting the standing of the journal in the field such as respondents’ surveillance of the field in terms of subscription to journal e-alerts (beta=-.15, p < .001), membership in other communication associations (beta=-.15, p < .001), and professional social media use (beta=.10, p < .01); familiarity with JMCQ in terms of knowledge of JMCQ (beta=.13, p < .01) and use of JMCQ (beta=.11, p < .01). However, the frequency of reading the JMCQ print issue did not affect any of the ratings.

Respondents’ credentials influenced ratings in different ways. Research recognitions of the respondents negatively predicted JMCQ’s standing in the field (beta= -.12, p < .01), overall JMCQ rating (beta=-.14, p < .01) and relevance of the journal (beta=-.14, p < .01). But respondents’ research productivity did not affect any of these three rating measures. It only negatively affected their ratings of JMCQ as serving the association members well (beta=-.09, p < .05). Academic rank of the respondents was a significant negative predictor of overall rating (beta=.14, p< 0.05) and relevance (beta=.20, p<0.01), but insignificant in predicting JMCQ’s standing in the field (beta=.02, n.s.) or other ratings.

Among the demographic variables, gender was not statistically significant in any of the rating measures. In contrast, the ethnicity of the author was significant in all the measures except rating the journal’s author-friendliness. Caucasian respondents rated JMCQ’s standing in the field more negatively compared to others (beta= -.08, p < .05), and the same trend was visible in the overall ratings (beta=-.19, p < .001), relevance to the field (beta= -.17, p < .01) and in whether it serves the members of the association well (beta= -.12, p < .05). JMCQ membership was a negative predictor for all the rating measures of JMCQ.

Manuscript acceptance in JMCQ was only significant in positively predicting the relevance of the journal (beta=.18, p < .01) and author friendliness (beta=.16, p < .01), which is consistent with the findings of Watson and Riffe,²⁹ but not related at all to JMCQ’s standing in the field (beta=-.02, n.s.) or overall rating (beta=.00, n.s.).

Discussion

The negative effect of the importance of impact factor in respondents’ institutions on the standing of JMCQ (but not on other ratings of the journal) shows that impact factor does play a role in the perceived standing of the journal in the field. Hence, achieving an impact factor level comparable to peer journals can enhance the standing of the journal. But the fact that it is not the most important predictor, seems to indicate other considerations such as the long history of JMCQ and its status as the flagship journal of AEJMC are able to somewhat mitigate the negative effect of importance of journal impact factor used by the respondents’ administration in the subjective rating.

There was no one dominating media scholar attribute found in predicting JMCQ’s standing in the field. This shows that a journal’s standing in the field is a very complex matter and is influenced by many factors. Media scholars’ attributes are just some of the factors and many other factors can contribute to the ratings, which can be further examined in future studies.

The fact that more junior and non-Caucasian scholars gave higher rating to JMCQ’s standing in the field, overall ratings and relevance to their research, reflects that the journal is a desirable outlet for these scholars and can contribute to diversity in the field. The tendency of scholars with higher credentials giving lower ratings of JMCQ indicated that they indeed set higher standards in their ratings for journals than other scholars and did not see JMCQ as much relevant to their research at their stage of career. The consistency among the predictors regarding JMCQ’s standing in the field, overall ratings, and relevance showed that these three ratings are influenced by a similar set of factors. But these same predictors did not significantly predict the journal’s author-friendliness or whether it serves association members well. This shows that these ratings were influenced by other factors that can be examined in the future. That AEJMC membership factor was negative in predicting all the ratings of JMCQ was an issue of concern because members were less favorable of JMCQ than non-members. As the flagship journal of AEJMC, JMCQ needs to identify specific areas of concerns of and increase engagement of AEJMC members.

Familiarity with JMCQ was favorable to the ratings of JMCQ. As the use of JMCQ and JMCQ knowledge were the consistent significant positive predictors in all the five rating measures, boosting the use of the journal for teaching and review service in addition to research, and increasing users’ knowledge of the journal, will be beneficial in enhancing the rating of the journal.

One remarkable finding of the study was the significance of book review readership to JMCQ’s standing in the field and how it served members of the association. Although research articles are the main use of the journal, book reviews can add authoritativeness to the journal and be seen as a useful service to the members. Nonetheless, book review readership does not increase perceived relevance and overall rating of the journal.

Surveillance of the field is a mixed blessing for a journal’s rating. When media scholars actively monitor the academe and research trends, such as those who subscribe to publishers’ journal alerts and have more memberships in other major communication associations, they were more likely to give lower rating when asked about JMCQ’s standing in the field. They might be more likely to compare JMCQ with other journals and feel that JMCQ does not fare as well. We did not ask specifically which journal’s e-alert respondents subscribed to. It is likely that it’s not JMCQ because many members might not see the need to subscribe to it with the print journals being delivered to their doorsteps. In fact, a majority of the respondents (65%) indicated interest in e-alert of JMCQ being sent to them via AEJMC e-mails. Hence, increasing e-alert of JMCQ may help create more top-of-mind awareness about the journal. It’s also interesting to see those who used more professional social media rated JMCQ more positively when it came to the journal’s standing in the field. Hence, increasing JMCQ’s presence in social media will benefit its standing in the field.

There are limitations to the study. The sample of non-AEJMC members was recruited by convenience rather than probability sample. Some measures might not be comprehensive such as parameters regarding knowledge of JMCQ that focused more on the positive facts of the journal. The considerable number of “don’t knows” and missing answers for the rating questions showed that many respondents felt incapable or were not willing to rate a journal.

This study shows that the ratings of an association journal such as JMCQ can be influenced by specific characteristics of media scholars with knowledge and use of the journal as consistent positive predictors. Respondents who were more knowledgeable about JMCQ and used JMCQ for teaching and service, tended to rate it more positively. Hence, any future subjective rating study should probe the respondents’ knowledge and use of each journal to provide a more informed judgment on a journal’s performance and standing in the field. Using subjective ratings can supplement other computation-based ratings to provide a more comprehensive picture of the quality and reputation of academic journals.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

*

This article is non-refereed.

Notes

Author Biographies

Louisa Ha is professor in the School of Media and Communication, Bowling Green State University and editor of Journalism and Mass Communication Quarterly. Chen Yang, Ling Fang, Tao Zhang, Dhiman Chattopadhyay and Fang Wang are doctoral students in the School of Media and Communication, Bowling Green State University.