Abstract
The aim of the study was to explore the impact of peer-reviewed psychology journals on Wikipedia articles. We are presenting a rank of academic journals classified as pertaining to psychology, most cited on Wikipedia, as well as a rank of general-themed academic journals that were most frequently referenced in Wikipedia entries related to psychology. We then compare the list to journals that are considered most prestigious according to the SciMago journal rank score. Additionally, we describe the time trajectories of the knowledge transfer from the moment of the publication of an article to its citation in Wikipedia. We propose that the citation rate on Wikipedia, next to the traditional citation index, may be a good indicator of the work’s impact in the field of psychology.
Journal quality and rankings have been a topic of recurring debates in psychology research (Mayor, 2010; Paulus et al., 2018). Some researchers approached this issue as early as in the 1970s by studying the ranking of journals by American Psychology Association (APA) members (Koulack & Keselman, 1975). There have been a number of studies using different measures of impact, including the impact factor, over the years as well (Feingold, 1989; Rushton & Roediger, 1978; White & White, 1977), pinpointing various problems of journal rankings in psychology (Boor, 1982; Christensen-Szalanski & Beach, 1984; Over, 1978) and revisiting the issue of how to use them (Fehr et al., 1981; Stamps & Fehr, 1980).
The publications have relied on some form of professional evaluation, either directly, through professional associations’ members polling or, indirectly, through citations in other academic publications. In this article, we take a different approach: We analyze which psychology journals are most commonly referred to in articles in Wikipedia, the most popular and largest online encyclopedia developed by volunteers. We also study which journals are referred to in articles about psychology in Wikipedia. By doing this, we return to the conversations that have been important for the academic community of psychology researchers but that have not moved forward significantly over the past decades.
We present an analysis of academic journals used in Wikipedia articles about psychology, comparing the results to the existing established measures, such as the SciMago journal rank (SJR), and interpret the results and conclusions for psychology as a field. Our study is a large data set computational bibliometric analysis, and as such, it does not build on a social theory. However, it brings important results to our knowledge of the social impact of computing on academic work, directly within the scope of Social Science Computer Review.
Our results are significant for at least two reasons. First, Wikipedia is the most read source of academic knowledge worldwide. Journals commonly used there are by default quite likely more successful in reaching the general audience than the others, as Wikipedia is the first-stop for scientific information (Teplitskiy et al., 2017). Thus, learning which journals result in the largest general public impact is of high interest for academic psychologists. Second, since Wikipedia relies on contributions from nonexperts, and is curated by a large worldwide nonexpert community, which somewhat is a proxy for the general public, it is extremely interesting to discover what contributions they consider relevant in terms of sources of knowledge about psychology. This result should be of wide interest to psychologists as a profession.
Since discovering ways of reaching the general public with psychology science has been a topic of intensive research (Kaslow, 2015), just as much as the appeal of certain topics for the general public (Beck, 2010), our findings are a contribution to these conversations. Additionally, we already know that general science impact should be distinguished from impact in psychology (Hegarty & Walton, 2012). Measuring which psychology journals are most commonly reflected in a general audience encyclopedia, as well as which journals are most commonly used in articles on psychology topics, develops this important topic further.
Journal Rankings
Journal rankings are a topic of hot debate in many fields. For some scholars, they are a great example of Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure, p. 308” (Strathern, 1997) and a way of perpetuating inequalities (Davidson, 2019). Imbued with conundrums undermining the purpose of social scientific research (Özbilgin, 2009), they are still appealing to the academic milieu (Nkomo, 2009): After all, they allow using a simple heuristics of evaluating published research just by checking against a ranking table instead of reading it.
According to some scholars, journal rankings are a powerful tool of exerting bureaucratic control over faculty through strict measures (Bok, 2009; Jemielniak & Greenwood, 2015; Martin-Sardesai et al., 2017), admittedly as well, journal rankings entail ranking games and shift focus from doing research to publishing (Osterloh & Frey, 2015). In doing so, they may result in political struggles and detour the professoriate from their mission to reliably communicate the research outcomes to the general public (Macdonald & Kam, 2007) and derail research public funding by incentivizing publishing research results in outlets that may be linguistically and economically not available to the domestic readers (Neff, 2018). According to some findings, journal rankings may also contribute to suppressing interdisciplinary research (Rafols et al., 2012) by prioritizing journals that have a narrow scope as well as suppress diversity of ideas in psychological science (De Rosa, 2017).
Some critics also raise that the most popular journal rankings may be prone to a significant range of errors, technical and methodological flaws, or various biases (Amin & Mabe, 2000; Ferrer-Sapena et al., 2016; Vanclay, 2012), as well as interdisciplinary variations (Leydesdorff & Opthof, 2010). The lack of transparency in the fine details of the methodology applied each year contributes to the perception of a lack of academic rigor of such measures and encourages manipulation (Archambault & Larivière, 2009; Moustafa, 2015). Also, relying on citations alone as a measure of quality is prone to the error of not distinguishing between building on some work and criticizing it. There are also some indicators that the journal impact factor (JIF) alone is a poor reflection of scientific quality (Brembs et al., 2013). Moreover, gaming the system is quite widespread. For instance, coercing journal self-citation is common (Humphrey et al., 2019) and so are citation cartels (Fister et al., 2016; Perez et al., 2019). These problems spur some journals to remove JIF information from their websites (Casadevall et al., 2016), the use of JIF as a measure for evaluating research has been criticized at least over the last 20 years (Seglen, 1997), and focusing overly on citation metrics is openly called an aberration (Moustafa, 2016). Yet, factor-mania may be a variation of the “tragedy of the commons”: Even though the consequences are detrimental to science as a whole, individual scholars act rationally in their self-interest perpetuating it (Casadevall & Fang, 2014).
In spite of the wide and often justified criticism, journal rankings also have some undisputable practical merits, even beyond serving as a rough estimate of academic quality. For instance, they may be useful to the members of the academic community by offering them a method of distinguishing between predatory and nonpredatory journals (Demir, 2018; Sorokowski et al., 2017). Even more importantly, they are a simple, externally verifiable measure of the average impact an article published in the journal can make. Given the fact that science advances through dialogue with other researchers, which is reflected through citations and building upon someone else’s work, citation rankings are one of the very few available measures of possible influence on the body of academic knowledge as a whole.
Irrespective of one’s opinion on the usefulness of the rankings, since they become proxies for assessing the quality of individual work (Eacott, 2018), there is an increasing pressure on journals to fare well in them. Their importance is also growing (Tadajewski, 2016). For instance, bibliometric indicators are proliferating in tenure reviews, hire-or-fire decisions and have spread in most academic fields (Kulczycki, 2019; Rushforth & de Rijcke, 2015). Psychology journals also react to the dynamic increase in access to information and the popularity of journal rankings (Adair & Vohra, 2003). One of the effects is that especially low- and middle-impact journals are often inflated with self-citations, while the top ones are less affected by it (Anseel et al., 2004).
One of the possible reasons for this phenomenon may be that the alternatives to mainstream journal rankings are not widely available. Even though some researchers discuss the possibility to rely on consensus-based evaluations (Aledo et al., 2018), and there are also some metrics that include, for example, social networks’ popularity of publications (Papakostidis & Giannoudis, 2018), they are much less universally recognized or accepted. Some scholars also consider refining methodologies of journal evaluations (Pagani et al., 2015). It is quite clear that increasing the number of available, sensible methodologies of ranking journals may at least partly restore the balance in these metrics (Okumus et al., 2018). One possible alternative that we would like to discuss in this article could be checking which psychology journals are most commonly referred to in Wikipedia. Additionally, it is quite clear that any single journal ranking is prone to certain kinds of biases and errors inherent to its methodology. However, combining different measures may get us closer to a more diverse and balanced reflection of academic publication impact. For that purpose, we believe that supplementing measures such as IF or SJR with the measurement of popular use in a peer-produced encyclopedia is highly valuable.
Wikipedia: A Crowd-Sourced Encyclopedia
Wikipedia was established in 2001 and has quickly become one of the most popular websites in the world, currently ranked in the top 10 in TopSites ranking by Alexa.com. Its success can be partially attributed to the fact that it is “an encyclopedia that anyone can edit.” It relies on the wisdom of crowds (Surowiecki, 2004) through a largely self-governed, a-hierarchical, and spontaneous cooperative organization (Jemielniak, 2016; Konieczny, 2017). As a large-scale collaboration project, it is successfully crowd-sourcing knowledge dissemination (Reagle, 2010; Uhlmann et al., 2019).
Although Wikipedia communities in different languages have developed elaborate systems of rules and policies over the years (Jemielniak, 2014), including detailed conduct and civility norms, as well as very specific regulations about notability, the rules about source inclusion are quite straightforward: They have to be verifiable and reliable (Niederer & van Dijck, 2010). This is so to reduce the need for trusting the editors and to transfer this trust onto procedures so as to fully capitalize on the collective intelligence design of peer production (Benkler et al., 2015). These rules do not prevent, for example, hindsight bias (Oeberst et al., 2018), but they enforce using proper academic sources for scientific topics.
Even though the standards of quality vary between different Wikipedia projects (Jemielniak & Wilamowski, 2017), the overall quality of Wikipedia articles is surprisingly high (Michelucci & Dickinson, 2016), even in highly specialized topics (London et al., 2019). As early as in 2005, a study published in Nature described Wikipedia as going “head to head” with Britannica (Giles, 2005). Since then Wikipedia has grown 6-fold in terms of the number of articles and is currently over 85 times the size of 120-volume Encyclopedia Britannica by word count (Wikipedia, 2019). The overall accuracy of Wikipedia is on par with the professional sources (Chesney, 2006; James, 2016), and a systematic overview of academic research on the topic confirms that Wikipedia’s quality and accuracy are high (Mesgari et al., 2015) even though errors of omission for older topics are frequent (Brown, 2011).
In psychology, the topical coverage on Wikipedia was already relatively comprehensive a decade ago (Schweitzer, 2008). In some studies, Wikipedia proved to be as reliable as or more reliable than the Britannica or psychiatry textbooks although it was rated lower for readability (Reavley et al., 2012).
Clearly, even though Academia still treats Wikipedia with suspicion (Aibar et al., 2015; Jemielniak & Aibar, 2016; Konieczny, 2016), its accuracy is already sufficient for most purposes, including higher education, and Wikipedia editing is becoming increasingly more popular as a course assignment in psychology classes (Shane-Simpson et al., 2016). This is possibly due to the call from APA to actively participate in Wikipedia editing (Breckler, 2010) or a similar initiative by the Association for Psychological Science (Banaji, 2011).
These facts make it even more interesting to discover which psychology journals are used on Wikipedia most frequently, and which references are cited in Wikipedia articles on psychology. While the usage of academic journals Wikipedia-wide may be a measure of the journal’s overall popularity and, quite likely, is biased toward general science publications, as well as toward journals with a popularizing mission, the references in articles specifically covering psychological concepts may be a good reflection of what publications are perceived as best addressing them.
Research Questions
The aim of our study was to check which psychology journals are most often cited in Wikipedia (1). Also, we wanted to know which academic journals are frequently referenced in Wikipedia articles on psychology topics (2). Additionally, we were interested in understanding the similarities and differences of such ranking of popularity, when compared to a standard journal ranking measures SJR. We also wanted to observe patterns related to the frequency of citations, the trajectory of citations in time, distribution of publication dates, and the time between the publication date of an article and the reference created in Wikipedia.
Method
We conducted a two-stage study. First, we checked which psychology journals are referred to in the entire English Wikipedia. To do that, we processed the full repository of Wikipedia articles with a self-developed script and cross-referenced it with a SciMago database of academic journals in the field of psychology, that is, 1,360 titles of academic journals classified as psychology journals. It should be noted that we relied on SciMago classification rigorously: In other words, we avoided any judgment calls. As a result, for example, some psychiatry journals have been included, as they are classified also as psychology in SciMago, and some have not. We consider this to be a feature rather than a flaw in our design. We looked for DOI and ISSN matches and also processed journal names if these identifiers were missing in Wikipedia. This way, we obtained a list of journal titles with their number of citations in Wikipedia.
Second, we checked which academic journals are referred to in Wikipedia in articles specifically related to psychology. For that purpose, we used 1,763 entries listed in the “index of psychology articles,” curated by WikiProject Psychology, a group of volunteer Wikipedians working in the area of psychology. The list is a set of psychology topics, prechosen by the Wikipedia community as being the core of knowledge about psychology. We then scraped the data containing the number of references to the journals listed in these articles.
For a comparison to existing journal citation metrics, we decided to use SJR (developed by Elsevier) for several reasons. First, even though both SJR and JIF (impact factor based on Journal Citation Reports [JCR], curated by Clarivate) rely on the number of citations to a given journal, SciMago is a database that contains roughly 3 times more titles than JCR, and we wanted to cover a larger data set of academic journals. Second, SJR is a free database, while JCR requires a subscription. Additionally, it is normalized within disciplines and thus allows cross-disciplinary comparisons, while JCR is not. Finally, there is research suggesting that SJR may be a better overall proxy of academic impact (Bollen et al., 2009). It was developed specifically to address some criticism that JIF received (Butler, 2008) and incorporates a measure of “inheriting prestige,” that is, citations in higher ranked journals are worth more than in the ones ranked lower, similarly to Google PageRank.
Results
In the entire body of English Wikipedia, we found 35,609 references to 357 psychology journals. The list of the 30 most cited psychology journals in the whole collection of English Wikipedia articles is presented in Table 1. The most frequently cited psychology journals are the Journal of Personality and Social Psychology, American Psychologist, and Psychological Bulletin. The three most cited journals make up 11% of all the references to psychology journals in Wikipedia, whereas the 10 most cited journals constitute as much as 24% of all the references. Quite clearly, journals that operate on a meta-level, that address the topics in the psychology field interesting predominantly to academic psychologists, are less popular.
Psychological Journals Most Cited in Wikipedia.
Since psychology journals might potentially be used as a source of knowledge about topics that are not necessarily strictly psychology-related, we then explored references to any journals in Wikipedia articles that had been classified as psychology-themed. The articles included 7,848 references to 360 journals. The rank of the 30 most often cited journals is presented in Table 2. The top three journals referenced in psychology articles in Wikipedia are the Journal of Personality and Social Psychology, SSRN Electronic Journal, and Psychological Bulletin. The three most frequently referenced journals within the psychology-themed articles constitute 13% of all the references within this article collection, whereas the top 10 cover 28% of all the references. In both of the rankings, the most cited journal remains the Journal of Personality and Social Psychology, with 1977 references on all articles on Wikipedia, and 463 when only articles categorized as psychology-themed are taken into account. We see that many well-established psychology journals may be referred to in encyclopedic entries on different aspects of human behavior, not necessarily strictly pertaining to psychology.
An absolute number of citations is a measure of a journal’s prominence, but it does not account for an average impact of one article published in it—after all, mega-journals, publishing thousands of articles per year, could be reflected more often just because of the sheer number of publications, even if their average quality was lower than in the journals publishing just a dozen articles each year. In order to address that issue, we calculated the ranking of journals weighted by the number of articles published per year (including only journals with at least 10 publications over the last three years, to eliminate artifacts). The purpose of this approach was to see whether the journals publishing thousands of articles per year do not dominate the ranking because of the sheer quantity rather than the pure quality of publications.
The most frequently cited psychology journals, when they are weighted by the annual number of articles published, are Psychological Review, Psychological Bulletin, and Journal of Personality and Social Psychology. The differences in our ranking after adjusting for the number of publications per year are largely dependent on the way weights are applied. In our approach, with a linear number of publications used, they definitely provide an additional, nonreducible point of view. We believe that this analysis adds an important perspective on evaluating the journal’s impact.
We prepared a similar weighted analysis for journals referenced in psychology-themed Wikipedia articles. The top three journals according to this criterion are Psychological Bulletin, Psychological Review, and Annual Review of Psychology. This additional analysis eliminated SSRN Electronic Journal from the top of the ranking, which we consider a valuable re-adjustment. Interestingly, taking the number of articles published per annum also significantly lowered the rank for journals such as Science, Cochrane Database of Systematic Reviews, PNAS, Nature, PLos ONE, or BMJ, which publish large numbers of articles. We interpret this result as indicating that the weighting process proves useful and adds an important perspective.
Journals Most Cited in Wikipedia Articles Classified as Psychology-Themed.
a SSRN Electronic Journal is a nonreviewed paper repository.
Coverage
It is striking that from among 1,350 psychology journals, only 357 are cited at least once in the entire Wikipedia. This shows that editors creating content in the world’s most popular encyclopedia refer to only about 26% of content-specific, academic resources available. Additionally, it is very surprising that open access journals do not make it to the top 30, with the notable exception of Frontiers in Psychology in the group of psychology journals most cited on Wikipedia as a whole, and SSRN Electronic Journal in the group of academic journals most referred to in Wikipedia entries classified as psychology-themed. In a study of medical journals on Wikipedia (Jemielniak et al., 2019), the open-access effect was more noticeable and explained by Wikipedia editors’ preference for open knowledge. Clearly, this preference is not predominant for editors who add information on psychology to Wikipedia—possibly due to the articles being more widely available on academic sharing websites, such as Researchgate.net or Academia.edu. The high position of SSRN Electronic Journal is interesting, and it may denote that Wikipedia editors do not recognize that it is a preprint, nonreviewed paper repository. However, as our additional analysis of citations after weighting the annual number of articles published by a journal shows, SSRN Electronic Journal high position in absolute numbers ranking is largely resulting from its massive number of non-peer-reviewed papers stored in the repository each year.
Nature, Science, PNAS, and PLos ONE are general focus mega-journals, and it is not surprising that they are top-cited in psychology-themed articles in absolute numbers. Their position is largely a result of the number of publications per year, as we can observe in the weighted ranking.
Correlation Between SJR and Wikipedia Citation Rate
In order to check whether there is a correlation between the number of citations the psychology journals get in Wikipedia and the journals’ SJR, we ran a correlation test in SPSS version 25. Results of the Pearson correlation indicated that there is a significant positive association between SJR measured by 2017 SciMago Ranking and Wikipedia citation rate, r = .436, p < .001, n = 285.
Interdisciplinary Character
Since it might be expected that journals that tackle a variety of broad topics may generate more citations in a source aimed at the general public than journals with narrow scope related to very specific subjects, we looked at how interdisciplinary character of the journal is aligned with the Wikipedia citation rank—or, in other words, to see whether being highly cited in the field of psychology is a good proxy of being highly cited in articles on Wikipedia from other disciplines. This is presented in Figure 1, where the value rank of the journals most frequently cited in Wikipedia are plotted, with the x-axis representing the rank based on the number of citations (the lowest rank represents the highest number of citations) of the journal within Wikipedia psychology-themed articles and the y-axis represents the rank values of all academic journals based on their citations in Wikipedia.
Correlation between SJR and Wikipedia citation rate.
As may be observed in Figure 2, the references in the two categories have a similar trajectory. This means that journals that are well-cited across psychology-related Wikipedia entries are also generally well-cited across the whole Wikipedia.
Wikipedia citation rank.
Based on the Wikipedia citation rank values, we have calculated the rank difference by subtracting the value of Wikipedia citation rank of journals which were referred to in psychology articles on Wikipedia from the value of the general Wikipedia citation rank, that is, the rank based on all citations of a journal in Wikipedia. If the difference is negative, the larger the negative values, the larger the relative interdisciplinary character of the journal. If the difference is positive, the larger the difference value, the more psychology-themed the journal. The Wikipedia citation rank differences of the top 70 academic journals are presented in Figure 3.
Journals’ Wikipedia citation rank differences.
Change in Citations in Time
We then traced the number of citations of the top 15 journals in Wikipedia in time. The number of citations in Wikipedia psychology-themed entries is shown in Figure 4, whereas the number of citations to psychology journals in all Wikipedia entries is shown in Figure 5.
The number of citations to journals in Wikipedia psychology-themed articles.
The number of citations to psychology journals in all Wikipedia articles.
Interestingly, the trajectories of the journals both for all-Wikipedia and for psychology-articles-only are relatively similar. We observe increasing value with time, and the incline seems steeper for a number of journals in 2011–2012.
The trajectory of the Journal of Personality and Social Psychology is steeper than the one of any other journal. The arising question is whether this might be explained by any changes within the journal productivity such as the onset of a higher number of articles published shortly before the trend started. This does not seem to be the case: the journal has been steadily producing six issues a year, with a similar number of articles each year. Another explanation might be the circularity of the citation process. Journals that are often referenced get more citations since the references are often the sources of information (Ball, 2002). This could potentially clarify the effect of cumulative advantage observed here for Journal of Personality and Social Psychology. We believe that the spike in Journal of Personality and Social Psychology may to some extent be related to the replication crisis in psychology (Stroebe & Strack, 2014), and the fact that a number of articles, which have had their replicability questioned, have been published in that journal. Speaking more broadly, given the fact that, arguably, the replication crisis has affected social psychology the most, it is not surprising that Journal of Personality and Social Psychology, the flagship journal of the field, has received more attention over the last years.
We ran additional analyses for journals weighted by the number of articles published per annum. The time analysis for the number of citations to journals in Wikipedia psychology-themed articles weighed by the number of articles published per annum is presented in Figure 6, and the number of citations to psychology journals weighted by the number of articles published per annum in all Wikipedia articles is shown in Figure 7. The Journal of Personality and Social Psychology trajectory is less prominent when we adjust for the number of articles per annum although it is still quite steep. In both of our weighted rankings, two journals are strong outliers: Psychological Review and Psychological Bulletin. This result indicates that the articles published in them are considered to be of top quality and relevance universally in psychology-themed articles and among psychology journals cited in all Wikipedia articles.
The number of citations to journals in Wikipedia psychology-themed articles weighed by the number of articles published per annum.
The number of citations to psychology journals weighted by the number of articles published per annum in all Wikipedia articles.
Trajectories of Transfer From Academic Journals to Wikipedia
Our next step was to check how much time passes between the publication of an article and its citation on Wikipedia. This is illustrated in frequency charts in Figure 6.
There is variability across journals in the time needed for an academic article to be cited in Wikipedia (see Figure 8). Whereas texts from Archives from Sexual Behavior, Psychological Science, or Personality and Individual Differences make their way into Wikipedia relatively quickly, with the number of citations decreasing with time, articles from other journals might either increase in citations over time (Journal of Personality and Social Psychology), have a steady citation rate (American Psychologist), or show a somewhat irregular pattern. Across the sample of the nine most cited journals, it takes on average 6,176 days for an article to be referenced in a Wiki article (SD = 5,925), including works published before Wikipedia started. We explain the speed of the propagation of research from Sexual Behavior, Psychological Science, or Personality and Individual Differences by the topic coverage relatively more interesting for lay readers. This result indicates that article readership and coverage is depending on the popularity of the topic covered, which is not surprising, but at the same time serves as an important reminder that scientific excellence should not be equaled with citations only.
The time between the publication of articles in academic journals and their citation on Wikipedia.
The distribution of the number of journal references within one Wikipedia article, depicted in Figure 9, is highly asymmetric (M = 16.9, SD = 23, min = 1, max = 300).
The number of articles (y-axis) with the number of academic articles referred (x-axis). Note. For readability, we truncated the figure. Articles containing more than 100 journal references were cut (1.6% of all analyzed Wikipedia articles).
Limitations
One important limitation our study is prone to is a lack of distinguishing between affirming and contesting character of references. A publication that is highly criticized receives a lot of citations even though they are not indicative of scientific quality. It is a flaw that is currently inherent to all citation metrics, including JCR (impact factor) and SJR, and one that cannot be easily resolved in a large data set computational study. We argue that while the problem is definitely real on the level of a specific article (or a person), it does not persist on the level of a journal. In other words, while it is possible that a single person writes an academic article that receives a lot of citations as a result of criticism, it is highly unlikely that there is a journal which consistently receives a lot of attention and criticism from other reputable journals because of publishing questionable results. However, this limitation should be kept in mind.
Our study relies on journals in the field of psychology. While they are valid within this discipline, we do not know whether similar robust results are observable in other fields.
Discussion
Our aim was to check which psychology journals are most often cited in Wikipedia and which academic journals are most frequently referenced in psychology articles of Wikipedia. We were surprised to find that only 357 of 1,350 are cited at least once in the entire Wikipedia. An interesting finding was that an effect of accumulative advantage may be observed in Wikipedia citation rate, that is, there is a pattern that journals that were cited a lot, get even more citations in time, whereas journals that only have a few citations grow at a very limited pace. This might be explained by the potential practice of people referring to sources that already have their citations in Wikipedia and using a reference list in a Wikipedia article as a guide for their further study, which later results in another Wikipedia article. Another possible explanation, requiring further research, is that there may be a core group of Wikipedia editors developing articles on psychology, who have strong preferences or default to the journals they already know.
Importantly, we have found a significant correlation between the Wikipedia citation rate of psychology journals and the journals’ SJR score. This is not surprising. However, while both SJR score and accessibility (open access) have been shown to be strong predictors of an article being referenced on Wikipedia (Teplitskiy et al., 2017), the latter (open access) has not proven true for psychology journals. It is also worth noting that the journals most cited in Wikipedia are APA or APA-affiliated journals (all top five journals according to our ranks, apart from SSRN Electronic Journal). This may be related to the general trust of the community to the organization which is known for producing and disseminating knowledge about psychology. We interpret the citability also as a function of the popularity of a given topic. For instance, online news about social psychology is generally more popular than other psychology topics (McPhetres, 2019). This may partly additionally explain the high position of the Journal of Personality and Social Psychology in the ranks.
We have found that journals that are well-cited across psychology-related Wikipedia entries are also generally well-cited across the whole of Wikipedia. This is an interesting finding, arguing against the expectation that journals with a narrowed scope would be cited more often in psychology articles, but not across the whole Wikipedia, whereas journals whose scope is broader, hence the topics covered there are of more interdisciplinary nature, would not be cited as often in psychology journals as they would across the whole Wikipedia. It is possible that the circularity of referencing may play a role in this result.
We were also interested in the pattern of citations of particular journals, as well as the trajectories that information has from the publication date of an academic journal to the citation of the article in Wikipedia. We found that the patterns vary across the most cited journals. It may be expected that the access policies of a particular journal may contribute to the creation of the patterns to some extent, but other interpretations are possible. One interpretation is that, again, the specific content may generate differential public interest, which in turn influences the speed of transferring the research results to texts popularizing them. This hypothesis aligns with the trajectories of Archives of Sexual Behavior—a topic that is, possibly, of broad public interest and that is tracked and described soon after the publication of the original research report.
Studying the aggregated data to see which journals are most effective at reaching the general public may be used as an interesting supplementary way of evaluating journals’ actual impact on knowledge dissemination in society. Online mentions of academic works show positive medium and high correlations to more traditional metrics such as impact factor when measured on the level of psychology as a whole field (Vogl et al., 2018). We propose that the citation rate on Wikipedia, next to the traditional citation index, may be a good indicator of the work’s impact.
We propose that future research directions may include studying different disciplines’ citations on Wikipedia, so as to determine the interdisciplinary variance. Additionally, we believe it would be useful to distinguish positive and negative citations when it is technologically viable, to increase the granularity of the results. One more possible path for future studies is discovering the average time between the publication date and the citation varying across disciplines. Finally, while difficult, we also believe it would be worthy to look for measures of a given article being accessible through university repositories or academic social networking websites such as Academia.edu or Researchgate.net, as it affects its possible reach and citations on Wikipedia.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a research Grant from a research Grant Number PPN/BEK/2019/1/00233/U/00001 from the Polish National Agency for Academic Exchange.