Research on differential and interactive impact of China-led and US-led open-access articles

Abstract

With the development of Web 2.0, social media dialogue has been increasingly important within the world of open access (OA), striving for more user-generated content and ease of use. In this article, we analysed the impact of OA articles published by both Chinese and the American researchers using PLOS ONE. Papers published in the same year, using citation and social media metrics, were all used to analyse the correlation between the level of social media metrics and citation. Overall, the impact of OA articles published within the United States is higher than OA articles published in China. The results showed that citations and number of Mendeley readers have a significant correlation, which reflect the similar impact in evaluation of OA articles. However, most social media metrics did not have an obvious correlation with impact evaluation, which indicates the social media metrics are useful when paired with citations, but not irreplaceable to citations. Social media metrics appear to be a useful alternative metrics to accurately reflecting the impact of OA articles within the scientific community.

Keywords

Altmetrics impact evaluation open-access article social media metrics

1. Introduction

Having open access (OA) to scientific publications has created a ‘movement’ of influence within a particular niche market of the distribution and sale of peer-reviewed scholarly articles today. The OA movement objective is in favour of the distributing of peer-reviewed literature freely available to the public through the Internet. The success of OA greatly depends on authors supporting the principle of OA using an open platform to circulate their academic successes. With the development of the modern Internet and communication, the interaction between scholars is growing more frequently and easily across the globe. More and more academic articles have shifted from hard copy prints to online archives, which liberated scholars from the physical library and lifted the potential barrier of access to those online articles. Scholars’ ability to obtain information and process information is gradually improved further with the development of Web 2.0, social media activities have become an alternative OA to academic exchange. OA indicates that scholars publish their work online, so that anyone who is interested in downloading and reading their work does not meet any financial or technical barriers, or fear of being ‘priced out’ or hitting a ‘pay-wall’. The OA not only aims to reform traditional publication systems but also aims to increase academic propagation. Scholars themselves are by large no longer satisfied with traditional knowledge dissemination, instead support a system of immediate academic exchange. Making full use of the existing knowledge has become the major focus in the field of OA.

The development of OA is a main concern from those within the scientific community around the world, and social media have helped supplement and shape OA. However, to use social media to evaluate academic resources becomes a difficult task with the spread of information even though social media play a large part. OA promotes international collaboration and author collaboration, often arising from informal communication between researchers [1 –6]. The research of impact originated in sociology and is considered to be the ability to change the minds and behaviours of others. Scholars in the field of communication regard ‘impact’ as the evaluation of the controlling effect of information on audiences’ psychology, cognition, tendency, emotion, attitude and behaviour. Scholars in the field of library and information science – based on a number of traits such as characteristics of academic papers and circulation of academic information – apply impact evaluation to papers, reflecting the role of papers in readers’ theoretical expansion as well as any practical applications. Xianwen Wang and colleagues indicated that impact can be divided into short-term and long-term impact, meaning research is not limited to the academic community, but its impact on society in general as well as professional practice, an important part of academic impact [7]. Journal impact factor is not equal to journal impact, nor is the impact of academic papers equal to the citation frequency, yet this idea is all too prevalent. It is important to note that the impact of academic papers also includes the social, academic, economic, application practice and so on.

The nature of the process of mutual impact between the leader and the follower, the resulting results, and how the personality behaviour of the leader, the cognition of the followers, the leader’s credit and the environment determine the pair of processes [8]. Scientific research cooperation is a scientific social process, and scientific research cooperation is a process in which researchers or scientific research groups act together for a common goal, and appropriate leadership is required in this process. Leadership in scientific research cooperation has the characteristics of general organisation cooperation but also has its particularity. Scientific research cooperation is an Organisational Behaviour. From the perspective of Organisational Behaviour, scientific research cooperation needs appropriate leadership [9]. Organisational Behaviour is related to leadership, and even leadership determines that Organisational innovation is also related to the scale of cooperation. The bylines in a multi-author paper reflect the degree to which an author contributes to a scientific collaboration. We distinguish the size of an author’s contribution by the order in which the author is signed, so the first author is one of the main contributors. The corresponding author is the main contributor to communication and collaboration, and the author with the main responsibility for the article. This article considers the first author as a leader who has made a major contribution to the paper.

The impact of academic papers depends on the reconstruction of readers’ cognition, by the content of the papers, the innovation of the methods of the papers and whether the research content of the papers is closely related to social needs. There is no unified definition on how to explain the impact of a paper, and the expression form of the impact of an academic paper is mainly explored from research results. Through the feedback of its research results, for example, being recognised by many scholars as feedback of others on the paper. Moreover, in the era of Web 2.0, there are various ways to obtain this feedback. Scholars’ attention, social software mention, document management software mention, database mention, comment, share, likes and reference data all reflect readers’ feedback on the paper. With the increase of OA international cooperation, the impact of OA papers is also increasing, but there are still a few studies on the impact of OA papers led by different countries. In order to further analyse the impact of OA papers led by different countries, this article aims to identify characteristics of impact by looking at two of the world’s leading countries - the United States and China within the scientific community, comparing various ways in which they impact society

2. Background

Social media are online services focusing on creating networks or social relationships that can give users space to participate in such things as blogs, Wikipedia, BBS, SNS, Twitter, Facebook and so on. This enables academic information or academic activities to spread and interact instantaneously. As the scope of scholar’s use of social media gradually expands, more and more scholars use social media for information exchange. With the development of Web 2.0, the rapid development of information technology has completely changed the traditional way of academic information exchange. To some extent, the development of social media has promoted the development of OA, Lukes stated that one should not overlook the potential of social media [10]. Social media are the most preferred medium for young scholars to interact with each other and to share information. Social metrics are one of the main usage metrics; social metrics are often applied in impact evaluation. Altmetrics.com is a tool specialised in analysing the social impact of academic papers on social media, newspapers and magazines. It uses alternative metrics to measure the social impact of academic papers, not traditional citations, but online downloads, social media mentions and online library collections to evaluate the impact of a paper.

Over the past 20 years, OA advocacy has inspired the modes of many stakeholders involved in this movement, including golden OA and green OA. From the perspective of academic exchange, OA papers may be accessed more frequently; however, traditional impact factors measure citations rather than social media attention. Using impact factor evaluations from sources such as Journal Citation Reports, moreover scientific journals with the highest impact factors are not OA. Social media can be an effective alternative indicator to measuring the impact of OA papers. The rapidly growing popularity of social media as a place for authors to post comments or viewpoint of their academic achievements was unexpected. It appears social media, through their general appeal and other services, as well as the ease of uploading illegal versions of the publishers’ articles, are starting to become more attractive than the more strictly curated institutional repositories.

Social media and OA cannot be separated. The impact of OA papers can be increased by social media activities, such as browsing, downloading and sharing which are all generated by social media. Steve Lawrence indicates that OA enables researchers to continuously analyse the OA papers with the usage of metrics, altmetrics and citations [11]. When OA is combined with social media, the impact of research papers shifts from publications to individual researchers. For example, Gunther Eysenbach, MD, et al. explored the feasibility of measuring social impact of public attention to scholarly articles by analysing attention in social media [12]. With the development of social metrics, alternatives to peer reviews are emerging; the most commonly employed indicators are based on comment crowdsourcing. Anonymous peer review is replaced with public reviews that can include the reviewer’s reputation to weigh the review score, such as h-index and altmetrics. Under an OA environment, scholars are actively participating to improve evaluation system, OA methods will be required to publicise research results, and social metrics will undoubtedly play a growing role.

OA is closely related to social media as an important means of spreading scientific knowledge. With the development of social media, scientific papers spread more widely and faster. On social media platforms, people are probably to be the publisher and the receiver of information. The academic application of social media is increasing. Some researchers have studied the academic application of social media platforms and concluded that social media can promote academic exchanges [13,14], expand audiences and even attract readers. Both social media and OA have, to some extent, spread knowledge. The development of OA papers is conducive to the formation of a more decentralised and flat scientific community, where ordinary researchers also have access to scientific research resources. For example, open data mean that teams other than data collection can also analyse and interpret data, which is very rare in the past. This flat structure of the scientific community may be more conducive to the discovery of science, because science itself requires the spirit of equality, criticism and doubt, while science also needs to be based on the predecessors. The OA and the flat structure of the scientific community it brings are very in line with the spirit of science. For young researchers, it also means that more abundant resources can be used and relatively low cost can be used to make research. While social media metrics and citations were used to evaluate impact of academic achievements have been researched earlier. The use of social media metrics to investigate the impact of OA articles led by China and the United States has not been studied. The general objective of the study is to explore differential and interactive impact of OA papers led by China and the United States.

3. Data and methodology

PLOS was founded as a non-profit OA publisher, innovator and advocacy organisation with a mission to advance progress in science and medicine by leading a transformation in research communication. PLOS is one of many currently successful, OA publishing models that provide immediate access to biomedical research articles, which is monitored by strict peer-review standards. It is believed that OPEN is a mind-set that represents the best scientific values, bringing scientists together to share work as rapidly and widely as possible, to advance science faster and to benefit society as a whole. PLOS has been a force for transformation in scholarly publishing, which proved the viability of OA, redefined publishing with PLOS ONE, the world’s largest multidisciplinary peer-reviewed journal. In this article, we conducted a bibliometric analysis of data in PLOS ONE. PLOS ONE is an inclusive journal community working together to advance science for the benefit of society, which is widely disseminated and freely accessible to all. We have collected research output in field of social science from PLOS ONE for the authors from China and the United States. We found a total of 3945 records from China. There are 14,624 records from the United States. The data collection was on 11 January 2020. We tracked the social media metrics from altmetric.com on 11 January 2020; there were 14,597 articles from the United States and 3942 articles from China after retrieval. We retrieved the altmetric data by digital object identifier (DOI) from altmetric.com, which derived from activities and engagement in response to the quest for better metrics for measuring research impact. The social media data were normalised to facilitate further analysis. Finally, we analysed the different impacts of OA papers from the American- and Chinese-led papers.

Visualising network analysis is the process of investigating network structures through the use of network and graph theories [15], which includes social media networks, friendship and acquaintance networks, collaboration graphs, kinship and other relationships [16,17]. We mainly use visualising network analysis to analyse and process data. With the rapid development of visualising map, such as CiteSpace, Pajek and VOSviewer, visualising network has been gradually applied as a key method of hybrid research design to address several important topics in management research, such as knowledge transfer, resource mobilisation and consensus building [18 –20]. In this article, visualising network was introduced in integration of digital library collections. Data processing and analysing tools used in this article include Excel, VOSviewer and SPSS, we used Excel for data processing, and to analyse further using CiteSpace and VOSviewer.

Publishing scientific research papers is an important form of the scientific research results. The signature of scientific research papers is the most direct embodiment of the leadership in scientific research cooperation. The signature of a multi-author paper reflects the degree of contribution of the author in scientific research cooperation. Generally, we distinguish the contribution of the author from the order of the author’s signature, so the first author is the main contributor of the article. This article takes the first author as the standard, that is, the leader who has made a major contribution to the paper, and measures the influence of the articles written by Chinese scholars and American scholars from the perspective of the author’s signature.

4. Results and discussion

4.1. Leadership in scientific research collaboration

Scientific research cooperation is a process in which scientific researchers or scientific research groups take intensive action for a common goal. Appropriate leaders are needed in this process. Scientific research cooperation leadership is closely related to the country’s scientific and technological leadership. Scientific research is important to support and promote a country’s high-quality development and occupies a vital position in the country’s overall strength. The leadership in a certain scientific research cooperation can reflect the country’s strength in this field. A comprehensive understanding of the status of a country and the changing trends is crucial to understanding its state of science and technology, and has positive influence on national scientific and technological investment. On this basis, the mutual relationship between nations, scientific influence is explored, as well as any resulting scientific innovation. Scientific research cooperation leadership is closely related to national scientific and technological leadership. Scientific research is an important support to promote a country’s high-quality development and occupies an important position in the country’s comprehensive strength. The leadership in a certain scientific research cooperation can reflect the country’s strength in this field. Comprehensive understanding of the status of countries in international scientific research cooperation and understanding of this change trend is crucial for understanding the national science and technology situation, and has positive reference value for national science and technology investment [21]. On this basis, to explore the interaction between leadership and scientific research impact can improve the impact of science and technology, accelerate scientific research innovation. In the world economic era, the field of scientific research has been forming a world community, which is mainly reflected in the trend of disciplinary background, cultural background and gender diversity. At different levels of cooperation, the scale of cooperation can be composed of different basic elements and restricted by different factors. Leadership in scientific research cooperation has the characteristics of general organisation cooperation but also has its particularity. Scientific research cooperation is an Organisational Behaviour. From the perspective of Organisational Behaviour, scientific research cooperation needs appropriate leadership, Organisational Behaviour is related to leadership, and even leadership determines that Organisational innovation is also related to the scale of cooperation.

In this new technological era, the phenomenon of a global community has been formed within the scientific field, which is mainly reflected in the trend of discipline background, cultural backgrounds and gender diversity. At different levels of cooperation, the scale can depend on a number of basic factors. Communication between members of the scientific online community costs time and money as any other enterprise, so different projects may involve different scales. An establishment of a core team in the field of physics, for example, can reach more than 1000 people. Often agencies are comprised of more than 100 people, such large-scale cooperation requires unified management and leadership positions to be formed; however, this is often the leading problem of cooperation. Different scholars have different definitions of leadership, and there is no unified position and can vary. This is important as leadership is reflected in the process of influencing others.

The leadership within research collaboration has the characteristics of a single collaboration, while also being shaped and directed on an individual level by each collaborator separately. Generally, the essence of scientific research collaboration is an academic exchange behaviour. From the perspective of academic exchange behaviour, appropriate leaders are needed in scientific research collaboration. The level of leaders’ contribution to collaborative articles can simply be measured by who plays the leading role in the collaboration; the order of authorship in each paper is a good indication of this. This academic collaboration among individuals across countries has been increasing over the past decades. Some studies have indicated that collaboration may increase research productivity [22 –24]. Moreover, associations between collaboration led by different countries and social media attention have also been examined in this article. Co-authorship network analysis has been studied in OA articles led by China and the United States. We analysed the collaboration by VOSviewer, which is a software tool for creating co-authorship based on network data. Figures 1 and 2 reflect the landscape of leadership in OA collaboration led by China and the United States. In the figure, the authors are presented by the label and a circle with different colours. The size of label and circle of each author is determined by the weight of collaboration. The stronger the weight of collaboration, the larger the label and the circle. The colour of the circle is determined by the cluster to the collaboration. Lines between authors represent links – the stronger links with authors, the thicker the line is. The distance between two authors in this figure indicates the relatedness of the authors in terms of collaboration.

Figure 1.

The landscape of scientific research collaboration led by China.

Figure 2.

The landscape of scientific research collaboration led by the United States.

From Figure 1, the co-authorship network of OA articles led by China consists of nodes and links: nodes represent the authors and links connect the authors in the form of co-authorships. There is a link between two authors if they have collaborated on academic research. The size of a node is proportional to the number of co-authorships. The network includes density, clusters, nodes and links. Network density reflects the portion of the potential connections in a network. Clusters are an indicator to reflect which nodes in a network form a close relationship.

Leadership is inseparable from collaboration, which has a wide range of collaboration scope and varied modes. It is necessary to pay attention to the collaborative group and environment in which the leadership is located. It is necessary to theoretically distinguish collaborative leadership (such as strategic leadership) from organisational leadership (such as first author ranking in the paper). Historical research on specific leadership issues, such as leadership and culture, leadership and gender, needs to be emphasised; it is necessary to focus on various states between leaders and partners in specific situations, within a certain period of time, at the collaborative level and at the individual level. The influence of leadership ethics on understanding the nature of leadership is becoming more and more obvious. This view holds that leadership is not about a person or a position, but about a complex academic relationship between scholars. Based on the analysis of leadership by various scholars and several different leadership theories, it can be concluded that leadership is not only about individual ability but also to be able to make use of resources to play an important role as the head of the whole team.

The network of collaboration in the Chinese OA articles is very dense, which indicates OA articles led by China have a frequent collaboration. The cluster visualisation of the network is similar to a small world network. Figure 2 shows the network of OA articles’ collaboration led by the United States. It is easy to find the major player in the network in terms of collaboration relations. The larger nodes are noteworthy and can be considered as a consequence of leading positions in collaborations. As figures show, there are more collaborations within OA articles. The globalisation of collaboration relations at current levels can be expected to have changed the structure of science and the academic exchange. The development of OA articles is more interactive than previous forms, which features more social media ‘openness’ with some regional differentiation than the core periphery grouping that characterised the non-OA in the past.

4.2. The distribution of social media metrics

Social media have been adopted by academic activities in recent years, from personal academic activities to group academic activities, such as blogs, twitter, Facebook and Mendeley. These social media offer new ways for researchers and users to communicate and collaborate. However, with so many different social media sites available – and with more emerging all the time – it is important that social media develop methods for measuring the usage and effectiveness of academic achievements. Since the altmetrics policy came out in 2010, interest in social media metrics has grown. There are more and more academic activities discussed on social media, and it has become relatively easy to evaluate impact by social media metrics. Social media metrics can be seen as motivated by Web 2.0, which is regarded as a new approach to fully explore the potential impact of scientific research. It is an enhancement to traditional citation under the environment of Web 2.0. We selected seven social media metrics in altmetric.com which are: new mentions, blog mentions, Twitter mentions, Facebook mentions, Google mentions, number of Mendeley readers and number of dimension citations.

There are some differences found in articles published in the same year, which were led by the US authors and Chinese authors, as shown in Figure 3. From the figure, it is easy to find that most articles led by American and Chinese authors have different levels of impact. Some articles led by Chinese researcher from news dimensions, Mendeley readers and blog mentions are higher than articles led by authors from the United States, as well as other indicators of OA papers show American published research has a clear advantage over Chinese research.

Figure 3.

The open-access articles of social media metrics led by the United States and China.

Social media metrics rely on alternative sources for measuring the impact of OA articles. Social media metrics are different from citation-based metrics; with the development of Web 2.0, a large amount of scholarly literature is being posted, shared, and commented on through social media platforms. Social media is often regarded as supplement to that of traditional academic communication. On most of social media platforms, it is possible to count mentions of OA articles, such as comments, favourites or retweets. Social media metrics offer the opportunity to not only aggregate the metric of individual indicator but also to reflect the mentions and the level of attention that the research receives throughout these indicators. The emergence of social media metrics based on data from online social media platforms offers the possibility of analysing different impact levels from that of what simply citations can calculate. The impact of OA articles evaluated by social media metrics through interactions between different scholars and audiences are regarded as a more realistic measurement of the attention of the research process. The social media metrics are relevant sources to study the interactions and relationships between academic and social activities, aligning more with what could be termed as the social impact, instead of sources of scientific recognition.

With the development of these evaluation systems, social media as sources of analysis are being taken into account within impact assessment. With the development of information technology, social media can be regarded as typical Web metrics. Social media have previously been used to measure the extent to which articles or ideas are mentioned on the Web [25]. Sérgio Moro and colleagues present social media performance metrics and evaluation of the impact on brand building, which use data mining for predicting the performance metrics of posts published on brands’ Facebook pages [26]. Hans N Rawhouser and colleagues describe the measurement of social impact, which involves four approaches to conceptualising social impact [27]. Kayvan Kousha and Mike Thelwall propose an integrated online impact indicator, which combines a range of online sources into one indicator for impact on the Web [28,29]. The social media metrics based on social media are in its infancy, and many questions have not yet reached an answer. It may be that these social metrics measures are completely different impacts that are not correlated with citation indicator; users are the core elements of social media. Moreover, users may interact online through social media platform differently. The impact of social media is the phenomenon that people change their own thoughts, feelings, attitudes and behaviours because of their interaction with other people’s information in social activities. In academic activities, people’s social interaction will produce a corresponding social impact. Users in the virtual social platform, through the dissemination of content, forwarding information and in other ways, make others identify with themselves, so as to follow their own ideas and behaviours.

Social media have become an indispensable communication tool, transmission medium and information carrier for daily work and academic activities. Academic behaviour determines the impact of social media, but individual impact and social media impact are not the same concept, that is, individual impact and social media impact cannot be seen to be the same thing. Individual ideas, information and communication content are delivered through a social platform between scholars. Therefore, the effect of a user on another user is related to the user’s own impact, which is called individual impact. Individual impact is the ability of users to influence others in a group, which is a measure of the importance of users in social networks, but not all of them represent the impact of social media. The impact of social media not only includes the impact of users but also includes the impact of social media and information resources. Social media metrics have been discussed as a potential source of evidence in research evaluation, particularly in response to the quest for better metrics for measuring research performance. A number of social media platforms provide social media data, which are about social media use, reception and impact. Some of social media metrics are web-based forms of traditional library data; these metrics conceived as traces of the academic exchange of the research process. To analyse social media metrics from the perspective of its total value, the total of social media metrics can indicate the overall level of impact in social media metrics as shown in Figure 4. As presented in Figure 4, it can be found that the articles of overall social media metrics led by the United States are higher than articles led by China.

Figure 4.

The total value of social media metrics of open-access papers led by the United States and China.

OA articles attracted more social media attention; the impact of OA articles shifts from publication to individual researchers. Social media metrics make a contribution to individual researchers, with an advantage of speed, openness and diversity. Traditional citation metrics measure long-time impact rather than readership. When OA articles combined with social media, the impact of OA articles can be measured more accurately. The increasing use of social media metrics in scholarly communication comes with OA, as new indicators instead of citations alone. Citation indicators are regarded as direct indicators to measure the impact of research outputs, and the social media metrics are regarded as indirect indicators to evaluate the impact of research outputs. Social media metrics, dissimilar to citations, although not clearly related to academic performance, it however can reflect the expanded impact of articles. Based on OA movement, social media metrics can be seen as new tools to evaluate research.

As presented in Figure 4, social media values combined with OA articles. As the amount of OA articles increase, the more social media metrics were being used to evaluate the impact of OA articles. Most social media metrics developed a way of evaluating these data points natively with the development of Web 2.0, but some social media metrics do not currently have a practical application in the evaluation of OA articles, such as policy mentions, peer-review mentions and Pinterest mentions. They cannot reflect the real impact of OA articles alone, due to the very limited scope. We selected seven social media metrics to evaluate the impact of OA articles, according to the mentioned coverage ratio of social media. Therefore, these seven social media metrics as scientific evaluative metrics were defined. Compared with the impact of OA articles led by China, the OA articles led by the United States are in top position, which indicates the OA articles by American contributors and authors have a much stronger impact than China.

4.3. The correlation of citation and social media metrics

Eugene Garfield, a founder in the field of bibliometrics, introduced the concept of citation index nearly 60 years [30]. He put forward the impact factors to measure the degree of a journal’s impact by counting the number of times an article is cited. Since performance-promotion is tied to publication, most scholars submit articles to high impact journals, even though acceptance rates are very low. However, several limitations are put forward to question the comprehensiveness of citation measurements. Citations as a metric, which is difficult to obtain and often available in proprietary databases, take time to accumulate, measuring only the type of articles [31], but it also is regarded as an important indicator in scholarly literature evaluations. Most social media metrics can be conceptualised beyond the citation evaluation approach. The correlation between citation and social media metrics is regarded as further research. Social media metrics measure the impact of OA articles by investigating the number of times that it gained attention, debated by the public community and suggested to others for mention. We calculated the distribution is significantly non-normal and positively skewed because the skewness value falls outside the range. We used a one-sample K–S test to determine whether the alternative metric variables are normally distributed. In general, if p < 0.05, then the data are considered to follow an abnormal distribution. The results showed that the p < 0.001 for all variables, therefore, we rejected the normality assumption. After calculated, the p < 0.001, which indicates these social media metrics do not follow a normal distribution. Spearman’s coefficient is calculated as correlation, which is expressed in $ρ$

ρ = \frac{\sum_{i} (x_{i} - \bar{x}) (y_{i} - \bar{y})}{\sqrt{\sum_{i} {(x_{i} - \bar{x})}^{2} \sum_{i} {(y_{i} - \bar{y})}^{2}}}

Table 1 shows the correlation of citation and social media metrics. From the table, the correlation coefficient is significant between citations and number of Mendeley readers, which indicate they have the same impact in evaluating OA articles. Mendeley has an online reference management, which shows the number of times OA articles have been saved, marked or used. The Mendeley includes the data that saved in private libraries. These data are accumulated based on documents, which is similar to the citation. There is no correlation between citations and other social media metrics, which indicates they reflect the differential and interactive impact in evaluation.

Table 1.

Spearman’s correlation coefficient matrix of citation and social media metrics.

			Newsmentions	Blogmentions	Twittermentions	Facebookmentions	Google+ mentions	Number ofMendeleyreaders	Citations
Spearman’srho	News mentions	Correlation coefficient	1.000	0.525**	0.274**	0.328**	0.341**	0.166**	0.085**
	News mentions	Significance (two-tailed)		0.000	0.000	0.000	0.000	0.000	0.000
	Blog mentions	Correlation coefficient	0.525**	1.000	0.266**	0.320**	0.373**	0.229**	0.143**
	Blog mentions	Significance (two-tailed)	0.000		0.000	0.000	0.000	0.000	0.000
	Twitter mentions	Correlation coefficient	0.274**	0.266**	1.000	0.301**	0.258**	0.274**	0.194**
	Twitter mentions	Significance (two-tailed)	0.000	0.000		0.000	0.000	0.000	0.000
	Facebook mentions	Correlation coefficient	0.328**	0.320**	0.301**	1.000	0.318**	0.232**	0.143**
	Facebook mentions	Significance (two-tailed)	0.000	0.000	0.000		0.000	0.000	0.000
	Google+ mentions	Correlation coefficient	0.341**	0.373**	0.258**	0.318**	1.000	0.197**	0.127**
	Google+ mentions	Significance (two-tailed)	0.000	0.000	0.000	0.000		0.000	0.000
	Number of Mendeleyreaders	Correlation coefficient	0.166**	0.229**	0.274**	0.232**	0.197**	1.000	0.729**
	Number of Mendeleyreaders	Significance (two-tailed)	0.000	0.000	0.000	0.000	0.000		0.000
	Citations	Correlation coefficient	0.085**	0.143**	0.194**	0.143**	0.127**	0.729**	1.000
	Citations	Significance (two-tailed)	0.000	0.000	0.000	0.000	0.000	0.000

When the confidence (double test) is 0.01, the correlation is significant.

Citation index and social media index reflect the impact of different leaders by reflecting the use of literature in different ways and the extent to which the information needs of users and the attention of the public are met by the dissemination on online social media and website platforms. Social media indicators pay more attention to the relevance of research objects, and the content that is followed is the content that is interesting to followers and valuable or meaningful to followers. Through the interaction and attention to the same theme, a cluster group is formed. The impact presented by social media indicators is closer to reality, emphasising more participation and interaction, which is in sharp contrast with citation indicators. As a supplement to citation indicators in measuring the impact of papers, social media indicators reflect their impact from the perspective of how papers are used. Citation index is arguably a traditional form of measurement for impact especially with the development of Web 2.0 seeing the development of open resources and social media usage increasing. A trend showing the scientific community moving from traditional communication, choosing a more active online social media presence for communication. With various platforms to choose from more and more scientific researchers begin to use online social networking tools and sites, accessing, sharing, communicating, and evaluating scientific research and academic achievements. Moreover, in response to academic achievements garnered through social networking, tools and platforms have been adapted and used for academia, the impact of social media index has become a new supplementary index for measuring impact and evaluation of academic achievements.

The impact of OA papers is inseparable from the interaction, communication and dissemination of OA academic papers. Only through interaction can impact be generated. In the process of interaction, knowledge increment is generated, and the inheritance of original knowledge and the development of subsequent academic research are also the processes of the impact of OA papers. When papers are open to sharing, the impact reflected through online communication and dissemination is the value of OA papers. The expression of this value can be measured by the number of citations of papers in the interactive process. To a certain extent, the high citation frequency of OA papers reflects its greater impact. This type of impact is generated through the online dissemination of OA papers. The impact of OA paper interaction is an important part of the impact of OA paper. The impact of OA paper is the ability to impact and inspire readers’ academic thoughts and behaviours in the interactive process of paper. The OA of each paper will produce different interactive impacts, but the intensity of impact in the interactive process is different. The impact of OA papers varies with opening times.

In this research, we set out to investigate differential and interactive impact of OA articles by China and the United States based on social media metrics. While the impact of OA articles seemed to have a connection to social media and citations with academic exchange, the impact for these OA articles is unclear. Many OA articles are actively on social media and thus receive more online attention. We could find that these online attentions accumulated in shorter time than citations. Especially, the OA articles contributed to increased social media attention. There are several limitations to this research. The main limitation of this study is that we only study the OA articles by China and the United Sates. One result of this limitation is that the research on only OA articles led by China and the United Sates limits the number of social media metrics studied. Despite these limitations, our research studies impact those of Chinese and American leadership in OA articles which remain a large portion of published articles through OA. There are many related research works on studying impact of OA articles from social media metrics and citations [32 –34]. Our results also provide a foundation for further study on different leadership of academic achievements. The social media metrics can be integrated with other bibliometric indicators such as collaboration indicators [35], h-index [36], and with socialisation data [37].

In an effort to discover higher impact of OA articles from China and the United Sates which received social media attention, we found that citations and number of Mendeley readers have a significant correlation, which reflect the similar impact in evaluation of OA articles. Most of social media metrics do not have an obvious correlation with impact evaluation, which indicates the social media metrics are useful when paired with citations, but not irreplaceable. However, social media metrics appear to be a useful alternative metrics to reflect the impact of OA articles within the scientific community. Based on our findings, we can conclude that OA articles led by the United Sates obtained more social media attentions than China from overall level; however, this social media attention is different to academic exchange. Some OA articles published by the US researchers have obvious advantages in social media attention, which indicates that the OA articles led by China have a relatively low impact with this approach.

5. Conclusion

This study is subject to OA articles led by China and the United Sates in research impact from the perspective of social media metrics. Citation-based metrics are slow and narrow in an increasingly fast and broad scholarly world. Social media metrics and article usage, valuable though they are, remain tied to Web 1.0 ways of thinking in an era of increasingly fast-moving, frequent exchange. In social media activities, citation analysis is also an important indicator that cannot be ignored. With the development of OA, the application of social media indicators is increasing in academic field. Web-based metrics embrace the move to new platforms for scholarly practice as a source for this broader array, and present a new way to improve impact evaluation. Web-based metrics have the ability to evaluate and predict impact on social media; with the scholarly landscape changing, new opportunities are presented for our metrics and transforming scholarly measures of impact as a whole. Future studies should explore social media indicators in research evaluation.

In this study, we have researched differential and interactive impact of OA articles led by China and the United Sates with different social media metrics and citation count. The result shows that the impact of OA articles led by the United Sates is higher than that of Chinese leaders of research. Our findings provide an important direction regarding the OA articles led by China and the United Sates. There is a big difference between Twitter mentions and the number of Mendeley readers. Differences also being found among news mentions, blog mentions, Facebook mentions and Google+ mentions. The impact of citation count to evaluate OA articles often underestimated the gap between Chinese and American leaders’ OA articles. OA articles should pay more attention to social media and from active alternative metrics.

In summary, social media metrics are socially constructed in impact evaluation. Unlike the citation count, the social media metrics have access to data points and are accumulated in a short period of time, regarded as immediate impact. The social media metrics are constructs which offer a window to evaluate wider impact. While bibliometric evaluations may be poor in registering immediate performance, they are indispensable in impact evaluation. With the diversified development of social media, the measurement of the impact of OA articles is more complex, diverse and dynamic, and its evaluation needs to be further improved.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This paper was supported by the Advanced Talents Incubation Program of Hebei University in China (grant no. 521000981329).

ORCID iD

Wei Mingkun

References

Price

Beaver

. Collaboration in an invisible college. American Psychologist 1966; 21(11): 1011–1018.

Press

. The scientific community. Physics Bulletin 1973; 24(9): 551.

Edge

. Quantitative measures of communication in science: a critical review. History of Science 1979; 17(36 Pt.2): 102–134.

Katz

Martin

. What is research collaboration? Research Policy 1997; 26(1): 1–18.

Bozeman

Corley

. Scientists’ collaboration strategies: implications for scientific and technical human capital. Research Policy 2004; 33(4): 599–616.

Tijssen

RJW

. Measuring and evaluating science – technology connections and interactions. In: Glänzel

Schmoch

Moed

(eds) Handbook of quantitative science & technology research, 2004, pp. 695–715New York: Springer.

Wang

Fang

Guo

. Tracking the digital footprints to scholarly articles from social media. Scientometrics 2016; 109(2): 1365–1376.

Barker

. The nature of leadership. Human Relations 2001; 54(4): 469–494.

Brand

Allen

Altman

et al. Beyond authorship: attribution, contribution, collaboration, and credit. Learned Publishing 2015; 28(2): 151–155.

10.

Lukes

. Social media. AAOHN J 2010; 58(10): 415–417.

11.

Lawrence

. Online or invisible. Nature 2001; 411(6837): 521.

12.

Eysenbach

. Correction: can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. J Med Internet Res 2011; 13(4): e123.

13.

Kim

Park

. Structural effects of R&D collaboration network on knowledge diffusion performance. Expert Syst Appl 2009; 36(5): 8986–8992.

14.

Min

. Scale-free network provides an optimal pattern for knowledge transfer. Physica A-Stat Mech Appl 2010; 389(3): 473–480.

15.

Wasserman

Faust

. Social network analysis: Methods and applications. Contemp Sociol 1994; 91(435): 219–220.

16.

Reis

PCA

. Social network analysis in telecommunications. Hoboken, NJ: John Wiley & Sons, 2011.

17.

Andrea

Ferri

Grifoni

. An overview of methods for virtual social networks analysis. London: Springer, 2010.

18.

Carlsson

. Network governance of the commons. Int J Common 2008; 2(1): 36–39.

19.

Bodin Crona

ÖBI

. The role of social networks in natural resource governance: what relational patterns make a difference? Global Environ Change 2009; 19(3): 366–374.

20.

Newig

Günther

Pahl-Wostl

. Neurons in the network learning in governance networks in the context of environmental management. Ecol Soc 2010; 19(1): 50–55.

21.

Ogbonna

Harris

. Leadership style, organizational culture and performance: empirical evidence from UK companies. Int J Human Resource Manag 2000; 11(4): 766–788.

22.

Barjak

Robinson

. International collaboration, mobility and team diversity in the life sciences: Impact on research performance. Social Geography 2008; 3(1): 23–36.

23.

Huang

Lin

. International collaboration and counting inflation in the assessment of national research productivity. Proc Am Soc Inform Sci Technol 2010; 47(1): 1–4.

24.

Sarwar

Hassan

. A bibliometric assessment of scientific productivity and international collaboration of the Islamic World in science and technology (S&T) areas. Scientometrics 2015; 105(2): 1059–1077.

25.

Maurice

Vergeer

. Introduction to webometrics: quantitative web research for the social sciences. J Inform Technol Politic 2013; 10(2): 241–243.

26.

Moro

Rita

Vala

et al. Predicting social media performance metrics and evaluation of the impact on brand building: a data mining approach. J Business Res 2016; 69(9): 3341–3351.

27.

Rawhouser

Cummings

Newbert

. Social impact measurement: Current approaches and future directions for social entrepreneurship research. Social Sci Electron Publish 2019; 43(1): 82–115.

28.

Kousha

Thelwall

. Google book search: citation analysis for social science and the humanities. J Am Soc Inform Sci Tech 2009; 60: 1537–1549.

29.

Thelwall

Kousha

. Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals. J Am Soc Inform Sci Tech 2008; 59(5): 805–815.

30.

Mccain

. The web of knowledge: a festschrift in honor of Eugene Garfield. Libr Inform Sci Res 2001; 71(3): 427–429.

31.

Brody

. Earlier web usage statistics as predictors of later citation impact. J Am Soc Inform Sci Tech 2006; 8(57): 1060-1072.

32.

Wang

Liu

Mao

et al. The open access advantage considering citation, article usage and social media attention. Scientometrics 2015; 103(2): 555–564.

33.

Sullo

. Open access papers have a greater citation advantage in the author-pays model compared to toll access papers in Springer and Elsevier open access journals. Evidence Based Libr Inform Pract 2016; 11(1): 60.

34.

Koler-Povh

Južnič

Turk

. Impact of open access on citation of scholarly publications in the field of civil engineering. Scientometrics 2014; 98(2): 1033–1045.

35.

Rodrigues

Nimrichter

Cordero

RJB

. The benefits of scientific mobility and international collaboration. Fems Microbiology Letters 2017: w247.

36.

Olmedo

Hernández

Pla

et al. Applicability of h-index in humanities and social sciences achievements evaluation. Libr Inform Serv 2013; 62(12): 299–307.

37.

Menon

. Information spillovers and semicollaborative networks in insurer fraud detection. MIS Quart 2018; 42: 407–426.