Metrics employed in the evaluation of research productivity: A systematic literature review

Abstract

The present paper makes a systematic review of 131 articles on research productivity basing upon a model developed by Denyer and Tranfield. The current review paper analyzed the quantitative and qualitative dimensions of the sample literature in the field of research productivity. The quantitative analysis incorporates the chronological distribution of articles, keyword wise distribution, form wise distribution, discipline wise distribution, publisher wise distribution, data sources used in the articles, most prolific journals of the publications and qualitative dimensions studied are top-cited articles and all the metrices used in these articles. The paper is based upon literature retrieved from Scopus bibliographic database and other online databases like Emerald, Taylor & Francis, Google Scholar, JSTOR, ProQuest, and EBSCO for full-text articles. It was revealed that there is significant growth in the number of articles during the fourth decade (2011–2020) whereas the highest numbers of citations (1530) have been received during the third decade from 2001 to 2010. Journal articles appeared to be the predominant source of information. A wide range of metrices found to be used in the sample literature to analyze the research productivity namely basic metrics, metrics dealing with growth, collaboration and metrics at institute level and author level metrics. It will serve as a reference tool to the scholars and practitioners in the field who will be acquainted with the publications on research productivity as well as various metrices, laws of bibliometrics, statistical tools, used in the articles.

Keywords

Bibliometric laws highly cited articles metrics quantitative analysis qualitative analysis research productivity

Introduction

Academic institutions are characterized by three major parameters namely: research, learning, and teaching. Out of these three, research stands as a significant medium for development (Ayeni and Adetoro, 2017). The intellectual outcome of the research is published in the form of research papers namely journal articles, conference papers, short communications, survey reports, etc. The research activity of an individual or an institution is reflected through their published literature. Research productivity refers to innovative thoughts and ideas which after theoretical and applied studies, lead to the publication of articles in leading journals, patent registration, or documentation (Hedjazi and Behravan, 2011). In the last decade, there is an increasing emphasis on the research assessment of government as well as private institutes which resulted in the emergence of some ranking systems at national and international levels. At present, some of the global ranking systems like The Times Higher Education World University Ranking, QS World University Ranking, Academic Ranking of World Universities, and Shanghai Ranking are gaining popularity and at the national level, NIRF (National Institutional of Ranking Framework) by MHRD and QS India University Ranking are the leading ranking systems. In this context, research and publications are declared as one key factor for ranking by MHRD, Government of India. This parameter is described by NIRF as research and professional practice (RP) which includes the combined metric for publications (PU) and quality of publication (QP). Thus, the next big challenge is analyzing the productivity of such institutes. Eliezer Geisler stated that the measurements of science and technology can take a large variety of forms and make use of different types of metrics, each one addressing a particular facet of the system’s complexity (Geisler and Abdallah, 2000). The various metrics such as “Bibliometrics,” “Scientometrics,” “Informetrics,” “Webometrics,” “Netometrics,” “Cybermetrics” are the metrologies for measurement of scientific communications and are sometimes used interchangeably.

Problem statement

An in-depth examination of the published literature showed that numerous studies have been conducted for evaluating the research productivity of various journals, prominent persons, subject areas, departments, institutions, regions of a country, number of countries, or the world as a whole. The studies on research productivity cover diverse perspectives on research output namely some focus on research performance using various metrics, some studies aim at exploring the correlation between productivity and collaboration (He et al., 2009; Hollis, 2001), some studies analyze how various academic, demographic and psychological factors influence research productivity (Abbott and Doucouliagos, 2004; Anyaogu and Iyabo, 2014), some of the papers emphasize on the impact of academic programs like conferences and seminars on the productivity (Agboola and Oduwole, 2005). In this context, the present paper is an attempt to analyze all the literature exclusively dealing with different metrics (indicators, bibliometric laws) used for measuring research productivity by employing the methodology of systematic literature review (SLR).

Rationality of the study

Though the evaluation of research productivity is gaining importance, as a major thrust area of research and a large number of studies are carried out in this field. Still, the current paper which is a systematic literature review on research productivity is the first attempt of its kind as it has included the selected 131 articles which analyzed the research productivity of various institutes, professionals, countries, etc. using a range of metrics. In other words, this SLR is exclusively includes studies of research productivity using various metrics. Thus, the researchers were motivated for conducting such a study and to deal with both the quantitative and qualitative aspects of the published literature. Quantitative aspects cover the growth of studies on research productivity, the forms of publications, disciplines of study, leading publishers, data sources used by the researchers, and leading journals that publish papers on research productivity. As regards the qualitative aspect, the paper highlights a wide range of indicators used by the researchers, highly cited papers and their thrust areas of research.

Methodology

Review work is termed “systematic” when it is based on clearly formulated questions, identifies relevant studies, appraises their quality, and summarizes the evidence by use of explicit methodology (Lagorio et al., 2016). SLR is defined as (Webster and Watson, 2002) “a process in which the main purpose is to overcome the drawback of the traditional literature review method by reducing the bias. It ensures that the reviewed documents are toward the objectivity of the focus area and provide transparency based on retrieval criteria.” Again an SLR gives a concept-centric approach (Senivongse et al., 2017). The present paper is based on the protocol as depicted in Figure 1 for the systematic literature review which is influenced by the model designed by Denyer and Tranfield (2009).

Figure 1.

Protocol for systematic literature review.

Framing of research questions

Based on the first step of the Denyer & Tranfield model, framing the research question is the first step toward carrying out the research in the right direction. So keeping the objectives of the research work, the following research questions have been formulated. Research questions 1–5 address the quantitative aspects of the published literature and gives insightful facts about the characteristics of the articles on research productivity and research question 6 and 7 deal with the range of indicators used in the literature.

RQ1: What is the chronological distribution and citation impact of articles on research productivity? What are the alternative keywords used by researchers and what is the form-wise distribution of publications?

RQ2: What are the disciplines of study in which works on research productivity have been carried out?

RQ 3: Who are the leading publishers in research productivity study?

RQ 4: What are the data sources that have been used by the researchers for articles on research productivity?

RQ 5: Which are the top Impact Factor (IF) journals as regards publications on research productivity?

RQ6: Which are the most cited papers and what are the thrust areas of research of those citing articles?

RQ7: What are the different metrics/bibliometric laws and statistical tools used by the researchers to assess the research productivity?

Forming of search terms and searching articles

The search terms were constructed with the help of Boolean Operator “OR” that is “Research productivity,” OR “Research output,” OR “Publication productivity” OR “Research Trends,” OR “Scholarly Research output” etc. In Scopus, under search documents, these keywords were used to search “Article title, abstract and keywords” and the published years included is all years till 2020. Apart from this, the search was made through other online databases like Emerald, Taylor & Francis, Google Scholar, JSTOR, ProQuest, and EBSCO for the full-text articles. Some articles were retrieved by tracking citations of the full-text articles downloaded from different online databases.

Data collection and evaluation of articles

By adopting the aforesaid procedure a total number of 205 articles were retrieved from various databases. After the elimination of duplicate articles, first-round screening was done taking into consideration the titles and abstract of the entire set of literature (N = 172) from which further elimination was done which were not appropriate as per the objectives of the study. In the second round of screening, only those articles were included where the study has dealt with metrics, bibliometric laws, statistical tools and highly cited papers and finally 131 articles were included for the study. Figure 2 illustrates the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram for the selection of articles. The final list of 131 papers covers 126 journal articles, 4 conference papers, and 1 project report. From the publication duration of the selected articles, it is observed that these cover a period of 40 years, that is, 1981–2020. The total corpus of literature covers 74 (56.49%) papers of national repute and 57 numbers (43.51%) papers of international repute.

Figure 2.

PRISMA flow diagram for selection of articles.

Analysis and synthesis

After the inclusion of the final list of articles (N = 131), all the articles were examined to gain insight into the different attributes of the studies. Once the attributes are defined, the details of all those identified articles are entered under the relevant field/attribute in an excel sheet. Based on the data collected, two types of analyses were made, that is, quantitative analysis and qualitative analysis of articles. As such the paper is structured in two parts. In Part-I, the dimensions which have been studied as components of quantitative analysis are the chronological distribution of articles, keyword-wise distribution of articles, form-wise, disciplines of the articles, publisher-wise distribution of articles, data sources used in the articles, and most prolific journals of the publications. Part-II of this paper is meant for qualitative assessment where top-cited articles with their number of citations and thrust areas of research as well as all the metrics used in the articles have been meticulously discussed.

Part-I: Quantitative analysis of articles

RQ 1: What is the chronological distribution and citation impact of articles?

Research question one (RQ 1) is about the distribution of articles published within the four decades and their citation impact which is represented in Table 1. The total number of articles (N = 131) have been published during a period of 40 years from 1981 to 2020 which has been divided into four decades. There is significant growth in the number of articles from the first decade (two papers) to 91 papers during the fourth decade (2011–2020). As regards the number of citations, the highest number of citations (1530) have been received during the third decade. A total number of 3417 citations are received for the entire article with average citations of 26.08 per paper.

Table 1.

Chronological distribution of papers.

Year	No. and % of papers	No. and % of citations	Average citation per paper (ACPP)
1981–1990	2 (1.53)	850 (24.88)	425
1991–2000	4 (3.05)	102 (2.98)	25.50
2001–2010	34 (25.95)	1530 (44.78)	45.00
2011–2020	91 (69.47)	935 (27.36)	10.27
Total	131 (100)	3417 (100)	26.08

The second part of RQ 1 is about the keywords used in the titles. The total number of articles (N = 131) was examined to assess the types of keywords used in the titles and the trend is depicted in Figure 3. It was found that many near synonymous and related terms like “research output,” “research performance,” “research trend,” “research productivity,” “publication productivity,” and “publication output” have been used to assess the research performance of different institutions. Among all these terms, “research output” has the highest occurrence (36) followed by research productivity (29) and publication productivity (12). There are eight articles were found with the terms Bibliometric and Scientometric analysis and 28 articles with various keywords like research contribution and impact, author productivity, research visibility, etc.

Figure 3.

Keywords appeared in the titles.

RQ1b: What is the form-wise distribution of these papers?

Form wise distribution of articles

The third component of RQ 1 is about the form-wise distribution of these papers. The total number of publications included in the study are consisting of 126 journal articles with 3400 citations and 4 conference papers with 13 citations and 1 project report with 4 citations (Figure 4). As there are 26.08 average citations received per article, it may be interpreted that the set of publications have received a good number of citations.

Figure 4.

Form wise distribution of papers with citations.

RQ 2: What are the disciplines of study in which works on research productivity have been carried out?

RQ-2 is regarding the disciplines of study where works on research productivity have been carried out. After a thorough examination, it is observed that the articles can be classified into five main categories (Figure 5) as per the 10 main classes of Dewey Decimal Classification (DDC 22nd edition). The sixth category is termed “Miscellaneous” which includes 20 papers which are 15.27% of the total contribution and it has included subjects covering more than one disciplines/DDC main classes. These categories of disciplines are (i) Computer science, information & general works (DDC No. 000, 5 papers, 3.82%), (ii) Social Sciences (DDC No. 300, 33 papers, 25.19%), (iii) Languages (DDC No. 400, 2 papers, 1.53%), (iv) Science (DDC No. 500, 23 papers, 17.56%), and (v) Technology (DDC No.600, 48 papers, 36.64%). Under the first category, the subfields covered are Computer Sc., Information System, Media Studies and Library and Information Science, the subjects under Social Sciences are Economics, Law, Criminology etc. Fields like English and TEFL are categorized under Language. The discipline of science includes a range of sub-fields likely Physics, Chemistry, Biochemistry, Botany, Plant Science, Earth Science, Mathematics, Pure Sciences, Physical Sc. etc. The fifth category is Technology mainly inclusive of technology (15 papers), Medicine & health (23), Engineering (4), Agriculture (1), and Management (5 papers).

Figure 5.

Discipline wise scattering of articles.

RQ 3: Which are the leading publishers in research productivity study?

Research question 3 is about the publishers of the articles included in the study. This RQ is addressed through Table 2 which portrays the publishers with their journal outlets and the number of publications. The total number of 131 articles are contributed by 34 publishers, published in 64 journal sources and five in other sources like 4 conference papers and 1 project report. Out of the publishers, it is the University of Nebraska, Lincoln (USA) has the highest number of publications (21), followed by DESIDOC (20 publications). Publishers like Taylor & Francis, Elsevier, and Emerald Insight have 10, 11, and 10 journal sources with 11 publications from each publisher. A publisher like Sage has two journals covered with three publications but a look into the numbers of citations, it is Sage that surpasses other publishers with the highest (1076) citations received for its three publications followed by Elsevier (574) and DESIDOC (370). So as regards the present set of publications, the University of Nebraska, Lincoln (USA) is the leading publisher but when citations are considered, Sage publications have a high impact.

Table 2.

Publisher wise distribution of articles.

Publisher	No. of journals/sources	No. of publications	No. of citations
University of Nebraska, Lincoln	1	21	83
DESIDOC, DRDO	1	20	370
Taylor & Francis	10	11	155
Elsevier	11	11	574
Emerald Insight	10	11	120
Springer	4	7	299
Indian Academy of Sciences	1	3	58
Sage	2	3	1076
BMC	2	2	89
ALISE	1	2	32
University of Malaya	1	2	77
Wiley Blackwell	2	2	55
Routledge	2	2	7
Others	21	34	422
Total sources (64 journal and five others)	69	131	3417

RQ 4: What are the data sources that have been used by the researchers for articles on research productivity?

To know the kinds of data sources used by the researchers for the studies on research productivity, RQ-4 was framed. For this, the methodology sections of all the articles were evaluated thoroughly and interesting results were obtained that address RQ-4. It is observed that many data sources like abstracting and indexing databases, full-text databases, websites of institutes, annual reports, discipline-specific journals, doctoral dissertations published in a particular subject and some other sources have been used. Out of all the data sources, the highest numbers of studies (79, 60%) have been conducted using data from various databases whereas only 52 articles are based on other data sources like websites, journals, annual reports, and doctoral theses (Table 3). Further out of all the databases, the highest number of studies (31) are based upon the data from the SCOPUS indexing and citation database followed by the Web of Science (28) database. The other databases used are PubMed, Google Scholar, EConlit, JGATE, INSPEC, INDCAT, etc.

Table 3.

Preferred data sources used for the articles on research productivity.

Source of data collection	No. of studies conducted
Scopus Indexing Database	31
Web of Science	28
PubMed Database	5
Google Scholar	3
EConlit/Jgate/INSPEC	2 each
INDCAT/CJPI/LISA/ALISE/American Medline database/France Pascal Database	1 each
Websites	7
Journals	7
Doctoral Theses	6
Annual Report	3
Others	29
Total	131

RQ 5: Which are the top Impact Factor (IF) journals as regards to publications on research productivity?

The Journal impact factor is a qualitative indicator that measures the frequency with which an average article in a journal has been cited in a particular year. It reflects the influence and standing of a particular journal in the field. To answer RQ-5 and to assess whether the articles on productivity study published in qualitative journals, the impact factor (IF) of all the journals as per JCR 2020 were derived and the top 10 sources were represented in Table 4. Of all the published journal sources, it is the journal Technological Forecasting and Social Change of Elsevier publication that has a high IF of 8.593. The IF of the other nine journals varies from 3.09 to 8.11. Out of the top 10 IF journals, only three journals namely, Journal of Global Antimicrobial Resistance, Health Research Policy and Systems, and BMC Infectious Diseases are the open access journals whereas the rest seven are hybrid journals. Most of the IF journals (5) were published by Elsevier publications, two by Sage Publication, two by BMC Ltd., and one by Springer Publication.

Table 4.

Journals with high IF in the corpus.

Journal name	Publisher	IF (JCR 2018)	Open access
Technological Forecasting and Social Change	Elsevier	8.593	Hybrid
Research Policy	Elsevier	8.11	Hybrid
Information Processing & Management	Elsevier	6.222	Hybrid
Sociology of Education	Sage	6.088	Hybrid
Journal of Informetrics	Elsevier	5.107	Hybrid
Journal of Global Antimicrobial Resistance	Elsevier	4.035	Open Access
Social Studies of Science	Sage	3.885	Hybrid
Health Research Policy and Systems	BMC Ltd.	3.318	Open Access
Scientometrics	Springer	3.238	Hybrid
BMC Infectious Diseases	BMC Ltd.	3.09	Open Access

Part-II: Qualitative analysis of articles

RQ6: Which are the most cited papers and what are the thrust areas of research of those citing articles?

RQ-6 is addressed in Table 5 which represents 10 highly cited articles on research productivity as well as the thrust areas of research. These 10 articles received a total number of 1897 citations which is more than half (55.51%) of the total citations received for the entire set of articles. The citation range of these articles varies from a minimum of 73 to a maximum of 619. All these highly cited articles are indexed in the Scopus database out of which eight articles are of international repute and two are of national repute. The article that received the highest citation (619) was published in the journal “Social Studies of Science” authored by Mary Frank Fox in the year 1983. Regarding the thrust areas of research, these articles highlight various dimensions of research productivity like determinants of productivity, academic and non-academic factors affecting productivity, collaboration and availability of ICT on productivity, co-authorship, and productivity, evaluation of the performance of scientists associated with Universities and research institutions, etc.

Table 5.

Highly cited papers.

Bibliographic details of article	Journal name	Publisher	Citations	Thrust area of research
Fox (1983). Publication productivity among scientists: A critical review.	Social Studies of Science	Sage	619	This paper depicts a critical assessment of the known and unknown factors about the correlations and determinants of productivity among scientists.
Wanner et al. (1981). Research productivity in academia: A comparative study of the sciences, social sciences and humanities.	Sociology of Education	Sage	231	This paper analyzed how the academic and non-academic factors affect the productivity in the field of Sc., Soc. Sc. and Humanities in Academics
Duque et al. (2005). Collaboration paradox: Scientific productivity, the Internet, and problems of research in developing areas.	Social studies of science	Sage	226	This paper examined how collaboration and availability of ICT have an effect on the productivity of scientists.
He et al. (2009). Research collaboration and research output: A longitudinal study of 65 biomedical scientists in a New Zealand university.	Research Policy	Elsevier	218	The paper highlighted that it is scientist’s ability rather than collaboration which affect the productivity.
Hollis (2001). Co-authorship and the output of academic economists.	Labour Economics	Elsevier	189	The paper evaluated the relationship between co-authorship and productivity of American economists.
Abbott and Doucouliagos (2004). Research output of Australian universities.	Education Economics	Taylor & Francis	103	The paper explored the links among research output, income, academic and non-academic factors in Australian Universities.
Falagas et al. (2006). A bibliometric analysis of research productivity in Parasitology by different world regions during a 9-year period (1995–2003).	BMC Infectious Diseases	BMC	87	The productivity of 9 world regions was analyzed using indicators in relation to GDP and total population.
Kumbar et al. (2008). Growth and impact of research output of University of Mysore, 1996-2006: A case study.	Annals of Library and Information Studies	NISCAIR	76	The paper evaluated the performance of scientists of Mysore University by using various indicators.
Jeevan and Gupta (2002). A scientometric analysis of research output from Indian Institute of Technology, Kharagpur.	Scientometrics	Springer	75	The paper evaluated the performance of the different departments of IIT, Kharagpur based on number of High Quality Papers.
Liu et al. (2015). China’s global growth in social science research: Uncovering evidence from bibliometric analyses of SSCI publications (1978–2013).	Journal of Informetrics	Springer	73	The paper assesses the productivity of China based upon SSCI dataset, depicted its global share and world ranking.

RQ7: What are the different metrics/bibliometric laws and statistical tools used by the researchers to assess the research productivity?

The focus of the present paper is to make a meticulous analysis of the various metrics employed in the articles that dealt with research productivity through the method of systematic review. It is observed that a wide range of metrics, bibliometric laws, and statistical tools have been employed in all those studies. For a systematic and scientific approach, all the attributes covered in the total literature have been categorized into three logical segments, that is, (i) Use of metrics, (ii) Application of Laws of Bibliometrics, and (iii) Application of Statistical Tools.

Use of metrics—(TP), (TC), (ACPP), (ACPY), (LCP), (DCP), (NCP), (ICP)

The indicators like Total papers (TP),Total citations (TC), Average citations per paper (ACPP), Annual citations per year (ACPY) are studied in a large number of papers by Abbott and Doucouliagos (2004), Abramo et al. (2017), Agboola and Oduwole (2005), Ahmad et al. (2018), Anderson and Tressler (2008), Anilkumar (2010, 2014), Anyaogu and Iyabo (2014), Chan et al. (2014), Cohn and Farrington (2014), Dalrymple and Varlejs (1995), Dean et al. (2011), Falagas et al. (2006), Gupta and Bala (2013), Gupta and Dhawan (2008), Gupta et al. (2010), Kaur et al. (2011), Johanson and Woods (2002), Khiste (2017), Khiste et al. (2018), Kostoff (2012), Krishnamurthy and Awari (2015), Kumar et al. (2009, 2015), Liu et al. (2015), McPherson et al. (2013), Maske Dnyaneshwar and Khiste (2018), Murugan and Ravanan (2016), Nagarkar et al. (2015), Naheem et al. (2017), Nakanjako et al. (2017), O’Brien and Cronin (2016), Ocholla et al. (2012), Peter et al. (2019), Ramos et al. (2009), Ray et al. (2016), Research Council UK (2010), Siwach and Satish Kumar (2015), Sudhier and Kumar (2010), Tsafe et al. (2016), Uma and Rao (2014), Vasishta (2011), Visakhi and Gupta (2013) and Webber (2012). The indicators that reveal the number of collaborative papers at the local, national and international levels are termed as Locally collaborated papers (LCP), Domestic collaborative papers (DCP), National collaborative papers (NCP), and International collaborative papers ICP respectively and reflected in many studies likely Dash et al. (2013), Gupta and Dhawan (2008), Dhawan et al. (2017), Gautam and Mishra (2015), Gupta (2013), Gupta et al. (2014a), Kumar and Gupta (2016), and Sevukan et al. (2007).

Gupta and Dhawan (2008) examined the Indian scientific research output and found that the NCP is higher than ICP and the collaborative research showed a faster growth rate (729.6%) than the country average (51.78%). In a study at PEC University of Technology, Vasishta (2011) studied that the share of NCP was 53.67% and ICP was 6.21% (11 papers). Out of these 11 papers, 10 are bilateral papers and 1 is a multilateral paper. USA, Iraq, and China were the most collaborative countries. The scientific research output of Odisha was studied by Dash et al. (2013) which revealed that the ICP was 12% and the overall Indian collaboration was 15%. Gautam and Mishra (2015) evaluated the productivity of BHU based on the Indian Citation Index (ICI). The 250 institutional collaborations are distributed among 67% academic institutes, 28% research institutes, and 5% scientific agencies. BHU collaborated with 18 countries out of which the USA, Japan, and the UK are the top collaborator. The total number of LCP, NCP, and ICP are 71, 227, and 52, respectively. In another study of MM University, Haryana conducted by Kumar and Gupta (2016) it was observed that the share of NCP is 39.20%, and ICP is 9.88%. The ACPP for NCP was 4.52 which is slightly higher than the university average of 3.27. The ACPP of ICP was 3.45. The major collaborative countries were the USA, South Korea, and South Arabia. The productivity of six major universities in Karnataka was studied by Gupta et al. (2014a). The findings revealed that the NCP was 56.69% with 3616 papers and ICP was 27.78% with 1635 papers. Germany, the USA, and Malaysia contributed 31.07%, 22.02%, and 13.88% ICP.

In the productivity study of the physicists of Jawaharlal Nehru University (JNU) by Singh and Kumar (2014), it was observed the ACPY for Satyabrata Pattanaik was found to be the highest at 42.40. The research performance of 16 older IITs was studied by Marisha et al. (2017) where it was found that the publication per capita (PPC) of IIT Kharagpur was 10.10 followed by IIT Madras with 9.70 and IIT Bombay with 8.30. Marisha et al. (2017) evaluated the performances of 39 Central Universities, where the PPC (publication per capita) of Delhi University, University of Hyderabad, and Tezpur University were found to be 7.36, 6.03, and 4.96, respectively. The Peak year citation per paper (PCPP) was analyzed by Wang et al. (2011) for NTU and PKU and it was 10.00 for NTU in multi-disciplinary physics whereas for PKU the PCPP was 4.7 in chemistry.

Use of metrics—(HCP)

The highly cited papers (HCP) are significant for the reputation of any individual researcher and the institution with which the researcher is associated. The papers that studied the HCP are contributed by Marisha et al. (2017), Chakravarty and Madaan (2016), Gupta (2013), Gupta et al. (2011), Kumar et al. (2015), Kumbar et al. (2008), Noruzi and Abdekhoda (2014), Satpathy and Sa (2015), Shettar and Angadi (2018), Uma and Rao (2014), and Vasishta (2011). Marisha et al. (2017) studied the productivity of 16 IITs. The first most cited paper from the IITs was contributed by IIT Kharagpur in the year 2002 which received 6500 citations, the second paper was published in 2005 by IIT Bombay and received 1495 citations, the third paper was published in 2000 by IITK in collaboration with Swiss Federal Institute of Technology that received 1376 citations. The scientific publication of Bangladesh was studied by Gupta (2013). The number of HCP was 39 scattered over 36 journals and each received 100 or more citations. Journal of Physical Chemistry and Lancet produced 5 HCP each. As studied by Kumar et al. (2015) there were three HCP papers produced by Gujarat University during the period of study from 2004 to 2013. Out of these three papers two were published in the journal Telanta (one with 133 and the other with 64 citations) and the third one in Corrosion Science with 83 citations. Kumbar et al. (2008) studied the research trend of Mysore University. The TP was 1518 and the number of HCP was 17 which received more than 30 citations each, out of these, 11 papers were international collaborative papers, 4 were contributed with national collaboration, and 2 papers were from Mysore University. Noruzi and Abdekhoda (2014) examined the scientific productivity of Iraqi-Kurdistan University. The 25 HCP received 484 citations (19.36% of TC) and Gastroenterology was the journal with the highest 103 citations. Satpathy and Sa (2015) studied the scientific productivity of six Government Universities in Odisha and reported two highly cited papers published in the journals Science (283 citations) and Tetrahedron (98 citations). Shettar and Angadi (2018) analyzed the research output of the National Institute of Technology, Suratkal (NITK, Surathkal), where the 10 HCP received citations in the range of 128–192. The paper that received 192 citations was published in Advances in Colloid and Interface Science. Vasishta (2011) analyzed the productivity of PEC University of Technology, Chandigarh. Out of 177 papers, 12 papers received 6 or more citations. These papers were published in 11 journals and one as a conference paper. In another study of performance evaluation of the University of Hyderabad by Uma and Rao (2014) the 100 HCP were found to have citations in the range of 50–757 times. The papers comprised of 86 articles, 8 review papers and 2 conference papers, and 4 short communications.

Use of metrics—(h index) (g index) (p index) (i10 index)

Other qualitative measures used for performance evaluation are the h-index, g-index, p-index, and i10 index. Several studies apply one or more of these indicators for evaluation and these papers were contributed by Cancino et al. (2017), Gupta (2010), Gupta et al. (2014b), Gupta and Mahesh (2013), Kaur et al. (2011), Kpolovie and Onoshagbegbe (2017), Gao et al. (2013), Pandita and Singh (2015), Parabhoi (2019), Prathap and Gupta (2009), Savanur and Konnur (2012), Singh and Kumar (2014), Sweileh et al. (2018), Vasishta (2011), and Wang et al. (2011). Cancino et al. (2017) studied the innovation research (IR) output of leading universities based on the Web of Science. The h index and HI7 (h index in top 7 IR journals) of the University of Pennsylvania were 96 and 13, respectively followed by Harvard University h-index of 93 and HI7 was 27 and for MIT it was 76 and 31, respectively. Gupta (2010) analyzed the productivity of 50 major Indian Universities. The h-index of the University of Hyderabad, Delhi University, and Punjab University were 49, 45, and 44 respectively. Gupta et al. (2014b) evaluated the performance of 25 Indian Universities in Social Sciences. The h index and p index of Jadavpur University were 15 and 12.10 respectively; the h index of Aligarh Muslim University, University of Calcutta, and Banaras Hindu University were 12 for each and with different p index 8.61, 7.74, and 5.77, respectively. Gupta and Mahesh (2013) analyzed the social science research of 4 South Asian countries, that is, Bangladesh, Nepal, Pakistan, and Sri Lanka from 1996 to 2011. In Social Science-general the highest h index was achieved by Bangladesh (24), followed by Pakistan (16), Sri Lanka (16), and Nepal (14). In Economics the h index of Bangladesh was highest with a value of 18, Pakistan achieved the highest h index in Decision Sciences (12) and Psychology (16).

The productivity of the Government Medical College and Hospital of Chandigarh was studied by Kaur et al. (2011). In the subject-wise analysis, Medicine, Biochemistry, and Pharmacology had h index 12, 9, and 6, respectively. In a disease-wise study, cancer papers had h index of 11 followed by tuberculosis-9 and cardiovascular-6. In the institute-wise study, PGIMER had the highest h-index 41. Harsh Mohan and Thami were found as the top authors with h index 10. Kpolovie and Onoshagbegbe (2017) evaluated the performance of Nigerian academics which revealed the mean h-index for male and female staff was 2.915 and 2.241 and the mean i10 index for them were 1.915 and 1.435 which concluded that there was no significant difference between the h index and i10 index of male and female academic staff. Gao et al. (2013) studied the productivity of six Chinese universities of traditional medicine. The h-index of Shanghai University, Beijing University, and Nanjing University were 32, 28, and 20, respectively. Pandita and Singh (2015) analyzed the global research output in LIS based on Scopus from 2004 to 2013. The countries with the highest h index were the USA (204), the UK (87), and Canada (74). In another study, Parabhoi (2019) examined the productivity of Indian academic librarians from 1989 to 2018. The top three authors ranked based on the h-index were M.G. Sreekumar (TP-25, h index-6), Angadi Mallikarjun (TP-60, h index-6), and Samir Kumar Jalal (TP-38, h index-6). Prathap and Gupta (2009) studied the research trend of 25 Indian universities. The h-index of the University of Hyderabad was 49 followed by Delhi University (DU, 45) and Panjab University (44). The p index for these universities was 37.0, 32.7, and 32.7, respectively. According to the study by Savanur and Konnur (2012), the most productive authors at Bangalore University were P.V. Kamath (h-index—21, p index—26.52), M. Jayanan (h-index—11, p index—20.38), and S.M. Mayanna (h-index—16, p index—17.91).

Use of metrics—(AGR), (CAGR), (RGR), and (DT)

To map the growth rate of publications indicators like the Annual Growth Rate (AGR), Cumulative AGR (CAGR), RGR (Relative Growth Rate), and Doubling Time (D) were applied in several studies. Those studies were carried out by Agyeman and Bilson (2015), Aswathy and Gopikuttan (2015), Baskaran (2013), Chaurasia and Chavan (2014), Elango (2019), Ghosh (2014), Gupta and Dhawan (2009), Gupta et al. (1999), Horri (2004), Jabeen et al. (2015), Jebaraj and Thirunavukkarasu (2019), Jeyshankar (2015), Jeyshankar and Vellaichamy (2015), Leeves and Poon (2015), Li et al. (2010), Majhi and Maharana (2012), Mulla and Konnur (2013), Singh (2013), and Tripathi and Kumar (2015). The RGR and DT were found to be fluctuating in many studies which showed that the growth pattern was not steady, the CAGR of Indian Lung Cancer research by Jeyshankar and Vellaichamy (2015) was found to be 16%.

Use of metrics—(DC), (CI), (CC), (MCC), and (CAI)

Other collaborative measures are Degree of Collaboration (DC), Collaborative Index (CI), Collaborative Coefficient (CC), Modified Collaborative Coefficient (MCC), and Co-authorship Index (CAI) which were used for analyzing the extent of collaboration among authors. The papers deal with these measures were studied by Aswathy and Gopikuttan (2015), Bharti and Meera (2017), Chaurasia and Chavan (2014), Das and Maharana (2016), Mini Devi and Lekshmi (2014), Gautam and Mishra (2015), Gopikuttan and Aswathy (2016), Kumar et al. (2015), Khan and Ahangar (2015), Koganuramah et al. (2002), Kumar and Dora (2012), Kumar and Naqvi (2014), Meera and Sahu (2014), Nandi and Kumar (2013), Nandi and Mondal (2017), Peter et al. (2019), Satpathy and Sa (2015), Sharma (2009), Varghese and Rajan (2009), and Vijay et al. (2019). The mean DC for the Botany Department; at the University of Burdwan is 0.70 as examined by Nandi and Kumar (2013). Meera and Sahu (2014) calculated the average DC for the University College of Medical Science (UCMS), the University of Delhi as 0.92. Kumar et al. (2015) derived the range of DC for Gujarat University between 0.89 and 1.00, Peter et al. (2019) studied the collaboration trend of health care professionals in Saudi Arabia where the average DC is as good as 0.93. The mean DC for most of the studies is found to be close to the value 1 which implies that there is an increasing trend toward multi-authored papers.

Another popular measure of author collaboration is the collaborative coefficient (CC). The value of CC is 0 when there is the dominance of single-authored papers and 1-1/j when there is the dominance of j authored papers. Gautam and Mishra (2015) found the average CC for Banarus Hindu University (BHU) was 0.52, Bharti and Meera (2017) studied the collaboration of Delhi University (DU) and JNU in the field of Arts and Humanities and the CC for these two universities were 0.20 and 0.14, respectively which implies that the trend is toward single authorship pattern. The MCC in the field of Natural Science in the case of the university, Jamia Milia Islamia was found to be varied from 0.32 to 1.00 as reported by Kumar and Naqvi (2014). Mini Devi and Lekshmi (2014) derived the MCC for Jawaharlal Nehru Tropical Botanic Garden and Research Institute (JNTBGRI), Thiruvananthapuram in the range 0.482–0.807.

The CI in the field of Natural Science for Jamia Milia Islamia over 37 years varies from 1 to 3.88 with a mean number of 2.38 authors as studied by Kumar and Naqvi (2014). Aswathy and Gopikuttan (2015) studied the productivity of teachers in the field of physics for the University of Kerala, Mahatma Gandhi University, and University of Calicut and reported that CI for these universities are 0.089, 0.177, and 0.169, respectively. The CAI for Gujarat University from 2009 to 2013 for multi-author and mega author papers are found to be 101.61 and 109.48 showing the shift from a single authorship trend to a multi-authorship pattern as examined by Kumar et al. (2015). In the same year, Jeyshankar (2015) found that the CAI for single-authored papers is lower than the CAI of multi-authored papers for Indira Gandhi Center for Atomic Research (IGCAR), Kalpakkam.

Use of metrics—(AI), (CI), (RQI), (PEI), and (PaI)

Activity Index (AI), Citability Index (CI), Relative Qualitative Index (RQI), Publication Effectiveness Index (PEI), and Participative Index (PaI) are some of the indicators that measure the level of activity in a specified time-span and geographical region and their relative impact. The papers analyzed these indicators were conducted by Gupta and Dhawan (2009), Jeevan and Gupta (2002), Mohan et al. (2010), Nagpaul (1995), Sevukan et al. (2007), Singh et al. (2005), and Tripathi and Kumar (2015). While studying Indian scientific research output, Gupta and Dhawan (2009) calculated the AI of Anna University for different time zones 1985–86 (0.33), 1993–94 (0.48), and 2001–02 (1.84). Nagpaul (1995) has used several performance indicators, that is, AI, CI, RQI, PEI to analyze the productivity of Indian Universities in the scientific literature. The AI for IIT Madras in the field of Physics is 195 and CI is 164 which revealed that IIT Madras put more than average research effort and earned more than average impact as well. In chemistry the AI and CI for Sardar Patel University (SPU), Vidyanagar were 184 and 227, respectively which depicts that the impact is higher than the effort put. The relative quality index of Hyderabad University, Bombay University, and IISc, Bangalore were 330, 168, and 156, respectively. The PEI of Hyderabad University was 1.42 and for Bombay University it was 1.35 and IISc was 1.24. The Productivity of Indian Universities in Plant sciences was studied by Sevukan et al. (2007) which revealed that the PAI for BHU was 47% followed by JNU-23.01%, and Aligarh Muslim University (AMU)—16.16%.

Advantage, disadvantages, and use of Metrics— Among all the metrics, TP is the indicator of productivity, whereas TC and ACPP show the impact. Hirsch’s h index has been proposed as a better alternative to other bibliometric indicators and it indicates excellence (Hirsch, 2007). The most-often expressed criticism of the h index is applicable to other bibliometric indicators also are the problems of field dependency, self-citations, and multi-authorship (Bornmann et al., 2010). The g-index was introduced by Egghe (2006) as an improvement of Hirsch’s h-index for measuring the overall citation record of a set of articles. The g-index is more sensitive than h-index in the assessment of selective scientists, since this type of scientist shows in average a higher g-index/h-index ratio and a better position in g-index rankings than in the h-index ones (Costas and Bordons, 2008). Prathap (2010), has introduced a new index, called the p-index or the mock h-index that maintains the best balance between activity (total citations C) and excellence (mean citation rate C/P). Collaboration among authors plays an important role in research production and it was found that ‘a major factor in research productivity is research collaboration and collaborative authorship (Lancaster, 1991). Thus, evaluating the NCP and ICP gives an idea about the visibility of the collaborative papers in its fields of knowledge. The scientific growth rate is always useful for any kind of study as it is used to verify needs of man power and finance for future and present research activity. In this regards, various indicators likely AGR, CAGR, RGR, and DT helps finding the growth trend in a discipline during a specific time period. Considering author collaboration, the extent of collaboration is computed either as the mean number of authors per paper, defined as Collaborative Index by Lawani and/or the proportion of multiple-authored papers, termed as Degree of Collaboration by Subramanyam. Ajiferuke devised the Collaboration Co-efficient as the single variable with merits of DC and CI (Ajiferuke et al., 1988). The study of HCPs in any academic discipline is vital for various reasons. Such studies are important to the authors whose work is counted among the most-cited works. Second, these studies identify the seminal works in an area of discipline. Again, the development of the literature illustrated over a time helps determining the central issues to a particular field and identifying authors with significant contributions to the field. Some of the indicators, that is, AI, CI accounted as relative research effort of a particular country in a given field.

Application of the classical laws of bibliometrics

Several studies tested the applicability of Lotka’s Law of productivity, Bradford’s Law of journal scattering, and Zipf’s key word distribution which were studied by Bhatia (2010), Gupta and Dhawan (2008), Gupta et al. (1998), Mishra et al. (2014, 2015), Muthukrishnan and Senthilkumar (2017), Poon and Leeves (2017), Savanur and Konnur (2012), Sevukan et al. (2007), and Sevukan and Sharma (2008). Gupta et al. (1998) found that Lotka’s Law and the negative binomial distribution model were applicable and valid in the initial stage of the growth of physics literature in India from 1900 to 1950. Sevukan and Sharma (2008) analyzed the research performance of biotechnology faculties of central universities. The study revealed that Lotka’s law was valid with the values n-2.12, C-0.669, and D-0.027, and the same data was not found fit Bradford’s law of distribution as the number of journals found in the three zones were in the ratio 4:16:55 which did not comply with the formula 1: n: n². The growth and research output of Bangalore University was studied by Savanur and Konnur (2012). The Bradford multiplier was 5.05 and the average Bradford multiplier was 4.85. The number of journals in the three zones was 21, 106, and 488. The number of journals in the third zone was 488 which was close to the expected value of 535 which means that though the value was close still the data did not strictly follow the law. Aswathy and Gopikuttan (2015) studied the research trends of three Universities in Kerala. Lotka’s inverse square law was found to be fit for the University of Calicut but did not fit to the data of the University of Kerala and Mahatma Gandhi University. Muthukrishnan and Senthilkumar (2017) analyzed the Indian research output in Oncology. This study calculated the value for D = 0.006, at 0.01 level of significance, K-S Statistics = 0.011. As D was less than the K-S Statistics, the data was found to fit Lotka’s law with the value n = 2.04.

Scope of bibliometrics laws

Lotka’s law can be applied both as inverse square law and in its generalized form to the author productivity data. The three methods used to evaluate the author productivity in a scientific field are straight counts, normal counts, and fractional counts. “Straight counts” assign all credits to the senior/first author; “Normal counts” give each author one credit; and “Fractional counts” assign a credit equal to l/n to each of the n co-authors. According to Crane (1972), the development in a research specialty passes through four stages. In stage one the initial development of research area with many transient authors, and few authors with more than short activity. Stage two is the pioneering stage, illustrating the exploding learning processes of a few pioneering contributors. Stage three symbolizes the condition, where the increasing productivity curve of the pioneers starts reducing and the activity of the followers of the pioneers starts increasing in a significant manner and new authors starts appearing. The last stage, that is, stage four, in which normally the perceived interest in the field falls sharply and consequently opportunities decline, and researchers/authors start migrating to other fields. One implication of Lotka’s law is that past publication increases the probability of further publication.

Bradford Law states that the number of group of journals to produce nearly equal numbers of articles is roughly in proportional to 1:n:n 2 . . ., where n is called the Bradford multiplier. In other words, no matter what the specialty, a relatively small core journals will amount for as much as 90% of the attempt together 100% of it will add journals to core at an exponential rate.

Application of statistical tools

A wide range of statistical tools and models have been used by researchers for some of the productivity studies. These tools included the mean, Coefficient of variance (CV), Standard Deviation (SD), Gini coefficient, diffusion model, regression model, and Kolmogorov Smirnov (K-S) statistical test. These papers were studied by Abouchedid and Abdelnour (2015), Bai and Hudson (2010), Bezzaoucha et al. (2014), Drivas et al. (2015), Duque et al. (2005), Fox (1983), Gayan and Singh (2019), Gupta et al. (1998), He et al. (2009), Hollis (2001), Hong et al. (2007), Hughes (1999), Lafuente and Berbegal-Mirabent (2019), Marisha et al. (2017), Muthukrishnan and Senthilkumar (2017), Neri and Rodgers (2006), Samadikuchaksaraei et al. (2018), Sevukan et al. (2007), Valadkhani and Ville (2009), and Wanner et al. (1981).

Samadikuchaksaraei et al. (2018) studied the research output of regenerative medicine in Iran in comparison to other world regions. He used the linear regression model (ANCOVA Model) to determine the best fit linear model to explain the general growth in different world regions. The highest growth rate was observed in East Asia, Europe, and North America. Gupta et al. (1998) studied the growth of Indian Physics and World Physics literature. The increased value g (Gini-coefficient) showed the rise in the concentration of author productivity and consequently, there was an increasing inequality between more productive and less productive authors which was reflected in the decreasing value of £. The research trends of faculties of plant sciences of Indian Central Universities were studied by Sevukan et al. (2007). The highest level of variation (CV) was found in Visva Bharati Shantiniketan (101.02) followed by Pondicherry University (91.18) and Delhi University (74.54).

Scope of statistical tools

Researchers use various statistical tools to study the correlation between two or more variables and implement several models to study the growth pattern. The most common statistical tools are standard deviation (SD) that indicates how far the average value lies from the mean, measuring the spread of values in a single dataset. Coefficient of variance (CV) is the ratio between standard deviation to the mean. These two variables reflect the variability in data distribution, whereas, another statistical tool Gini coefficient reflects the degree of inequality in a dataset. Regression analysis is used to study the influence of one or more independent values on a dependent variable. Besides the growth metrics, various growth models namely exponential, logistic and power models are used by the library scientists to evaluate the growth in a discipline. Pearson Correlation and Spearman correlation are used to study the correlation between two variables. The value ranges from −1 to +1 and 0 denotes no correlation between the variables.

Findings and conclusion

The present paper is a comprehensive analysis of the metrics used in a set of publications (N = 131) on research productivity through the method of systematic literature review. It analyzed 131 articles covering 40 years published by 34 publishers in 69 sources. In this systematic review process, several aspects related to the study of research productivity are explored. The findings of this study were derived in two ways, that is results of quantitative analysis and results of qualitative analysis. The prime objective of quantitative analysis is to show the growth pattern of publications, the leading journals, publishers where studies on research productivity are reported and the datasources used by researchers. As regards quantitative analysis, it was observed that there is significant growth in the number of articles during the fourth decade (2011–2020) in which 91 articles have been published whereas the highest number of citations (1530) have been received during the third decade from 2001 to 2010. Journal articles appeared to be the predominant source of information and as many as 36 articles were published with the keyword “research output” which is highest than other synonymous words. About the number of studies across the disciplines, it is Technology that dominates the other disciplines with the highest (48) number of publications covered in the dataset. Though the total number of (131) articles are contributed by 34 publishers it is University of Nebraska, Lincoln (USA) has the highest number of publications (21), followed by DESIDOC (20 publications) from India. In the list of top 10 IF journals, Research Policy , of Elsevier ranked as number 1 with IF 5.425. Among the data sources used for publication on research productivity, the highest numbers of studies (79, 60%) have been conducted using data from various databases, out of which SCOPUS has the largest share.

In the second part of the study which is meant for qualitative analysis, it was observed that the 10 highly cited articles received a total number of 1897 citations which is more than half (55.51%) of the total citations received to the entire set of articles. The review article that received the highest citation (619) published in the journal “Social Studies of Science” authored by Mary Frank Fox in the year 1983 highlighted the determinants of productivity among scientists. This indicates that the review articles are also referred by the library science researchers. Further, it was observed that the share of NCP was larger than the ICP. However, the ICP had a greater growth rate in comparison to the country average. Among ICPs, the bilateral papers were more than the multilateral papers. The USA was the top player in collaborative research in almost all subject fields. The DC was close to the value one except for a counted few studies which suggested the multi-authorship pattern in practice. AGR and DT were found to be fluctuating that showing the growth trend in both upward and downward trends. The h index of Hyderabad University (49) was highest followed by DU (45) and Panjab University (44) in the study of scientific productivity whereas in social science Jadavpur University achieved the h index of 15, p index 12.10 followed by AMU (12 and 8.61) and University of Calcutta (12 and 7.74) and BHU (12 and 5.77). East Asia produced the highest research papers in physics followed by Europe and North America. It was observed that the wide range of metrics used in the sample literature to analyze the research productivity can be classified based on their usage patterns like basic metrices associated with publications and their citation impact like—TP, TC, ACPP, ACPY, PPC, PCPP, HCP; metrics that measure the growth of publications are AGR, CAGR, RGR, DT; metrices that measure collaboration among authors are LCP, NCP, ICP, DC, CI, CC, MCC, CAI; metrices that measure the productivity and impact of publications of authors and institutions are h index, g index, p index, i10 index, PEI; metrices that measure the productivity of a country or region in a specific period are AI, CI, PEI, and PaI.

Implications

The evaluation of research productivity, whether at the level of an individual researcher, research groups, universities or institutions is an important factor. The results of such studies have major consequences in terms of funding and career growth of the individual academicians. The primary driver of an evaluation process is the assessment of the quality of research output for which a number of metrics are currently being used. The present study which is a systematic literature review focused on various metrics is unique in its own right and value addition in the area of research evaluation. The study has several implications for the researchers, practitioners and policy makers. It gives an idea about the major publishers and journals in which the researchers prefer to publish their papers on research productivity. It will be helpful to other researchers to target their publication in the appropriate journal outlet. The data sources highlighted in the paper will guide the other researchers to conduct their studies by collecting data from pertinent data sources. As the study has made a comprehensive analysis of the metrics on research productivity (28 metrics) it will serve as a reference tool to the scholars and practitioners in the field who will be acquainted with the various metrices, and laws of bibliometrics, used in the articles. Previously, policy makers used publication count, citation analysis and Impact Factor (IF) to judge the quality of scholarly research. But the present study concluded that a wide range of metrics have been developed to rank the academic standing of journals, authors, educational institutions, as well as collaboration patterns of countries in a more robust way. As every research work is a unique genre of information, the present study will be helpful to provide valuable insights into various dimensions of the research metrices.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Namita Mahapatra

Author biographies

Namita Mahapatra is working as a Deputy Librarian in KIIT deemed to be University, Bhubaneswar and a research scholar in Sambalpur University, Odisha.

Dr. Jyotshna Sahoo is presently working as an associate Professor and Head of the department of Library and Information Science in Berhampur University. She has authored 3 books, more than 35 research papers.

References

Abbott

Doucouliagos

(2004) Research output of Australian universities. Education Economics 12(3): 251–265.

Abouchedid

Abdelnour

(2015) Faculty research productivity in six Arab countries. International Review of Education 61(5): 673–690.

Abramo

D’Angelo

Murgia

(2017) The relationship among research productivity, research collaboration, and their determinants. Journal of Informetrics 11(4): 1016–1030.

Agboola

Oduwole

(2005) Staff seminars and publications productivity: A study of academic librarians in Ogun State, Nigeria. Library Management 26(8-9): 478–486.

Agyeman

Bilson

(2015) Research focus and trends in nuclear science and technology in Ghana: A bibliometric study based on the INIS database. Library Philosophy and Practice 2015(1): 1–16.

Ahmad

Jian Ming

Rafi

(2018) Assessing the digital library research output: bibliometric analysis from 2002 to 2016. The Electronic Library 36(4): 696–704.

Ajiferuke

Burell

Tague

(1988) Collaborative coefficient: A single measure of the degree of collaboration in research. Scientometrics 14(5-6): 421–433.

Anderson

Tressler

(2008) Research output in New Zealand economics departments 2000–2006: A stock approach. New Zealand Economic Papers 42(2): 155–189.

Anilkumar

(2010) Scientometric study of doctoral theses of the physical research laboratory. In: Library and information services in astronomy VI: 21st century astronomy librarianship, from new ideas to action, vol. 433, pp.102–208. Pune: Astronomical Society of the Pacific.

10.

Anilkumar

(2014) Identification of research trends at Physical Research Laboratory (PRL), Ahmedabad, India. Library Philosophy and Practice (e-journal) 1–20.

11.

Anyaogu

Iyabo

(2014) Demographic variables as correlates of lecturers research productivity in faculties of law in Nigerian universities. DESIDOC Journal of Library & Information Technology 34(6): 505–510.

12.

Aswathy

Gopikuttan

(2015) Bibliometric observation of publication output of university teachers: A study with special reference to physics. Journal of Scientometric Research 4(1): 14–19.

13.

Ayeni

Adetoro

(2017) Growth of predatory open access journals: Implication for quality assurance in library and information science research. Library Hi Tech News 34(1): 17–22.

14.

Bai

Hudson

(2010) Examining research productivity of Chinese TEFL academics across departments and institutes. International Journal for Researcher Development 1(3): 191–205.

15.

Baskaran

(2013) Research productivity of Alagappa University during 1999-2011: A bibliometric study. DESIDOC Journal of Library & Information Technology 33(3): 236–242.

16.

Bezzaoucha

Atif

Bouamra

, et al. (2014) Algerian medical teachers’ research output and its determinants during the 2000–2009 decade. Epidemiology and Public Health 62(1): 33–40.

17.

Bharti Meera (2017) Research output of University of Delhi and Jawaharlal Nehru University in the field of arts and humanities: A study of Thomson Scientific Database during 2004-2016. International Journal of Research in Library Science 3(2): 40–63.

18.

Bhatia

(2010) Innovations publications productivity of National Institute of Occupational Health: A scientometric study. SRELS Journal of Information Management 47(2): 219–227.

19.

Bornmann

Mutz

Daniel

(2010) The h index research output measurement: Two approaches to enhance its accuracy. Journal of Informetrics 4(3): 407–414.

20.

Cancino

Merigó

Coronado

(2017) A bibliometric analysis of leading universities in innovation research. Journal of Innovation & Knowledge 2(3): 106–124.

21.

Chakravarty

Madaan

(2016) SCOPUS reflected study of selected research and higher education institutions (HEIs) of Chandigarh: A city of education and research. Library Hi Tech News 33(2): 12–14.

22.

Chan

Chang

Y. Tong

, et al. (2014) A long-term assessment of research productivity in accounting and finance departments in UK: 1991-2010. Managerial Finance 40(4): 416–431.

23.

Chaurasia

Chavan

(2014) Research output of Indian Institute of Technology Delhi (IIT Delhi) during 2001-2010: A bibliometric analysis. International Journal of Information Dissemination and Technology 4(2): 141–147.

24.

Cohn

Farrington

(2014) Publication productivity of Criminologists. Journal of Criminal Justice Education 25(3): 275–303.

25.

Costas

Bordons

(2008) Is g-index better than h-index? An exploratory study at the individual level. Scientometrics 77(2): 267–288.

26.

Crane

(1972) Invisible Colleges. Chicago, IL: University of Chicago Press.

27.

Dalrymple

Varlejs

(1995) Trends in publication productivity of Library and Information Science Faculty, 1978-1988. Journal of Education for Library and Information Science 36(2): 87–103.

28.

Dash

Rout

Parida

(2013) Publications productivity of Odisha in S&T: A quantitative study. DESIDOC Journal of Library & Information Technology 33(4): 330–337.

29.

Das

Maharana

(2016) Research productivity of Sikhya ‘O’ Anusandhan University (SOAU) during 2009- 2013: A bibliometric analysis. International Research Journal of Library and Information Science 6(1): 62–107.

30.

Dean

Lowry

Humpherys

(2011) Profiling the research productivity of tenured information systems faculty at U.S. institutions. MIS Quarterly 35(1): 1–15.

31.

Denyer

Tranfield

(2009) Producing a systematic review. In: Buchanan

Bryman

(eds) The Sage Handbook of Organizational Research Methods. London: SAGE Publications Ltd, pp.671–689.

32.

Dhawan

Gupta

, et al. (2017) Comparative performance evaluation of six Haryana state universities in S&T research: A bibliometric study 2006-15. International Journal of Information Dissemination and Technology 7(1): 39–48.

33.

Drivas

Balafoutis

Rozakis

(2015) Research funding and academic output: Evidence from the Agricultural University of Athens. Prometheus 33(3): 235–256.

34.

Duque

Ynalvez

Sooryamoorthy

, et al. (2005) Collaboration paradox: Scientific productivity, the Internet, and problems of research in developing areas. Social Studies of Science 35(5): 755–785.

35.

Egghe

(2006) An improvement of the h-index: The g-index. ISSI Newsletter 2(1): 8–9.

36.

Elango

(2019) A bibliometric analysis of literature on engineering research among BRIC countries. Collection and Curation 38(1): 9–14.

37.

Falagas

Papastamataki

Bliziotis

(2006) A bibliometric analysis of research productivity in parasitology by different world regions during a 9-year period (1995–2003). BMC Infectious Diseases 6(1): 1–6.

38.

Fox

(1983) Publication productivity among scientists: A critical review. Social Studies of Science 13(2): 285–305.

39.

Gao

Tian

, et al. (2013) Papers published from 1995 to 2012 by six traditional Chinese medicine universities in China: A bibliometric analysis based on science citation index. Journal of Traditional Chinese Medicine 33(6): 832–844.

40.

Gautam

Mishra

(2015) Scholarly research trend of Banaras Hindu University during 2004-2013: A scientometric study based on Indian citation index. DESIDOC Journal of Library & Information Technology 35(2): 75–81.

41.

Gayan

Singh

(2019) Citation analysis of doctoral theses of physics : A bibliometrics study. Library Philosophy and Practice.

42.

Geisler

Abdallah

(2000) The Metrics of Science and Technology. Westport, CT: Greenwood Publishing Group.

43.

Ghosh

(2014) CSIR-IICB research productivity during 2001-2010: A scientometric analysis of publications. International Research. Journal of Library and Information Science 4(2): 287–314.

44.

Gopikuttan

Aswathy

(2016) Publication productivity of University of Kerala: A scientometric view. DESIDOC Journal of Library & Information Technology 34(2): 131–139.

45.

Gupta

(2010) Ranking and performance of Indian universities, based on publication and citation data. Indian Journal of Science and Technology 3(7): 837–839.

46.

Gupta

(2013) Bangladesh: A scientometric analysis of national publications output in S&T, 2001-10. DESIDOC Journal of Library & Information Technology 33(1): 32–44.

47.

Gupta

Bala

(2013) Glaucoma research: A scientometric study of Indian publications output, 2002-11. Library Philosophy and Practice (e-journal).

48.

Gupta

Dhawan

(2008) A scientometric analysis of S&T publications output by India during 1985-2002. DESIDOC Journal of Library & Information Technology 28(2): 73–85.

49.

Gupta

Dhawan

(2009) Status of physics research in India: An analysis of research output during 1993–2001. Scientometrics 78(2): 295–316.

50.

Gupta

Kaur

Bala

(2011) Mapping of Indian Diabetes Research during 1999-2008: A scientometric analysis of publications output. DESIDOC Journal of Library & Information Technology 31(2): 143–152.

51.

Gupta

Kshitij

Singh

(2010) Indian computer science research output during 1999-2008: Qualitative analysis. DESIDOC Journal of Library & Information Technology 30(6): 39–54.

52.

Gupta

Mahesh

(2013) A comparative analysis of social sciences research publications in four south Asian countries. Library Philosophy and Practice (e-journal) 1–12.

53.

Gupta

Sharma

Kumar

(1998) Literature growth and author productivity patterns in Indian physics. Information Processing & Management 34(1): 121–131.

54.

Gupta

Sharma

Kumar

(1999) Growth of world and Indian physics literature. Scientometrics 44(1): 5–16.

55.

Gupta

Kumbar

Gupta

(2014a) Contribution and citation impact of Indian universities: A case study of Karnataka universities, 2007-2011. KIIT Journal of Library and Information Management 2(1): 40–63.

56.

Gupta

Kumbar

Tiwari

(2014b) Ranking of Indian universities in social sciences using bibliometric indicators during 2008-12. DESIDOC Journal of Library & Information Technology 34(3): 197–205.

57.

Hedjazi

Behravan

(2011) Study of factors influencing research productivity of agriculture faculty members in Iran. Higher Education 62(5): 635–647.

58.

Geng

Campbell-Hunt

(2009) Research collaboration and research output: A longitudinal study of 65 biomedical scientists in a New Zealand university. Research Policy 38(2): 306–317.

59.

Hirsch

(2007) Does the h index have predictive power?. Proceedings of the National Academy of Sciences of the United States of America 104(49): 19193–19198.

60.

Hollis

(2001) Co-authorship and the output of academic economists. Labour Economics 8(4): 503–530.

61.

Hong

Xuezhu

(2007) On the relationship between research productivity and teaching effectiveness at research universities. Frontiers of Education in China 2: 298–306.

62.

Horri

(2004) Bibliometric overview of library and information science research productivity in Iran. Journal of Education for Library and Information Science 45(1): 15–25.

63.

Hughes

(1999) Faculty publishing productivity: The emerging role of network connectivity. Campus-Wide Information Systems 16(1): 30–38.

64.

Jabeen

Yun

Rafiq

, et al. (2015) Research productivity of library scholars bibliometric analysis of growth and trends of LIS publications. New Library World 116(7/8): 433–454.

65.

Jebaraj

Thirunavukkarasu

(2019) Research productivity of faculty working in Christian institutions of higher education in India: A bibliometric analysis. Library Philosophy and Practice (e journal).

66.

Jeevan

Gupta

(2002) A scientometric analysis of research output from Indian Institute of Technology, Kharagpur. Scientometrics 53(1): 165–168.

67.

Jeyshankar

(2015) Research productivity of the scientists of Indira Gandhi Centre for Atomic Research (IGCAR) Kalpakkam (Chennai): a scientometric analysis. Library Philosophy and Practice (e journal).

68.

Jeyshankar

Vellaichamy

(2015) Research productivity of lung cancer by Indian scientists during 1984-2013. Productivity 56(1): 24–35.

69.

Johanson

Woods

(2002) Analyzing U.S. Hospitality faculty’s research output. Journal of Hospitality & Tourism Education 14(3): 52–56.

70.

Kaur

Mahajan

Gupta

(2011) Scientometric analysis of the research output: A study of Government Medical College & Hospital (GMCH), Chandigarh. Collnet Journal of Scientometrics and Information Management 5(2): 217–226.

71.

Khan

Ahangar

(2015) Research productivity of Government Medical College Jammu: A bibliometric analysis. Library Philosophy and Practice (e journal).

72.

Khiste

(2017) Publication productivity of ‘Consortia’ by Scopus during 1989-2016. International Journal of Current Innovation Research 3(11): 879–882.

73.

Khiste

Maske

Deshmukh

(2018) Analysis of publication productivity of consortia by J-Gate database. Research Journal of Humanities and Social Sciences 9(1): 277–280.

74.

Koganuramah

Angadi

Kademani

(2002) Bibliometric dimension of innovation communication productivity of Tata Institute of Social Sciences. Malaysian Journal of Library & Information Science 7(1): 69–76.

75.

Kostoff

(2012) China/USA nanotechnology research output comparison—2011 update. Technological Forecasting and Social Change 79(5): 986–990.

76.

Kpolovie

Onoshagbegbe

(2017) Research Productivity: h-index and i10-index of academics in Nigerian Universities. International Journal of Quantitative and Qualitative Research Methods 5(2): 62–123.

77.

Krishnamurthy

Awari

(2015) Research productivity of social science departments as reflected in INDCAT: A study of Karnataka University, Dharwad. International Research: Journal of Library and Information Science 5(2): 269–274.

78.

Kumar

Gupta

(2016) Research performance of Maharishi Markandeshwar University, Mullana-Ambala during 2007-2015: A scientometric study. Library Philosophy & Practice (e journal).

79.

Kumar

Prakasan

Mohan

, et al. (2009) Bibliometric and scientometric studies in physics and engineering: recent ten years analysis. In: National conference on “Putting knowledge to work: Best practices in librarianship,” Mumbai, India, 1–2 May, pp.214–229.

80.

Kumar

Dora

(2012) Research productivity in a management institute: An analysis of research performance of Indian Institute of Management Ahmedabad during 1999-2010. DESIDOC Journal of Library & Information Technology 32(4): 365–372.

81.

Kumar

Dora

Desai

(2015) A bibliometrics profile of Gujarat University, Ahmedabad during 2004-2013. DESIDOC Journal of Library & Information Technology 35(1): 9–16.

82.

Kumar

Naqvi

(2014) Collaboration pattern in the field of natural sciences at Jamia Millia Islamia, New Delhi During 1971-2007. In: 9th Convention PLANNER-2014, Dibrugarh University, Assam, 25–27 September.

83.

Kumbar

Gupta

Dhawan

(2008) Growth and impact of research output of University of Mysore, 1996-2006: A case study. Annals of Library and Information Studies 55(5): 185–195.

84.

Lafuente

Berbegal-Mirabent

(2019) Contract employment policy and research productivity of knowledge workers: An analysis of Spanish universities. The International Journal of Human Resource Management 30(16): 2360–2386.

85.

Lagorio

Pinto

Golini

(2016) Research in urban logistics: A systematic literature review. International Journal of Physical Distribution & Logistics Management 46(10): 908–931.

86.

Lancaster

(1991) Bibliometric Methods in Assessing Productivity and Impact of Research. Bangalore: Sarada Ranganathan Endowment for Library Science.

87.

Leeves

Poon

(2015) Chinese universities economic research output 2000–2010. Journal of Asian Economics 40: 1–9.

88.

Liu

Tang

, et al. (2015) China’s global growth in social science research: Uncovering evidence from bibliometric analyses of SSCI publications (1978–2013). Journal of Informetrics 9(3): 555–569.

89.

Wan

, et al. (2010) Pathological research output in China and other top-ranking countries: 10-year survey of the literature. Pathology - Research and Practice 206(12): 835–838.

90.

Majhi

Maharana

(2012) Research productivity of physical science disciplines in Sambalpur University (Orissa): A scientometric study. Journal of Arts, Science and Commerce 4(4): 108–115.

91.

Marisha Banshal

Singh

(2017) Research performance of central universities in India. Current Science 112(11): 2198–2207.

92.

Maske Dnyaneshwar

Khiste

(2018) Mapping of publication productivity of ‘Public Library: A study. Vidyawarta 432–440.

93.

McPherson

Kim

Dorman

, et al. (2013) Research output at US economics departments. Applied Economics Letters 20(9): 889–892.

94.

Meera

Sahu

(2014) Research output of University College of Medical Science, University of Delhi: A bibliometric study. Collnet Journal of Scientometrics and Information Management 8(2): 401–418.

95.

Mini Devi

Lekshmi

(2014) Scientometric assessment of publication productivity of JNTBGRI, Thiruvananthapuram. DESIDOC Journal of Library & Information Technology 34(2): 147–151.

96.

Mishra

Gawde

Solanki

(2014) Bibliometric study of Ph.D. thesis in English. Global Journal of Academic Librarianship 1(1): 19–36.

97.

Mishra

Sahu

Brahma

, et al. (2015) A bibliometric study on growth of research in an Indian Medical College. Library Philosophy and Practice (e-journal).

98.

Mohan

Prakasan

Kademani

, et al. (2010) Research Trends in Nanoscience and Nanotechnology in India. DESIDOC Journal of Library & Information Technology 30(2): 40–58.

99.

Mulla

Konnur

(2013) Mapping of engineering research trend in Karnataka: A special reference to Visvesvaraya Technological University. DESIDOC Journal of Library & Information Technology 33(1): 55–62.

100.

Murugan

Ravanan

(2016) A scientometric study of the research productivity of anna university based on Scopus database in the year 1979-2014. Fevischrift Volume of Prof. M. Nagarajan Library and Information Science Evergreen Trends, pp.184–198.

101.

Muthukrishnan

Senthilkumar

(2017) Author productivity of oncology research output in India: Testing Lotka’s law. International Journal of Information Dissemination and Technology 7(3): 187.

102.

Nagarkar

Veer

Kumbhar

(2015) Bibliometric analysis of papers published by faculty of life science departments of Savitribai Phule Pune University during 1999-2013. DESIDOC Journal of Library & Information Technology 35(5): 368–375.

103.

Nagpaul

(1995) Contribution of Indian universities to the mainstream scientific literature: A bibliometric assessment. Scientometrics 32(1): 11–36.

104.

Naheem

Nagalingam

Ramesha

(2017) Chronic liver disease (CLD) research in SAARC countries: A scientometric analysis of research output during 1996-2015. Annals of Library and Information Studies 64(1): 59–68.

105.

Nakanjako

Akena

Kaye

, et al. (2017) A need to accelerate health research productivity in an African university: The case of Makerere University College of Health Sciences. Health Research Policy and Systems 15(1): 1–11.

106.

Nandi

Kumar

(2013) Scientometric dimensions of research productivity of the Botany Department from 1960-2000. Library Philosophy & Practice (e-journal).

107.

Nandi

Mondal

(2017) Research productivity in the departments of chemistry and physics of the University of Burdwan during 1960 - 2000: A comparative study. Library Philosophy and Practice (e-journal).

108.

Neri

Rodgers

(2006) Ranking Australian economics departments by research productivity. Economic Record 82: S74–S84.

109.

Noruzi

Abdekhoda

(2014) Scientometric analysis of Iraqi-Kurdistan universities’ scientific productivity. The Electronic Library 32(6): 770–785.

110.

Ocholla

Bosire Onyancha

(2012) Research visibility, publication patterns and output of academic librarians in sub-Saharan Africa: the case of Eastern Africa. Aslib Proceedings 64(5): 478–493.

111.

O’Brien

Cronin

(2016) Research output of academic librarians from Irish Higher Education Institutions 2000–2015: Findings from a review, analysis, and Survey. New Review of Academic Librarianship 22(2-3): 203–224.

112.

Pandita

Singh

(2015) Research growth in LIS during last decade: A study. Library Review 64(8-9): 514–532.

113.

Parabhoi

(2019) Scholarly publications of academic librarians in India from 1989-2018: A bibliometrics analysis. Library Philosophy and Practice (e-journal).

114.

Peter

Samuel

Idhris

, et al. (2019) Bibliometric analysis of research productivity of health care professionals in Saudi Arabia: An exploratory study. Library Philosophy and Practice (e-journal).

115.

Poon

Leeves

(2017) Research output: Evidence from economics departments in the Asia-Pacific region. Journal of the Asia Pacific Economy 22(4): 604–620.

116.

Prathap

(2010) Is there a place for a mock h-index? Scientometrics 84(1): 153–165.

117.

Prathap

Gupta

(2009) Ranking of Indian universities for their research output and quality using a new performance index. Current Science 97(6): 751–752.

118.

Ramos

Masiá

Padilla

, et al. (2009) [Spanish scientific research output on tuberculosis indexed in MEDLINE, 1997-2006]. Archivos de Bronconeumologia 45(6): 271–278.

119.

Ray

Shah

Nundy

(2016) The research output from Indian medical institutions between 2005 and 2014. Current Medicine Research and Practice 6(2): 49–58.

120.

Research Council UK (2010) Bibliometric Study of India Research Output & International Collaboration. Research Council UK.

121.

Samadikuchaksaraei

Mohammadhassanzadeh

Shokraneh

, et al. (2018) A bibliometric trend analysis of regenerative medicine research output in Iran: Comparison with the global research output. Materials Today Proceedings 5(7): 15506–15515.

122.

Satpathy

(2015) Research outputs of State Government Universities of Odisha: A bibliometric study. Library Philosophy and Practice (e-journal).

123.

Savanur

Konnur

(2012) Growth and impact of research output of Bangalore University, 1971-2010: A scientometric study. International Journal of Library and Information Science 4(5): 71–80.

124.

Senivongse

Bennet

Mariano

(2017) Utilizing a systematic literature review to develop an integrated framework for information and knowledge management systems. VINE Journal of Information and Knowledge Management Systems 47(2): 250–264.

125.

Sevukan

Nagarajan

Sharma

(2007) Research output of faculties of plant sciences in central universities of India: A bibliometric study. Annals of Library and Information Studies (ALIS) 54(3): 129–139.

126.

Sevukan

Sharma

(2008) Bibliometric analysis of research output of biotechnology faculties in some Indian Central Universities. DESIDOC Journal of Library & Information Technology 28(6): 11–20.

127.

Sharma

(2009) Research publication trend among scientists of Central Potato Research Institute: A bibliometric study. Annals of Library and Information Studies 56(1): 29–34.

128.

Shettar

Angadi

(2018) Publication Productivity and Impact of Research in NITK, Surathkal: A Scientometric View. Dr. B. S. Kademani Festschrift Metamorphosis of Librarianship: Connecting People in Digital Environment. Navi Mumbai: NavVishnu Publications, pp.123–135.

129.

Singh

(2013) Contribution of scholarly research output in Physics: A study with special reference to the Banaras Hindu University. International Journal of Information Dissemination and Technology 3(1): 15–21.

130.

Singh

Kumar

(2014) Application of h and g indices to quantify scientific productivity of physicists at JNU, India. Library Philosophy and Practice (e-journal).

131.

Singh

Gupta

Kumar

(2005) Research contributions and impact of research of Indian Institute of Technology, Roorkee, 1993 to 2001. Annals of Library and Information Studies 52(1): 8–14.

132.

Siwach

Satish Kumar

(2015) Bibliometric analysis of research publications of Maharshi Dayanand University (Rohtak) during 2000-2013. DESIDOC Journal of Library & Information Technology 35(1): 17–24.

133.

Sudhier

KGP

Kumar

(2010) Scientometric study of doctoral dissertations in biochemistry in the University of Kerala, India. Library Philosophy and Practice (e-journal).

134.

Sweileh

Al-Jabi

Zyoud

, et al. (2018) Global research output in antimicrobial resistance among uropathogens: A bibliometric analysis (2002-2016). Journal of Global Antimicrobial Resistance 13: 104–114.

135.

Tripathi

Kumar

(2015) A quantitative analysis of research output of Jawaharlal Nehru University, New Delhi, India. The International Information & Library Review 47(1-2): 39–51.

136.

Tsafe

Chiya

Aminu

(2016) Scholarly publications of librarians in universities in Nigeria: 2000-2012: A bibliometric analysis. Library Philosophy and Practice (e-journal).

137.

Uma

Rao

(2014) Evaluation of research performance using Journal citations of University of Hyderabad: A case study. International Journal of Information Dissemination and Technology 4(2): 168–173.

138.

Valadkhani

Ville

(2009) Discipline-specific forecasting of research output in Australian universities. Applied Economics Letters 16(18): 1875–1880.

139.

Varghese

Rajan

(2009) Productivity of scientists of Rajiv Gandhi Centre for Biotechnology (RGCB): An analysis. Annals of Library and Information Studies 56(3): 156–162.

140.

Vasishta

(2011) Assessment of academic research output during 1996-2009: A Case Study of PEC University of Technology, Chandigarh. DESIDOC Journal of Library & Information Technology 31(2): 136–142.

141.

Vijay

Rao

Sivasubraminiyan

(2019) Solid waste research in India during 2008-2017: A Bibliometric analysis. Library Philosophy and Practice (e-journal).

142.

Visakhi

Gupta

(2013) Contribution and impact of faculty and scholars of IISER Mohali: A scientometric study, 2008–12. Library Philosophy and Practice (e-journal).

143.

Wang

(2011) Comparison of universities’ scientific performance using bibliometric indicators. Malaysian Journal of Library & Information Science 16(2): 1–19.

144.

Wanner

Lewis

Gregorio

(1981) Research productivity in Academia: A comparative study of the sciences, social sciences and humanities. Sociology of Education 54(4): 238–253.

145.

Webber

(2012) Research productivity of foreign- and US-born faculty: Differences by time on task. Higher Education 64(5): 709–729.

146.

Webster

Watson

(2002) Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly 26(2): xiii–xxiii.