H-index and research evaluation: A suggested set of components for developing a comprehensive author-level index

Abstract

The H-index has been investigated in various studies; this index has many strengths that have made it popular. However, it also has weaknesses, due to which other indicators have been developed. This study aims to identify the strengths and weaknesses of the H-index and provide the minimum set of necessary components for developing a comprehensive author-level index. In this systematic literature review, Scopus, PubMed, Web of Science, Emerald, and ProQuest databases were searched to identify relevant studies. From the number of 14,253 retrieved studies, after two stages of screening, 81 studies were selected according to the eligibility criteria for data extraction. The findings of the study led to the identification of 15 strengths in the three categories of Quality Features, Simplicity, and Suitability, and 13 weaknesses in the six categories of Publications, Citations, Academic Age, Author Credit Allocation, Variety of Fields, and mathematical calculation for H-index. Finally, 28 components were identified as the minimum set of necessary components to develop a comprehensive author-level index to help evaluate researchers more realistically and fairly. The minimum components that need to be considered in developing a comprehensive author-level index can be proposed as follows: Quality Features, Simplicity, Suitability, Publications, Citations, Academic Age, Author Credit Allocation, Variety of Fields, and mathematical calculation.

Keywords

Author-level metrics bibliometrics citations H-index publications scientific evaluation scientometrics

1. Introduction

Conducting different types of research is necessary for the advancement of science, improvement of processes and improvement of human life. Today, scientists from different subject areas are increasingly competing with each other for recruitment as faculty members [1 –4], holding organisational positions [5], receiving funding and research grants [1,2,4,6], winning scientific prizes and awards [3,4], obtaining Scholarships and study opportunities [7], directing research projects [7 –9] and the like. Therefore, it is necessary to measure the quality of their research correctly. The need to measure the quality of research has led to the growth of scientific metrics and the emergence of new types of combined indicators [10]. Each of these scientometrics indicators has been created to evaluate the scientific productions of researchers based on a specific aspect and have their strengths and weaknesses. In other words, none is completely inclusive and exclusive.

One of the most famous indices for measuring the research performance of authors is the H-index [11], which has its strengths and weaknesses. The H-index, combining quantity and quality [12], evaluates a person’s research performance through the balance between the number of articles and the number of citations [13]. Among the strengths of the H-index, we can point out the simplicity of its calculation [14] and the display of only one integer [15]. Some of the weaknesses of the H-index include considering only scientific productions with citations and insensitivity to high citations and low citations, and negative citations [16]; not paying attention to the scientific age of researchers [17]; and not calculating the role and position of the author [18].

Since other developed indicators also have their strengths and weaknesses and the philosophy of creating them is usually to eliminate the weaknesses of other indicators [10], the process of criticising the existing indicators and compiling new indices to develop a more suitable index is continuous and endless. The development of a comprehensive index that measures both the quantity and quality of scientific productions in different aspects and takes into account the factors affecting these aspects requires identifying the strengths of the compiled indices and using them, and being aware of the weaknesses and correcting them in the index [10]. It seems for developing a comprehensive index, considering all quantitative and qualitative aspects of people’s scientific output in a precise way, the needs and subject area of different scientific productions [19], and various aspects of individuals’ research evaluation and giving weight to these aspects, are essential.

One of the most important components that should be considered in the evaluation of researchers is the role of the author, the number of authors, the order of authors, the position of each author, and the subject area of expertise of individuals [20]. For example, is the position of authors the same in articles with a small number of authors as in kilo papers with a large number of authors? [21], or is the subject of the article related to the author’s specialisation and field of study considered in the indicators? [22]. It seems that less attention has been paid to the importance of such cases in compiling indices.

The existence of similar names is another effective factor in the research evaluation of authors, which can cause many problems in calculating the index score of each researcher, so that the scientific productions of one author are calculated for another author with the same name [23]. Different spellings of names can also lead to problems in the calculation of indices. The type of scientific productions, such as letters to the editor, short articles, original articles, review articles, books, conference articles, educational guides, etc. [24], is another case that is often considered the same in scientometric indicators. However, each of the scientific productions can also be published in different information media, such as books, conference proceedings or scientific journals that have different indices [25], which have different views and citations. Among the types of information media, journals have different peer review systems or even without them, which makes both the value of the journal and the value of the published articles different [26]. Although in most of the indicators, all these items have been given the same value.

The method of assigning credit to authors is also different. So, in some indices, all authors are assigned the same value; in some, the fractional value [2], and in some, it has been calculated harmonically [27]. However, different subject areas [28] and citation patterns specific to each subject area [29] have also received less attention in the compilation of metrics. Another thing that has received less attention in the compilation of indicators is the effectiveness of scientific productions and their applicability [30].

Several review studies have been published regarding the types of scientometric indicators, each of which has only described some indicators individually [10,13,18,19,22,31 –41]. The difference between this review and other review studies related to scientometric indices is that this study tries to extract and categorise the strengths and weaknesses of the H-index by examining relevant evidence. The premise of this study is that each of the weaknesses of the H-index has created a strong point in the next new index, which will be identified in this study. These results can pave the way for compiling a more appropriate and comprehensive global author-level index.

2. Method

This systematic literature review was conducted to identify the strengths and weaknesses of the H-index and the strengths of the existing global author-level indices based on the H-index using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). In this research, four stages of the PRISMA flowchart were performed. In the first stage, the desired articles were retrieved in the databases. In the second stage, the retrieved articles were screened and a decision was made regarding the eligibility of the studies, and finally, the eligible articles were included in the study. In the ‘Results’ section, these steps and their results are presented.

2.1. Search strategy

In this study, Web of Science, Scopus, PubMed, Emerald and ProQuest databases were searched to retrieve relevant sources from the beginning until December 31, 2022. Also, to complete the results, a hand search was performed in related scientometrics journals and Google Scholar. In addition, after the final studies were identified, their references were reviewed to find further relevant studies. Retrieving related sources in databases was done using the search strategy presented in Table 1. Also, the search strategy for each database is in Supplementary material 3.

Table 1.

General search strategy.

	Indicator* OR index OR indices OR metric* OR benchmark OR criteri* OR ‘research assess’ OR ‘academic promo’ OR ‘research evaluat’ OR ‘research measur’
AND	‘Author* level’ OR ‘Author-level’ OR ‘Journal level’ OR ‘Journal-level’ OR ‘Institute level’ OR ‘Institute-level’ OR ‘Article level’ OR ‘Article-level’ OR Alternative OR Altmetrics OR ‘researcher excellence’ OR ‘researcher funding’ OR ‘researcher perform’ OR ‘researcher potential’ OR ‘research* career pathway’ OR ‘academic career pathway’ OR ‘researcher impact’ OR ‘scientific* productivity’ OR ‘academic productivity’ OR ‘top researcher’ OR ‘researcher ranking’ OR ‘researcher output’ OR ‘individual researcher’
AND	Disadvantage OR Weakness OR demerits OR Advantage OR Strong OR merits

2.2. Inclusion/exclusion criteria

The inclusion criteria were (1) all quantitative and qualitative studies; (2) original and review articles, theses and conference papers and (3) studies published in English. Exclusion criteria were (1) abstracts, short communications and book reviews and (2) studies whose full text could not be accessed by the research team.

2.3. Study selection

The process of selecting studies was done in two stages. In the first stage, the title and abstract of the studies were matched with the eligibility criteria by the researcher (Reviewer 1), and irrelevant items were removed. In the second stage, the full text of the remaining studies was reviewed by two reviewers (Reviewer 1, Reviewer 2) independently based on the eligibility criteria. Disagreements between the two browsers were resolved through consensus.

2.4. Data extraction

The information of each study including the author’s name, title, journal name, year of publication, country, study design, study sample, strengths of H-index, weaknesses of H-index and solutions was extracted using the data extraction form. For each study, information was extracted independently by two reviewers (Reviewer 1, Reviewer 2) and disagreements were resolved through consensus. If the two reviewers did not agree, the disagreements were resolved with the opinion of the third reviewer (Reviewer 3).

2.5. Data synthesis

Data were summarised, combined and classified using a descriptive method. Then, it was reported narratively and in the form of a table.

3. Results

During the database search, a total of 14,253 studies were found. After removing duplicates using EndNote X8, 8,342 items remained. After screening the title and abstract, 1,242 records of relevant studies were identified, and 68 studies remained in the full-text screening stage. The selection of the full text of the studies was done by two reviewers independently. Finally, in the reference review of the selected studies, 13 other studies were included. Finally, 81 studies found the inclusion conditions (Table 2 and Figure 1).

Table 2.

Retrieved documents based on each database.

Database	Retrieved documents	After removing duplicate	Title/abstract screening	Full-text screening
Scopus	3,116	1,895	315	15
Web of Science	2,680	1,458	219	13
PubMed	2,821	789	78	10
ProQuest	411	321	71	11
Emerald	4,645	3,638	512	16
Google Scholar	580	241	47	3
All Databases	14,253	8,342	1,242	68
Added recorded by examining references of eligible documents				13
Total				81

Figure 1.

PRISMA flow diagram.

Based on the findings, 15 strong points for the H-index were identified in three general categories, which can be taken into consideration in compiling a comprehensive index (Table 3).

Table 3.

Strengths of the H-index obtained from studies.

Category	Components	Strengths of H-index	Relevant studies(based on Supplementary material 3)
Quality features	Singularity	The H-index is a simple and single indicator that combines quantity of publications and their citation’s impact.	[2,8,10,11,14,17 –19,22,24,25,42 –58]
	Robustness	The H-index is insensitive to an accidental set of uncited or lowly cited papers or several, outstandingly highly cited papers or small changes in the citation count.	[2,10,11,14,17 –19,22,30,42 –44,46,47,51 –53,58 –64]
	Objectivity	The H-index is an objective indicator and plays an important role when making decisions about promotions, fund allocation and awarding prizes.	[19,65,66]
	Accuracy	The H-index has accuracy applicability.	[30]
	Comprehensibility	The H-index is a simple and comprehensive index to quantify an individual’s scientific achievement.	[67,68]
	Field dependency	The H-index is an index for comparing the scientists in a field or homogeneous group.	[10,11,14,29,43,51,53,69 –73]
Simplicity	Simple to calculate	The H-index is a mathematically simple index.	[4,8,12 –14,16 –19,30,44,48,49,51,53,62,67,69 –71,74]
	Simple to obtain	The H-index can be easily obtained from any publication indexing database.	[8,10,11,14,19,44,50,71,73]
	Simple to compute	The H-index is simple to compute and does not need any data processing.	[4,8,10,11,19,24,46,48,62,66,75]
	Simple to understand	The H-index is easy to understand.	[4,8,10,11,14,19,24,44,46,48,58,66]
	Simple to define	The H-index has the simple definition.	[4,14,18,19,24,30,44,48,49,53,62,67,69,70,76,77]
	Simple to count	The H-index is an integer number.	[14,19,22,44,45,58,78]
Suitability	Suitable for all types of documents	Any document type can be included in H-index.	[71]
	Suitable for all levels of aggregation	The H-index can be applied to any level of aggregation.	[10,71]
	Suitable for individual’s research performance	The H-index is a valid and useful scientometrics tool of an individual’s research performance.	[11,79]

Also, the review of studies led to the identification of 13 weak points of the H-index in six general categories. Paying attention to these cases and avoiding them can be useful in creating a comprehensive author-level index (Table 4).

Table 4.

Weaknesses of the H-index and suggestions for improving them obtained from studies.

Category	Components	Weaknesses of H-index	Suggested index for these weaknesses	Relevant studies(based on Supplementary material 3)
Publications	Insensitivity to the overall performance of a researcher	The H-index considers only h-core publications.	Consider cited and uncited papers.	DS-index [4]
		The H-index fails to evaluate the overall output of a researcher because it considers only the cited papers and specially high-impact papers.	Consider cited and uncited papers.	[24]
		The H-index is insensitive to performance changes.	The q2-index considers both the number of papers and impact of papers.	q2-index [46]
	Insensitivity to frequency of the citations	The H-index and its variants do not consider the frequency with which individual citations are mentioned in an article.	The WL-index uses the number of citations mentions received by the papers of a researcher.	WL-index [3]
		The H-index is only weakly sensitive to the number of citations received.	Consider citers to measure of an author’s reach or influence in a field.	[45,71,80,81]
Citations	Insensitivity to excess citation count	In H-index, when the citation count of an article exceeds H, no additional credit is given.	The HT-index (Tapered H-index) measures citation intensity in h-core or gives more weight to highly cited papers.	hT-index (Tapered h-index) [43]
		The original H-index does not take the exact number of citations of articles included in the h-core into account.	The Hw-index considers citation intensity in h-core or gives more weight to highly cited papers.	Hw-index [82]
		The H-index is insensitive to the distributions of citations within the h-core.	The J-index aims to takes into account the excess of citations within the h-core.	J-index [62]
		The H-index ignores the effect of massive citations by papers/patents.	The R-index measures the citation intensity.	R-index [71]
			AR-index measures the h-core’s citation intensity and takes the age of publications into account. This allows for an index that can actually increase and decrease over time	AR-index [71]
			The F-index, the hg-index and the m-index measure citation intensity in h-core or give more weight to highly cited papers.	F-index [83]hg-index [8]m-index [22]
			The h⁽²⁾-index gives more weight to highly cited articles.	h⁽²⁾-index [84]
			The ψ-index designed to reflect the ‘offset-ability’ of top-cited papers.	ψ-index [85]
		The original H-index does not take the exact number of citations of articles included in the h-core into account.	The A-index gives more weight to highly cited papers and measures the citation intensity of all publications in the h-core.	A-index [52]
		The H-index ignores the effect of excess citations.	The e-index considers a real number for the ignored excess citations and can be useful for evaluating highly cited scientists or for comparing the scientific output of a group with an identical H-index.	e-index [59]
		The H-index is insensitive to one or several outstandingly highly cited papers.	The g-index solves the insensitiveness of ‘big hits’ by summing the citations of the top-cited publications.	g-index [16,47]
		The H-index does not capture the citation curve and the whole production of a researcher.	The w-index captures the citation curve instead of providing a point estimate.	w-index [86]
	Irregular citations/ self-citation	The H-index may be influenced by irregular citations.	The b-index excluding self-citation from H-index values.	b-index [87]
			The H int-index prevents overrating of a citation record earned chiefly by self-citations or by a narrow circle of coworkers.	H int-index [88]
			The h⁽²⁾-index excludes the number of self-citations from the number of citations.	h⁽²⁾-index [84]
			The hs-index takes into account self-citations.	hs-index (sharpened index hs) [89]
			The Hw-index not counting self-citations in the H-index calculation	Hw-index [82]
			The k-index uses only independent citations and excludes self-citations and also accounts only partial citations for each author of multi-authored papers.	k-index [45]
		The H-index considers the same weight to self-citation.	The P-Rank indicator includes self-citations but considers a lower weight for them.	P-Rank [90]
		The H-index considers the same weight to co-terminal citations.	The f-index minimises the impact of co-terminal citations.	f-index [83]
		The H-index considers the same impact for new collaborative citations.	The Robust H-index (rhi-index) minimises the impact of new collaborative citations, maintaining the importance of their citations previous to their collaborative work.	Robust H-index [48]
		The h-index does not Consider honorary authorship.	The H-index (hbar) discourages honorary authorship to more prominent/senior researchers.	H-index hbar [91]
	Insensitivity to citation patterns	The H-index assigns the same index value to two different authors with diverse citation pattern.	The DS-index considers every small change in the citation pattern of authors’ publications.	DS-index [4]
Academic age	Insensitivity to different scientific age	The H-index is strongly dependent on the scientific age of a scientist.	The normalised H-index considers the age of someone still at the beginning of his or her career as a reason for few publications.	Normalised H-index [49]
			The GG-index helps mitigate the bias against researchers with long-term breaks for maternity, childcare, and other personal reasons.	GG-index [92]
			The Productivity index (Pr-index) can compare scientists of different age groups.	Productivity index (Pr-index)
			The trend H-index reveals the significant young scientists and trendsetters.	Trend H-index [49]
			The H-index sequence and the H-index matrix make the scientists at different scientific age comparable.	H-index sequenceH-index matrix [93]
			The FWCI can evaluate the scientific visibility and impact of researchers of any age and science field.	FWCI (Scopus)
	Insensitivity to citation age	The original H-index assigns the same importance to all citations, no matter what their age.	The A-index can increase the same as citation counts increase.	A-index [52]
			The HPD-index (H-index per decade) can be applied to compare the citation records of scientists in different ages.	HPD-index h-index per decade [72]
	Insensitivity to publication age	The H-index does not consider age of publications.	The AR-index takes the age of publications into account.	AR-index [71]
			The contemporary H-index considers career length or age of the publications.	Contemporary H-index [49]

			The t-index uses Shannon entropy and yearly average of H-index.	t-index [94]
			The L-index depends on the age of publications.	L-index [95]
			The m parameter is the result of dividing h by the scientific age of a scientist to consider the lifetime of an individual scientist’s scientific work.	m quotient or m parameter [11]
			The mh-index considers the age of each published paper.	mh-index (Multiple H-index) [30]
Author credit allocation	Counting method of authorship score	The H-index gives full credit of citations to all scholars.	The group of scholars is rarely contributed equally in writing an article.	[96,97]
			In Ab-index (An Absolute Index), the first author and the corresponding author get equal credit if both of them are different while the rest of the co-authors get credit in a decreasing arithmetic progression.	[21]
			The Y-index considers the first author and corresponding author positions into account.	Y-index [20]
			The h-maj considers multi-authorship which includes only first-authored papers and papers for which a scientist has been tagged as the corresponding author.	h-maj: major contribution h-index [77]
			The H α-index counts only those papers where the scientist is the leading author. First authors are given extra credit, last authors are given extra credit, corresponding authors are given extra credit, individual contributions are considered as described by the authors themselves.	Hα-index [98]
			One method considers fractional citation counts where for each m-authored paper, a citation count of y is divided by m. The corresponding g-index is denoted by gf.	gf-index [51](fractional h and g indices)
			The hf-index considers only citation count is fractioned by the number of authors of the paper.	hf-index [51](fractional h and g indices)
			The hm-index counts each paper only fractionally according to (the inverse of) the number of authors.	hm-index [77]
			The author’s golden productivity index considers the principal role of the first authors in paper writing.	p-index (golden productivity index)
			This index considers the first-authored papers.	H-index based on the first-authored papers [99]
			The Hi-index divides H by the average number of researchers in the publications of the Hirsch core.	Hi-index [44]
			The Pure H-index takes the actual number of co-authors, and the scientist’s relative position in the byline into account.	Pure H-index pure r-index [100]
			The Adapted pure H-index divide by the square root of the (equivalent) number of authors. Taking a square root reduces the influence of mega-authored articles on an author’s h_ap-index.	Adapted pure H-index [101]
	Co-authorship issues	The H-index does not differentiate between authors that have very different co-authorship patterns.	The RC-index is an index for quantifying collaboration activities of researchers and scientific communities.	RC-index [6]
			The ħ-index (hbar) uses to discriminate between scientists with different co-authorships patterns.	ħ-index hbar [91]
			The gf-index considers different collaboration patterns and is sensitive to papers with large collaborations.	gf-index (fractional g-index and hf-index) (fractional H-index) [51]
		The H-index does not consider the number of co-authors of each paper.	The ħ-index only counts the papers that contribute to the H-index of all its co-authors.	ħ-index hbar [91]
Variety of fields	Insensitivity to differences between fields	The H-index is strongly field-dependent. An H-index in a field or subfield is often not comparable with H-indices in other fields or subfields.	The FWCI is useful to evaluate the scientific visibility and impact of researchers of any age and science field.	FWCI [28]
			The hi-index takes into account field dependence.	hi-index [44]
			The hs-index can compare the impact of scholars across scientific disciplines.	hs-index (sharpened index hs) [89]
			The hu-index can assess cross-disciplinary researchers.	hu-index [2]
			The hα-index allows for a fairer comparison of scientists from different fields than the H-index.	Hα-index [98]
			The IQP-index reflects both the quality of an author’s work in his or her top three fields and his or her total productivity.	IQP-index [70]
			The normalised H-index normalises H to compare scientist’s achievement based across fields.	Normalised H-index [49]
			The x-index developed to meet three requirements: reflect both publication quantity and quality; modest data extraction and processing effort; ability to compare diverse scientific field profiles.	x-index [29]
	Average H of the authors in the same discipline	The H-index is inapplicable to comparing individuals from different scientific disciplines because it does not consider the difference in collaboration patterns and citation patterns of disciplines.	The h_r-index normalises a researcher’s H-index by introducing the average or maximum H-index of his or her discipline.	[102]
			The hs-index normalises the H-index by the average H of the authors in the same discipline.	hs-index (sharpened index hs) [89]
			The hu-index uses equal fractional counting method that is used to calculate the normalised citation count and paper count.	hu-index [2]gu-index [2]
Mathematical calculation	Natural numbers instead of real numbers	The H-index composed of natural numbers has a much lower potential than a set of real numbers.	Use real numbers to compare researchers.	[10,44,59,103]
		The H-index produces a single natural number that affects the discriminative power of H-index when comparing two scholars with the same H-index value.	The RP-Index and the CP-Index use real numbers, which allows a more detailed evaluation than existing indices.	[103]The RP-Index and the CP-Index [42]

4. Discussion and conclusions

There are many scientometrics indicators to evaluate the scientific production of researchers. These indicators claim to examine the quantity and quality of a researcher’s scientific production [6,24,29,69]. However, the main weakness of all existing indicators is that they consider one or more specific components for measurement and are not comprehensive. Some review studies have categorised the existing indicators into different groups of quantitative, qualitative, combined, supplementary H and adjusted indicators based on the scientific age of the researcher and the like [10]. In some cases, they have pointed out the strengths and weaknesses of each index, and how to calculate and use it in research evaluation [22,24]. In this study, while examining the strengths and weaknesses of the H-index, using the solutions provided in the H-based indices [8,16,32,47], the authors extracted, introduced and proposed the effective components in compiling a comprehensive index to evaluate the scientific productions of researchers. These proposed components are minimum necessary for compiling a comprehensive index.

In a general summary, the advantages of the H-index can be divided into three general categories: (1) quality features; (2) simplicity and (3) suitability. The details of these three categories are presented in Table 3 in the ‘Results’ section. Therefore, in developing an index, it is necessary to pay attention to all the strengths in these three general categories (Table 3).

The disadvantages and shortcomings of the H-index can also be divided into six general categories: (1) Insensitivity to Publication Types; (2) Insensitivity to Citation Patterns; (3) Insensitivity to Academic Age; (4) Insensitivity to Author Credit Allocation; (5) Insensitivity to type of fields and (6) Mathematical Calculation. The details of each of these six categories are presented in Table 4 in the ‘Results’ section. Therefore, it is necessary to avoid all the weak points in these six general categories in developing a comprehensive index (Table 4).

Also, the H-based indices have proposed solutions to solve the deficiencies of the H-index. For example, the H-index only calculates scientific productions that have citations and ignores a significant amount of published scientific productions without citations and unpublished ones [4]. Therefore, the H-index alone is not able to evaluate the comprehensive performance of a researcher. Another shortcoming of the H-index is its inability to distinguish between authors who have very different co-authorship patterns [6,91,104,105]. Since, in different subjects, the number of authors and their level of co-authorship are different, assigning credit to the contributors of research-h work is a challenging issue [20,21,77,98,106]. In some evidence, it has been suggested that the way to assign credit to each researcher is better to be based on the citation pattern of that subject area and the position of the researcher in the research work, and maybe some other factors [99]. The scientific age of the researcher [49,93], the scientific age of scientific productions [11,30,49,71,95], the received citation age [52,72], the citing resource and its credibility level [90], the frequency of citations in that resource [45,71,80,81], the type of source [90,107] and its affect in the scientific community and its affect in the scientific community should also be considered in developing an index. The issues related to the type of citation to distinguish true citation from false citation [91], self-citation [45,82,84,88,89,108,109] and other types of citation should also be considered in the evaluation of a researcher, although these are a bit difficult to calculate. Paying attention to the differences between field [2,29,44,49,70,89,98] citation patterns [4] and group co-authorship [44,101] of each field is also an important issue that has always been criticised by researchers.

Finally, by integrating the strengths of the H-index with the proposed solutions of H-based indices (Tables 3 and 4), the minimum components that need to be considered in developing a comprehensive author-level index can be proposed as follows (Figure 2). The first three categories are currently observed in the H-index and H-based indices, but other cases have been neglected, and it is hoped that all these cases will be included in a new index, so that researchers can be evaluated more realistically, and with less error.

Figure 2.

The minimum set of suggested components for developing a comprehensive author-level index.

Supplemental Material

sj-docx-1-jis-10.1177_01655515241293761 – Supplemental material for H-index and research evaluation: A suggested set of components for developing a comprehensive author-level index

Supplemental material, sj-docx-1-jis-10.1177_01655515241293761 for H-index and research evaluation: A suggested set of components for developing a comprehensive author-level index by Ali Norouzi, Parastoo Parsaei-Mohammadi, Firoozeh Zare-Farashbandi, Elahe Zare-Farashbandi and Ehsan Geraei in Journal of Information Science

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Isfahan University of Medical Sciences with Scientific code 3400516 and ethical code IR.MUI.NUREMA.REC.1400.110.

ORCID iD

Ehsan Geraei

Data availability

Data are available.

Supplemental material

Supplemental material for this article is available online.

References

Marchant

Score-based bibliometric rankings of authors. J Am Soc Inf Sci Technol 2009; 60: 1132–1137.

Shen

Bao

, et al. Hu -index: a unified index to quantify individuals across disciplines. Scientometrics 2021; 126: 3209–3226.

Wan

Liu

WL-index: leveraging citation mention number to quantify an individual’s scientific impact. J Assoc Inf Sci Technol 2014; 65: 2509–2517.

Farooq

Khan

Iqbal

, et al. DS-index: ranking authors distinctively in an academic network. IEEE Access 2017; 5: 19588–19596.

Pradhan

Paul

Maheswari

, et al. C3 -index: a PageRank based multi-faceted metric for authors’ performance measurement. Scientometrics 2017; 110(1): 253–273.

Abbasi

Altmann

Hwang

Evaluating scholars based on their academic collaboration activities: two indices, the RC-index and the CC-index, for quantifying collaboration activities of researchers and scientific communities. Scientometrics 2010; 83: 1–13.

Salman

Ahmed

Afzal

MT.

Assessment of author ranking indices based on multi-authorship. Scientometrics 2021; 126(5): 4153–4172.

Alonso

Cabrerizo

Herrera-Viedma

, et al. hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics 2010; 82: 391–400.

Abbas

AM.

Weighted indices for evaluating the quality of research with multiple authorship. Scientometrics 2011; 88(1): 107–131.

10.

Bihari

Tripathi

Deepak

A review on h-index and its alternative indices. J Inf Sci 2023; 49(3): 624–665.

11.

Hirsch

JE.

An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 2005; 102: 16569–16572.

12.

Mohammed

Nyantakyi

Morgan

, et al. Use of relative extra citation counts and uncited publications to enhance the discriminatory power of the h-index. Scientometrics 2020; 126(1): 181–199.

13.

Schreiber

Malesios

Psarakis

Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators. J Informetrics 2012; 6(3): 347–358.

14.

Hirsch

Buela-Casal

The meaning of the h-index. Int J Clin Health Psychol 2014; 14(2): 161–164.

15.

Bornmann

Mutz

Neuhaus

, et al. Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci Environ Polit 2008; 8: 93–102.

16.

Egghe

Theory and practise of the g-index. Scientometrics 2006; 69: 131–152.

17.

Paul

Saha

Research rating: some technicalities. Med J Armed Forces India 2018; 78: S24–S30.

18.

Bornmann

Mutz

Hug

, et al. A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. J Informetrics 2011; 5: 346–359.

19.

Alonso

Cabrerizo

Herrera-Viedma

, et al. h-Index: a review focused in its variants, computation and standardization for different scientific fields. J Informetrics 2009; 3: 273–289.

20.

H-Z

Y-S.

Top cited articles in adsorption research using Y-index. Res Eval 2014; 23(1): 12–20.

21.

Biswal

AK.

An absolute index (Ab-index) to measure a researcher’s useful contributions and productivity. PLoS ONE 2013; 8: e84334.

22.

Bornmann

Mutz

Daniel

HD.

Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. J Am Soc Inf Sci Technol 2008; 59: 830–837.

23.

Costas

Bordons

The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level. J Informetrics 2007; 1: 193–203.

24.

Mohammed

Morgan

Nyantakyi

EK.

On the influence of uncited publications on a researcher’s h -index. Scientometrics 2020; 122(3): 1791–1799.

25.

Buela-Casal

Scientific journal impact indexes and indicators for measuring researchers’ performance. Rev Psicodidact 2010; 15: 3–19.

26.

Okubo

Bibliometric indicators and analysis of research systems: methods and examples. Paris: OECD Publishing, 1997.

27.

Hagen

NT.

Harmonic publication and citation counting: sharing authorship credit equitably – not equally, geometrically or arithmetically. Scientometrics 2010; 84(3): 785–793.

28.

Zanotto

Carvalho

Article age-and field-normalized tools to evaluate scientific impact and momentum. Scientometrics 2021; 126: 2865–2883.

29.

Claro

Costa

CAV

. A made-to-measure indicator for cross-disciplinary bibliometric ranking of researchers performance. Scientometrics 2011; 86: 113–123.

30.

Yaminfirooz

Gholinia

Multiple h-index: a new scientometric indicator. Electron Libr 2015; 33: 547–556.

31.

Bihari

Tripathi

Akshay

, et al. h-index and its alternative: a review. arXiv 2018.

32.

Norris

Oppenheim

The h-index: a broad review of a new bibliometric indicator. J Doc 2010; 66: 681–705.

33.

Rubem

APdS

de Moura

Soares de Mello

JCCB

. Comparative analysis of some individual bibliometric indices when applied to groups of researchers. Scientometrics 2015; 102: 1019–1035.

34.

Wildgaard

A critical cluster analysis of 44 indicators of author-level performance. J Informetrics 2016; 10: 1055–1078.

35.

Wildgaard

Schneider

Larsen

A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics 2014; 101: 125–158.

36.

Huang

Chi

PS.

A comparative analysis of the application of h-index, g-index, and a-index in institutional-level research evaluation. J Libr Inf Stud 2010; 8(2): 1–10.

37.

González-Valiente

Pacheco-Mendoza

Arencibia-Jorge

A review of altmetrics as an emerging discipline for research evaluation. Learn Publ 2016; 29: 229–238.

38.

Tahamtan

Safipour Afshar

Ahamdzadeh

Factors affecting number of citations: a comprehensive review of the literature. Scientometrics 2016; 107: 1195–1225.

39.

Wani

Zainab

A review of eminence of scientometric indicators in scientific research productivity. COLLNET J Scientomet Inf Manage 2017; 11: 273–285.

40.

Lydon

Madden

De Bhulbh

, et al. Do gender-based disparities in authorship and editorship exist in healthcare simulation journals? A bibliometric review of the research. Simul Healthc 2021; 16: 136–141.

41.

Lee

Exploring the determinants of research performance for early-career researchers: a literature review. Scientometrics 2024; 129(1): 181–235.

42.

Altmann

Abbasi

Hwang

. Evaluating the productivity of researchers and their communities: the RP-index and the CP-index. International Journal of Computer Science and Applications 2009; 6(1): 104–118.

43.

Anderson

Hankin

Killworth

Beyond the Durfee square: enhancing the h-index to score total publication output. Scientometrics 2008; 76(3): 577–588.

44.

Batista

Campiteli

Kinouchi

Is it possible to compare researchers with different scientific interests?

Scientometrics 2006; 68(1): 179–189.

45.

Kaptay

The k-index is introduced to replace the h-index to evaluate better the scientific excellence of individuals. Heliyon 2020; 6: e04415.

46.

Cabrerizo

Alonso

Herrera-Viedma

, et al. q2-Index: quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core. J Informetrics 2010; 4(1): 23–28.

47.

Egghe

An improvement of the h-index: the g-index. ISSI Newslett 2006; 2: 8–9.

48.

Poirrier

Moreno

Huerta-Cánepa

Robust h-index. Scientometrics 2021; 126: 1969–1981.

49.

Sidiropoulos

Katsaros

Manolopoulos

Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics 2007; 72(2): 253–280.

50.

Costas

van Leeuwen

Bordons

Self-citations at the meso and individual levels: effects of different calculation methods. Scientometrics 2010; 82: 517–537.

51.

Egghe

Mathematical theory of the h- and g-index in case of fractional counting of authorship. J Am Soc Inf Sci Technol 2008; 59: 1608–1616.

52.

Jin

H-index: an evaluation indicator proposed by scientist. Sci Focus 2006; 1: 8–9.

53.

Panaretos

Malesios

Assessing scientific research performance and impact with single indices. Scientometrics 2009; 81(3): 635–670.

54.

van Eck

Waltman

. Generalizing the h- and g-indices. J Informetrics 2008; 2: 263–271.

55.

Vinkler

The π-index: a new indicator for assessing scientific impact. J Inf Sci 2009; 35: 602–612.

56.

Vinkler

The π -index: a new indicator. J Inf Sci 2009; 35: 602–612.

57.

Schreiber

Self-citation corrections for the Hirsch index. Europhys Lett 2007; 78(3): 30002.

58.

Vanclay

JK.

On the robustness of the h-index. J Am Soc Inf Sci Technol 2007; 58(10): 1547–1550.

59.

Zhang

CT.

The e-index, complementing the h-index for excess citations. PLoS ONE 2009; 4: e5429.

60.

García-Pérez

MA.

A multidimensional extension to Hirsch’s h-index. Scientometrics 2009; 81(3): 779–785.

61.

Schreiber

Restricting the h-index to a publication and citation time window: a case study of a timed Hirsch index. J Informetrics 2015; 9(1): 150–155.

62.

Todeschini

The j-index: a new bibliometric index and multivariate comparisons between other common indices. Scientometrics 2011; 87(3): 621–639.

63.

Haley

McGee

MK.

A flexible functional method for jointly valuing journal visibility and author citation count. Scientometrics 2023; 128(6): 3337–3346.

64.

Rodríguez-Navarro

Brito

Rank analysis of most cited publications, a new approach for research assessments. J Informetrics 2024; 18(2): 101503.

65.

Aliguliyev

Adigozalova

(eds). ht-index and At-index for evaluating scientific performance of researchers. In: 13th IEEE international conference on application of information and communication technologies, AICT 2019 – proceedings, Baku, Azerbaijan, 23–25 October 2019.

66.

Costas

Bordons

A critical view of the h-index: observations based on a practical application. Prof Inf 2007; 16: 427–432.

67.

The w-index: a significant improvement of the h-index. J Am Soc Inf Sci Technol 2008; 10: 609–614.

68.

Imperial

Rodríguez-Navarro

Usefulness of Hirsch’sh-index to evaluate scientific research in Spain. Scientometrics 2007; 71: 271–282.

69.

Qiu

Cheng

New exploratory work of evaluating a researcher’s output. Scientometrics 2008; 77: 335–344.

70.

Antonakis

Lalive

Quantifying scholarly impact: IQp versus the Hirsch h. J Am Soc Inf Sci Technol 2008; 59: 956–969.

71.

Jin

Liang

Rousseau

, et al. The R- and AR-indices: complementing the h-index. Chin Sci Bull 2007; 52: 855–863.

72.

Kosmulski

New seniority-independent Hirsch-type index. J Informetrics 2009; 3: 341–347.

73.

Navon

The h-i index: a proposed new metric of individual scientific output. Cybermetrics 2009; 13(1): 1–4.

74.

The w-index: a measure to assess scientific impact by focusing on widely cited papers. J Am Soc Inf Sci Technol 2010; 61: 609–614.

75.

García-Pérez

MA.

An extension of the h index that covers the tail and the top of the citation curve and allows ranking researchers with similar h. J Informetrics 2012; 6(4): 689–699.

76.

García-Pérez

MA.

Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google scholar: a case study for the computation of h indices in psychology. J Am Soc Inf Sci Technol 2010; 61: 2070–2085.

77.

Schreiber

A case study of the modified Hirsch index hm accounting for multiple coauthors. J Am Soc Inf Sci Technol 2009; 60: 1274–1282.

78.

Y-S.

Top-cited articles in chemical engineering in science citation index expanded: a bibliometric analysis. Chin J Chem Eng 2012; 20(3): 478–488.

79.

Assimakis

Adam

A new author’s productivity index: p-index. Scientometrics 2010; 85: 415–427.

80.

Kosmulski

. MAXPROD-a new index for assessment of the scientific output of an individual, and a comparison with the h-index. Cybermetrics 2007(11): 5.

81.

Ajiferuke

Wolfram

Citer analysis as a measure of research impact: library and information science as a case study. Scientometrics 2010; 83: 623–638.

82.

Egghe

Rousseau

An h-index weighted by citation impact. Inf Process Manage 2008; 44: 770–780.

83.

Katsaros

Akritidis

Bozanis

The f index: quantifying the impact of coterminal citations on scientists’ ranking. J Am Soc Inf Sci Technol 2009; 60: 1051–1056.

84.

Kosmulski

A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newslett 2006; 2: 4–6.

85.

Lathabai

HH.

ψ-index: a new overall productivity index for actors of science and technology. J Informetrics 2020; 14: 101096.

86.

Wohlin

A new index for the citation curve of researchers. Scientometrics 2009; 81: 521–533.

87.

Brown

RJ.

A simple method for excluding self-citation from the h-index: the b-index. Online Inf Rev 2009; 33(6): 1129–1136.

88.

Kosmulski

Hirsch-type index of international recognition. J Informetrics 2010; 4: 351–357.

89.

Kaur

Radicchi

Menczer

Universality of scholarly impact metrics. J Informetrics 2013; 7(4): 924–932.

90.

Yan

Ying

Sugimoto

, et al. P-Rank: an indicator measuring prestige in heterogeneous scholarly networks. J Am Soc Inf Sci Technol 2011; 62: 467–477.

91.

Hirsch

JE.

An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics 2010; 85: 741–754.

92.

Gunthe

Gettu

A new index for assessing faculty research performance in higher educational institutions of emerging economies such as India. Scientometrics 2022; 127(8): 4959–4976.

93.

Liang

H-index sequence and h-index matrix: constructions and applications. Scientometrics 2006; 69(1): 153–159.

94.

Singh

PK.

T-index: entropy based random document and citation analysis using average h-index. Scientometrics 2022; 127(1): 637–660.

95.

Belikov

VV.

A citation-based, author- and age-normalized, logarithmic index for evaluation of individual researchers independently of publication counts. F1000Res 2015; 4: 884.

96.

Bihari

Tripathi

. A new method for key author analysis in research professionals’ collaboration network. In: Chaki

Saeed

Cortesi

, et al. (eds) Advances in intelligent systems and computing. Singapore: Springer, 2017, pp. 133–143.

97.

Burrell

QL.

Hirsch’s h-index: a stochastic model. J Informetrics 2007; 1(1): 16–25.

98.

Hirsch

JE.

h α: an index to quantify an individual’s scientific leadership. Scientometrics 2019; 118: 673–686.

99.

Opthof

Wilde

AA.

The Hirsch-index: a simple, new tool for the assessment of scientific output of individual scientists: the case of Dutch professors in clinical cardiology. Neth Heart J 2009; 17(4): 145–154.

100.

Wan

J-K

Hua

P-H

Rousseau

The pure h-index: calculating an author’s h-index by taking co-authors into account. COLLNET J Sci Inf Manage 2007; 1(2): 1–5.

101.

Chai

Hua

Rousseau

, et al. The adapted pure h-index. In: Proceedings of WIS 2008. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting Humboldt-Universität zu Berlin, Institute for Library and Information Science (IBI)

102.

Mazurek

A modification to Hirsch index allowing comparisons across different scientific fields. arXiv:170305485 2017.

103.

Tol

RSJ

. A rational, successive g-index applied to economics departments in Ireland. J Informetrics 2008; 2(2): 149–155.

104.

Vinkler

Impact of the number and rank of coauthors on h-index and π-index. The part-impact method. Scientometrics 2023; 128(4): 2349–2369.

105.

Tzitzikas

Dovas

How co-authorship affects the H-index?

Scientometrics 2024; 129: 4437–4469.

106.

HH.

Four problems of the h-index for assessing the research productivity and impact of individual authors. Scientometrics 2023; 128(5): 2677–2691.

107.

Chen

Koch

Zakiyeva

, et al. Article’s scientific prestige: measuring the impact of individual articles in the web of science. J Informetrics 2023; 17(1): 101379.

108.

Bornmann

Marx

Schier

, et al. Convergent validity of bibliometric Google Scholar data in the field of chemistry-citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. J Informetrics 2009; 3: 27–35.

109.

Vercelli

Pellicciari

Croci

, et al. Self-citation behavior within the health allied professions’ scientific sector in Italy: a bibliometric analysis. Scientometrics 2023; 128(2): 1205–1217.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB