Abstract
Background:
Citation-based indexes may objectively measure scholarly production, and previous studies have shown these to be superior predictors of scientific achievement and academic rank than more traditional measures. However, absolute values of these indexes are discipline specific. Therefore, the aim of the current study was to define normative values of the h-index and m-quotient for academic pediatric pulmonologists.
Methods:
A sample of 85 pediatric pulmonologists were examined from 10 large academic children's hospitals, chosen from the top 50 U.S. News rankings of children's hospitals in pulmonary departments. H-index and m-quotient were tabulated using Web of Science.
Results:
Both h-index and m-quotient were significantly associated with academic rank (p<0.001). Median and interquartile ranges were as follows: assistant professor (h-index 1 [1–4], m-quotient 0.25 [0.04–0.50]), associate professor (h-index 10 [4–12], m-quotient 0.57 [0.38–0.66]), and full professor (h-index 23 [12–33], m-quotient 0.88 [0.47–1.07]). U.S. News ranking correlated with both h-index (ρ=−0.21, p=0.04) and m-quotient (ρ=−0.24, p=0.02), with better rank associated with increased scholarly productivity. Female faculty had lower h-indexes (p=0.002) but not significantly lower m-quotients (p=0.09), suggesting the sex difference was largely due to academic age.
Conclusions:
These data may serve as objective benchmarks of scholarly productivity for academic pediatric pulmonologists. Within this specialty, h-index levels of 4 and 12 identify cutoff points between assistant/associate and associate/full professors, respectively.
Introduction
T
One of the more recent and widely used such bibliometric indexes is the Hirsch index (h-index), 2 which is defined as the number of papers with citation number ≥h. For instance, if an individual has published 10 papers, four of which have been cited at least four times, and the other six cited fewer than four times, their h-index would be 4. The basic idea is that this metric aims to capture both quantity and quality of an individual's published scholarly work. The h-index has been found to predict future scientific achievement better than other measures such as total citation count, citations per paper, and total paper count. 3 In addition, previous investigations have demonstrated a link between h-index and academic rank within medical education, 4 plastic surgery, 5 radiation oncology, 6 neurosurgery,7,8 and general surgery. 9 Importantly, the h-index also appears to be specialty specific. 10 Hirsch also developed the m-quotient, which takes into account an individual's “academic age.” 2 Therefore, the purpose of the present investigation was to examine the h-index and m-quotient among academic pediatric pulmonary department faculty in order to provide reference values by academic rank.
Methods
Sample
Pediatric pulmonary departments were identified via U.S. News and World Report's Best Children's Hospitals 2014–2015 rankings of best pediatric pulmonary departments. 11 While all of the top 50 departments on the list are certainly of high academic quality, 10 departments were chosen based on equally spaced rank on the list in order to provide a range of department size and likely research emphasis: #1 Children's Hospital of Philadelphia, #5 Children's Hospital Colorado, #10 Lucile Packard Children's Hospital at Stanford, #15 North Carolina Children's Hospital at UNC, #20 Children's Healthcare of Atlanta, #25 Monroe Carell Children's Hospital at Vanderbilt, #30 Mount Sinai Kravis Children's Hospital, #35 Duke Children's Hospital, #40 Nemours Alfred duPont Hospital for Children, and #45 UCSF Benioff Children's Hospital. Individual faculty were identified via their respective children's hospital or affiliated academic institution online pediatric pulmonary faculty listings. Faculty were included if they had an MD degree or equivalent (e.g., not nurse practitioners or solely PhD) and had an identified academic rank of either assistant, associate, or full professor. Clinical instructors and emeritus professors were not included.
Measures
For each individual faculty, the h-index was determined from the citation database, Web of Science. Searches were restricted to the Web of Science Core Collection, from years 1974 to 2015, in English language, and to articles (rather than abstracts, books, book chapters, etc.). The author's last name and first initial were entered for search. In addition, in order to increase specificity for the author of interest, the city of their corresponding institution was also included in the search; city was chosen over institution name because a single institution may be referred to in multiple different ways (e.g., Children's Hospital Philadelphia vs. University of Pennsylvania), but the city remains constant. In cases where searches yielded what appeared to be unrelated fields or dates of publications outside of the expected range for faculty age, specificity was further increased by including some combination of the middle initial of the faculty member, restricting the category to “pediatrics,” and/or by individual article review. The h-index was then recorded. The m-quotient was calculated by dividing the h-index by the number of years elapsed since the faculty member's first paper was published. For each faculty, the sex, hospital affiliation, and academic rank were also recorded.
Statistics
Measures were summarized with descriptive statistics. The h-index, m-quotient, and U.S. News ranking were treated as ordinal variables, and appropriate statistical tests were employed. Specifically, descriptive statistics included median and interquartile range, correlations were performed with Spearman's rank-order correlation test, and between-group differences were assessed with the Kruskal–Wallis rank sum test. All analyses were performed in R. Because this study did not involve interactions with human subjects and was a secondary analysis of publicly available data, Institutional Review Board approval was not required.
Results
A total of 85 faculty members were included in the analysis, of which 33 were assistant professors, 29 associate professors, and 23 full professors. The sample had a substantial number of members of each sex, with 35 women and 50 men. The overall medians (interquartile range) for h-index and m-quotient were 7 (1–14) and 0.50 (0.25–0.86), respectively.
H-index and m-quotient values by academic rank are presented in Table 1 and Figure 1. Rank was significantly associated with h-index (H=39.05, df=2, p<0.001) and m-quotient (H=18.68, df=2, p<0.001). U.S. News ranking correlated with both h-index (ρ=−0.21, p=0.04) and m-quotient (ρ=−0.24, p=0.02), and both indexes correlated with each other (ρ=0.75, p<0.001). Female faculty had lower h-indexes (median 3 vs. 11, H=8.81, df=1, p=0.002) but not significantly lower m-quotients (median 0.41 vs. 0.57, H=2.83, df=1, p=0.09), related to a later year of first publication (median year 2003 vs. 1992, H=11.10, df=1, p<0.001). Sex was not related to U.S. News ranking (median 10 vs. 12.5, H=1.12, df=1, p=0.28).
Graphs represent distribution of h-index (left) and m-quotient (right) among faculty by academic rank. Bins represent quintiles of the entire sample. Color images available online at www.liebertpub.com/ped
Indexes are expressed as median (interquartile range).
Discussion
Scholarly productivity is an important and commonly used metric in faculty assessment and promotion. While multiple previous reports have found a relationship between bibliometric indexes and academic rank, this has never been examined within pediatric pulmonology to the best of the author's knowledge. These results fill that void in the literature and support an association between two bibliometric metrics of scholarly productivity, the h-index and m-quotient, and academic rank within pediatric pulmonology. In addition, these data provide reference ranges for the h-index and m-quotient based on a sample of faculty from a range of large academic institutions in the field. The approximate h-index cutoff points between academic ranks based on these data appear to be 4 (between assistant and associate professor) and 12 (between associate and full professor); for the m-quotient, possible cutoff points are 0.3 (between assistant and associated professor) and 0.7 (between associated and full professor), although there was more overlap compared to the h-index.
These data are largely in agreement with previous studies of bibliometric indexes in other medical specialties. The current reported absolute and relative cutoff points for academic rank are within reasonable range of those reported for other medical specialties, although there is a wide range,10,12 highlighting the specialty-specific nature of bibliometric measures. The current finding of those institutions with better U.S. News ranking also having somewhat higher bibliometric indexes is consistent with previous studies. 13 Although women had lower h-indexes than men did in the current study, this was likely related to younger “academic age,” as the m-quotient did not differ between sexes. This same finding was reported in an analysis of academic neurosurgeons. 8
These results have important real-world implications. The reported h-index and m-quotient may serve as reference ranges for individual faculty member self-assessment or be used as a case for promotion within a department. Conversely, division chairs may use these as benchmarks for their faculty and identify those faculty that are more or less scholarly productive. Presumably, the level of productivity should be commensurate with that individual's allocated time for research. These results also highlight the difference between h-index, which represents cumulative productivity, and m-quotient, which measures productivity over a given time period. In colloquial terms, an accomplished faculty may have an impressive h-index, but if they rest on their laurels, their m-quotient will suffer.
Certainly, any one metric—citation-based or otherwise—cannot encapsulate a faculty member's academic productivity or value to an institution. Therefore, it should be stressed that while citation-based metrics can provide valuable objective information, they are only a narrow measure of academic productivity and cannot supplant the inherently subjective assessment of any one individual's complete body of work. Furthermore, it should be emphasized that there is likely variability in the emphasis that divisions place on various factors for promotion readiness, such as teaching, clinical care, academic administration, and participation in multicenter clinical trials. So, different thresholds of bibliometric achievement may be appropriate, depending on the practice setting and institution.
Despite these important findings, the current study has several limitations. First, both bibliometric indexes examined do not account for any self-citation. Nevertheless, in an analysis of 22 different bibliometric indexes, Ruscio et al. found that h-index and m-quotient are robust to self-citation as well as outliers. 14 Second, as with any other bibliometric analysis, author misidentification is a potential problem. In the current study, multiple safeguards were employed (city of origin, review of outliers, selective inclusion of middle initial or category) with the goal of increasing specificity at the potential cost of missing articles. Certainly, any search strategy will balance sensitivity and specificity, and the strategy employed in the current study seemed to have a good balance. The optimal solution to author identification would be for authors to self-identify with tools such as Google Scholar. Third, because the faculty were sampled from large academic children's hospitals, the current results are likely not generalizable to less research-oriented institutions. That said, those institutions may place less emphasis on scholarly productivity for their faculty. Fourth, in the current analysis, distinctions were not made between different academic tracks, such as clinical, educational, administrative, and research. Certainly, different faculty vary in the amount of time specified for research versus clinical and other service duties. However, the small size of the field would make that analysis difficult. Fifth, the h-index and m-quotient do not account for author order and shared authorship. As discussed by Ruscio et al., while it may be intuitively appealing to adjust for this, it is practically difficult to do and, in their analysis, had minimal impact on author bibliometric indexes.
In conclusion, these data provide h-index and m-quotient reference ranges for academic pediatric pulmonologists. The hope is that these results may serve as objective benchmarks of scholarly productivity by which individual faculty members may be judged.
Footnotes
Author Disclosure Statement
No competing financial interests exist.