Research diversification and its relationship with publication counts and impact: A case study based on Australian professors

Abstract

This research aims to investigate whether multi/inter-disciplinary research activities are related to research impact and publication counts of scholars. Since researchers with very high levels of multi/inter-disciplinarity might be able to target complex problems, we would expect them to receive more credits than their colleagues with a stronger disciplinary orientation. We analysed Web of Science (WoS) indexed publications of all associate and full professors from a random sample of Australian universities in physics, chemistry and biology (1980–2014). Australian Fields of Research (FoR) codes assigned to journals were used to calculate the diversification of authors’ publications. The number of citations in the first 3 years, number of 10% most frequently cited papers, and citation impact percentile were used for impact assessment. A few indicators were used to measure the diversity including ‘extent of diversification (ED)’ (number of distinct FoR codes divided by the number of publications) and ‘diversification ratio (DR)’ (ratio of the publications falling outside the dominant code to the total number of publications). A total of 47.76% of biologists’ publications, 35.23% of physicists’ publications and 20.36% of chemists’ publications were published in journals assigned to fields other than the Australian associate and full professors’ fields. Publications from biologists had the largest values of diversification. Women (compared with men) and associate professors (compared with full professors) in chemistry, biology and overall were more probably to publish diversely. ED was negatively correlated with output and citation impact. DR also had a negative but weak correlation with the number of publications and 10% most frequently cited paper.

Keywords

Australia biology chemistry citation diversification diversity field of research gender inter-disciplinary journal classification multidisciplinary physics rank research research impact specialisation

1. Introduction

Researchers may choose different pathways in their career focusing on one or a few research topics or areas, or they might diversify their research activities. While concentration of research activities can result in accumulation of great expertise, diversification may result in transfer of knowledge and techniques across specialisations that can help to solve complex problems. Diversification of research has gained increasing attention in recent years. Diversification or changing patterns of scientific activity has also been discussed in the context of interdisciplinarity [1]. Interdisciplinarity can be considered as communication or work across academic disciplines [2]. Inter-disciplinary research activities have been promoted during the last decade [3] and there is an assumption that inter-disciplinary research results in better outcomes and higher citation impacts [2,4] more scientific breakthroughs [5]. However, the existing evidence about the relationship between interdisciplinarity or diversification and publication output or citation impact is not conclusive.

A comparison of some past studies in Yegros-Yegros et al. [6] shows that they found different results. It is difficult to draw a conclusion about the relationship between citation impact and diversification, and this requires further research. Therefore, we address this challenge and aim to answer the following research question: Does researchers’ diversification have any effect on their overall research output and citation impact? It is believed that solving complex problems require inter-disciplinary work [7,8]. Therefore, we might expect that researchers with high levels of multi/inter-disciplinarity should be able to address complex real-world problems, and therefore might receive more credits (publication success and wide reception) than their colleagues with a stronger disciplinary orientation.

2. Literature review

In the literature, the terms ‘inter-disciplinarity’ and ‘multi-disciplinarity’ have been used interchangeably. Among very few studies that distinguish these terms, Carayol and Thi [9] and Choi and Pak [10] consider inter-disciplinarity for knowledge or outcome (e.g. publications) based on research from different disciplines and multi-disciplinarity for teams from different disciplines working together towards the outcome.

Here, we first review the existing literature on research diversity and/or inter-/multi-disciplinarity analysing its different levels of analysis and then present and discuss proposed indicators for measuring research diversity in the literature. The review focuses on the research that has used the output/outcome for measuring diversity (as opposed to research team composition). There are also studies that measure diversity or inter-disciplinarity by looking at the disciplinary compositions of grants or projects. For instance, a study of cross-disciplinarity of grant proposals [11] showed that mono-disciplinary projects were very well represented in physics/astronomy; Bromham et al’.s [12] study of Australian national research grants showed that inter-disciplinary research had consistently lower funding success.

2.1. Studies focussing on different levels of analysis for research diversity

2.1.1. Article-level

Rinia et al. [13] considered the percentage of articles for any given field published in journals not classified in that field as the degree of inter-disciplinarity of the article. They looked at physics research in the Netherlands and found that 36% of publications by physicists were not published in physics journals.

Levitt and Thelwall [14] analysed the association between the number of subject categories (SCs) assigned to journals and citation impact of articles. Their results showed that the citation rate of mono-category articles (single subject journals) was roughly twice that of multi-category articles (multi-disciplinary journals) in life sciences, health sciences and physical sciences. No difference was found for social sciences. They concluded that one could not assume that in general, multi-disciplinary research will be more highly cited, and the converse is probably true for many areas of science. In contrast to this research, Larivière and Gingras [15] found that articles with a higher level of inter-disciplinarity (operationalised as the percentage of their citations to journals of other disciplines) in some disciplines received higher citation rates, although no clear correlation was found for all disciplines combined. Moreover, they found that highly disciplinary and highly inter-disciplinary articles in all disciplines had a low citation impact. Therefore, they maintained that there might be an optimum of inter-disciplinarity beyond which the research is too dispersed to find its niche and below which it is too mainstream to have high impact.

Yegros-Yegros et al. [6] found, somewhat confirming Larivière and Gingras [15] findings, that all three dimensions of inter-disciplinarity (i.e. variety, balance and disparity) displayed a curvilinear (inverted U-shape) relationship with citation impact. Wang et al. [16] also investigated the same three dimensions of inter-disciplinarity and showed that long-term (13-year) citations: (1) increased at an increasing rate with variety, (2) decreased with balance and (3) increased at a decreasing rate with disparity. Their results also showed that although variety and disparity had positive effects on long-term citations, they had negative effects on short-term (3-year) citations, and that although balance had a negative effect on long-term citations, its negative effect was statistically insignificant in the short run. Enduri et al. [17] in their study of physics using the Weitzman diversity measure showed that in general high diversity papers receive more citations, but too much diversity can reduce their total citations (again a curvilinear relation).

Pan and Katrenko [18] used SCs of journals in which references of articles are published to measure inter-disciplinarity of publications between 2009 and 2013 and found that inter-disciplinary research was associated with a lower citation impact overall, but a higher level of citations in patent applications. They also reported a correlation between inter-disciplinary research and lower levels of international collaboration, which was more prevalent in emerging countries (e.g. Brazil and China).

On the contrary, Chen et al. [19] found that the top 1% most cited papers exhibit higher levels of inter-disciplinarity than papers in other citation rank classes (in more than 90% of National Science Foundation, NSF, specialties). They concluded that inter-disciplinary research plays a more important role in generating high impact knowledge.

2.1.2. Author-level

Leahey et al. [20] analysing 993 US researchers’ publications in Web of Science (WoS) found that while inter-disciplinary research (considering citing diverse WoS SCs) increased cumulative citation counts to researchers’ articles, it reduced their productivity (i.e. number of publications); a 10% increase in inter-disciplinary research boosted citations by 15.7% and reduced productivity by 9.1% over the course of one’s career. They also found that women were slightly more probably than men to engage in inter-disciplinary research and that young scholars engage in inter-disciplinary research more than middle-age counterparts. They also found that both high- and low-status scientists engage in inter-disciplinary research, whereas middle-status scientists hover towards the more normative, disciplinary core.

Chakraborty et al. [1] found that highly cited researchers typically follow a ‘scatter-gather’ policy in choosing areas and lowly cited researchers work on various areas at a time as well as in the entire career. Scatter-gather policy means that researchers focus on one or two areas at any given time but over their career they study various areas.

Abramo, D’Angelo and Di Costa did a series of studies on research diversity using WoS journal classification, Italian scientific disciplinary codes for researchers and three measures of diversity including extent of diversification (ED), intensity of diversification and relatedness ratio (RR) (defined in the ‘Measuring diversity’ section below). Their findings showed disciplinary differences in diversification as the ED was lowest in mathematics and highest in chemistry, intensity of diversification was lowest in earth sciences and highest in industrial and information engineering and the degree of relatedness was lowest in earth sciences and highest in chemistry [21]. They also showed that age, rank and gender made a difference in diversity, for instance, younger scientists appeared more probably to diversify their research activity than older scientists [22]. They found that diversified publications of scientists are often the result of their collaboration with researchers from other disciplines [23] and that at least in a third of cases, inter-disciplinary authored articles gain more impact than others [24] Furthermore, they reported that diversification (i.e. publishing in journals classified in fields other than the field in which an author publishes most of his or her work) at the aggregate level does not pay off in terms of citation impact (measured by the median of citation frequency distribution, and most-cited publications), with some exceptions in individual disciplines [25]

2.1.3. Institution-level

Bourke and Butler [26] looked at diversity of publications between 1981 and 1994 of academic departments in Australia using the Australian and New Zealand Fields of Research (FoR) classifications for journals and found that 70% of publications of university departments in physical sciences, 71% of chemical sciences and 51% of biological sciences were published in journals classified to the same category and the rest were published in journals of other fields. For instance, for physical sciences, 28% of the articles were published in applied sciences and 25% in mathematical sciences; for chemical sciences, 9% were published in applied sciences; and for biological sciences, 19% were published in agricultural sciences. They suggested that evaluation of universities’ research in a field by analysing publications from a department with the same classification can be seriously misleading. A national level study [27] on agricultural sciences showed that R&D if balanced with diversification could improve research performance and eventually productivity in an agricultural sector.

2.2. Measuring diversity

So far, a few measures have been suggested in the literature to measure diversity. These metrics are calculated at different levels (e.g. article, researcher and institution). Here, we present them under the level for which they have been originally proposed but some of them are applicable to other levels. While the majority of indicators rely on subject classification of journals, a few use other data such as citations (e.g. for co-citation analysis).

2.2.1. Author-level metrics

Specialisation: the spread of SCs in which the works of a given author are published in a specified time period [28].

ED: the number of topics (i.e. WoS SCs assigned to journals) covered in an author’s scientific portfolio [21].

RR: the ratio of the number of papers in the dominant discipline (i.e. the discipline in which the researcher publishes most) to the total number of papers [21].

Intensity of diversification or diversification ratio (DR): the share of the researchers’ output falling outside their sector of specialisation [21]. DR can be considered as ‘1 – RR’.

Plain entropy: the entropy of the number of times an author (a) publishes papers in a particular field (i), for all his or her fields. It can be presented by the following equation (1): $H_{p}^{a} (F) = \sum_{i \in F} P_{i} Log (P_{i})$ where $p_{i} = \frac{number of papers writteh by a in field i}{total number of of papers written by a}$ .

Window entropy – (denoted by $H_{w}^{a} (F)$ ): a window of size k slides over the sequence of fields in F. The plain entropy for the sequence contained within that window at each position is calculated. The mean of all these positions is computed to measure the window entropy of the entire sequence as described in the following equation: $H_{w}^{a} (F) = - \frac{1}{n - k + 1} \sum_{i = 1}^{n - k + 1} H_{w}^{a} (w_{i})$ where w_i is the set of fields in the ith sliding window of size k, ranging from i to (i + k − 1). The window entropy indicates the diversity in the selection of fields in short spans of time by considering only previous k fields in the sequence [1]

2.2.2. Article-level metrics

Integration: measures the extent to which a research article cites diverse SCs [28].

Weitzman’s diversity index: the sum of distances from each element to its nearest neighbour. In principle, it characterises the degree of dissimilarities between the elements of a set and can be used whenever a clear metric distance is defined between elements of a set. Physics and Astronomy Classification Scheme (PACS) codes of papers were used as the set for calculations [17]. This can be computed at the author level as well as by combining all PACS codes of papers published by an author as the set.

Variety (N): the distribution of WoS SCs in the references of references of an article. To compute this distribution, a frequency list of the journals in which the references of references were published was constructed and converted into a frequency list of SCs using the SC attribution of each journal as given in the Journal Citation Reports [29].

Shannon H: the indicator uses the same concept of plain entropy (as stated above) but for articles [29].

Simpson diversity I: it is defined as 1 – Simpson Index, which is the commonly used measure of concentration (also known as Herfindahl-Hirschman Index in disciplines such as economics) [29].

Stirling index Δ: the indicator ‘can be understood as a Simpson diversity in which the products of proportions of categories are weighted by distance/similarity’ between categories (fields) [29].

Leinster-Cobbold diversity indices: a diversity measure based on third generation of biodiversity measures (the effective numbers) that takes into account the similarity/distance of elements in a system (i.e. references of an article) [30].

2.2.3. Institution-level metrics

Inter-disciplinary profile of the institution’s research: percentage of its publications in each SC [31].

Inter-disciplinary profile of knowledge users: citations received from each SC, and a field-specific breakdown of all publications citing the work of the institution [31].

Spill out: the percentage of publications of university departments published outside the field of their research (measured by journal classification) [26].

3. Methodology

The following subsections explain the data, the research procedures and the indicators used in this study.

3.1. Sampling

Excellence in Research for Australia (ERA) is the Australian national research evaluation process that is undertaken every 3 years, and ERA 2015 is the current one and ERA 2018 is currently under investigation. ERA rates the performance of higher education institutions in a range of subject fields by a specific classification of academic fields known as FoR. A full list of FoR codes is available at http://www.arc.gov.au/rfcd-seo-and-anzsic-codes. The Australian Research Council (ARC) that is responsible for ERA maintains a list of journals with their FoR codes. Each journal can have a maximum of three FoR codes. FoR is increasingly used by the bibliometric community for research purposes [32].

There is no database of all scholars in Australia which could have been used to draw a random sample from the database. Thus, we decided to undertake cluster sampling at the institutional level. To draw our sample, ERA 2015 data were used to identify the universities that are active in research in the three areas of physics, chemistry and biology. Then for each of the three areas four universities were randomly selected. We then extracted the publication lists and attributes such as age and rank of all full professors and associate professors affiliated with the selected universities using their websites.

Overall, the sample included 75 authors from physics, 63 from chemistry and 125 from biology. The data were obtained from Scopus and WoS core collections. In summary, the specifications of our data are as follows:

Document type was restricted to three types ‘Article’, ‘Review’ and ‘Article, conference paper. These were chosen as they are mainly peer-reviewed publications known as substantial items [33].

The period was set as 1980–2014 (inclusive) because some of the indicators used in the study were only available for this period in an in-house database maintained by the Max Planck Society and based on WoS. It is also a reasonable period as it covers 34 years and allows 3–4 citing years at the most recent publication years for receiving citations. The citation window for impact measurements should be at least 3 years [34].

Full and associate professors were included in the study because they are well established in their research and have a sufficient number of publications for a meaningful analysis.

The three broad areas of physics, chemistry and biology were used because citation based bibliometric techniques can be rigorously applied to these fields for research evaluation [35] but not to some other fields (e.g. engineering or social sciences).

3.2. Data collection processes

The website of each university was visited to locate the relevant academic unit (school, department, faculty or research centre) that fell into the given research area. Full and associate professors were identified and their institution, gender, rank and field were recorded. Then the authors were searched in Scopus for their profile. Some further information about each author was recorded based on their Scopus profile including their first year of publication, total number of publications and total citations. A full list of publications for each author was downloaded as tab delimited files. The reason Scopus was used at this stage was that Australian universities have been using Scopus for their research data and for ERA evaluations and, therefore, the majority of Australian academics monitor and maintain their Scopus profile. This makes it an easy and reliable method for obtaining full lists of publications for each author. Scopus author profiles also cover publications from authors’ previous affiliations as well as variations of authors’ names.

Scopus publications lists were randomly checked with the scholars’ university profile and curriculum vitae (CV) of authors to ensure that the publication list was accurate. Once the full lists of publications were obtained, all publications were searched in WoS through their digital object identifier (DOI) or if the publication had no DOI, through a combination of publication title and publication year. Full WoS records for the publications were obtained. Further checks were done to make sure that the publications were not incorrectly assigned to authors in the sample. Then further citation metrics were added to the data from the WoS in-house database.

3.3. Authors’ publication counts and impact indicators

A range of indicators was used in this study for publication counts and citation impact evaluation of researchers. However, authors’ academic age (i.e. the number of years during which an author has published: ‘last year of publication – first year of publication + 1’) was used to normalise some of the indicators. They include the following:

Number of publications: total number of publications for each researcher.

Average publications per year.

TC3: Total citations received in the first 3 years after publications (not including the publication year). The average of publications’ TC3 values for an author was used in our analysis.

Hazen: citation impact percentile calculated by the Hazen formula [36]. The higher the percentile, the more citation impact the publication has received. A percentile of 90 means that the paper belongs to the 10% most frequently cited papers in the corresponding SC and publication year. The median percentile was calculated for each author. Percentile indicators were used since the use of such indicators have been recommended in bibliometrics [37].

P10: number of papers which belong to the 10% most frequently cited papers in the corresponding SC and publication year. The number for each author was divided by the total number of years since publishing the first publication in the dataset (i.e. maximum year of publication – minimum year of publication + 1) in order to normalise the data for authors with different lengths of activity. The result was multiplied by 10.

3.4. Authors’ research diversity indicators

The ARC journals list with their assigned FoR codes were used to assign FoR codes to each publication. In the case of publications where the journals had more than one FoR code, only the first code, as the main code, was included in the analysis. It should be noted that some journals (such as Nature or Science) are classified as multidisciplinary (MD) and we have used specific indicators for such journals. We calculated four measures for diversification in order to cover different aspects. They include the following:

ED: the ratio of distinct FoR codes for each author divided by his or her publication counts. Instead of simply counting the number of FoR codes (similar to Abramo et al. (21)), which does not take into account the number of publications of authors, we divide it by the number of publications so the result shows the ED per publication. This measure does not include publications coded as MD.

Intensity of diversification or DR: the ratio of the publications falling in FoR codes other than the dominant code (i.e. the FoR code in which the researcher has published most) to the total number of publications [21]. This measure does not include publications coded as MD.

MD Ratio: the ratio of publications with MD codes to all publications.

Divergence: FoR codes are integer values and have a logical order. Furthermore, neighbouring disciplines have codes closer. For instance, 02 and 03 are assigned to physical and chemical sciences, respectively, while 08 is assigned to library and information science and 18 to law and legal studies. For measuring divergence in this study, the variance of FoR codes of all publications was calculated. The higher the variance, the higher the diversification as it takes into account the number of distinct FoR codes and their distance.

For the analysis, we have performed regression analysis and correlation. Ordinary least squares (OLS) regression was used to analyse the relationship between rank or gender and diversification indicators. Gender was coded as 1 = male and 0 = female, and rank was coded as 1 = full professor and 0 = associate professor. The assumptions for regression analysis (e.g. normal distribution of residuals, multicollinearity of independent variables etc.) were tested and met.

4. Results

4.1. Output and impact by field

Tables 1 –4 show the output and impact indicators (i.e. average number of publications, average of citations received after 3 years for each publication, median of Hazen percentiles and average of P10) for the authors of three fields separately (Tables 1 –3 for physics, chemistry and biology, respectively, and combined in Table 4) by gender and rank. The field-normalised citation scores are especially interesting in the overall view across the three fields. The results show that the average number of publications is greatest in physics and is lowest in biology. While TC3 is larger in biology than in chemistry and physics, median Hazen percentiles is larger in chemistry than in physics and biology. P10 is larger in physics than in chemistry and biology. Generally, men performed better than women in all disciplines (and overall) in all indicators, except in one case where TC3 for women in physics (14.13) was slightly higher than men’s (12.46).

Table 1.

Average of publications and citation indicators for physics.

Authors’ attributes/indicators	N	Average no. of publications	Average publications per year	Average TC3	Median Hazen	Average P10
Male	63	124.41	5.24	12.46	63.02	10.5
Associate professor	18	95.44	5.04	11.15	60.33	10.1
Professor	45	136	5.32	12.98	64.09	10.6
Female	12	100.67	5.02	14.13	61.58	8.2
Associate professor	3	65	4.4	26.15	74.17	13.5
Professor	9	112.56	5.22	10.12	57.39	6.4
Overall	75	120.61	5.2	12.72	62.79	10.1

Table 2.

Average of publications and citation indicators for chemistry.

Authors’ attributes/indicators	N	Average no. of publications	Average publications per year	Average TC3	Median Hazen	Average P10
Male	53	117.47	4.68	12.09	64.86	7.2
Associate professor	16	58.88	3.28	11.52	65.97	4.9
Prof	37	142.81	5.28	12.34	64.38	8.2
Female	10	43.6	2.15	8.62	55.90	2.8
Associate professor	6	41	2.13	10.64	57.42	3.3
Prof	4	47.5	2.18	5.59	53.63	2
Overall	63	105.75	4.27	11.54	63.44	6.5

Table 3.

Average of publications and citation indicators for biology.

Authors’ attributes/indicators	N	Average no. of publications	Average publications per year	Average TC3	Median Hazen	Average P10
Male	92	77.4	3.11	15.65	63.05	5.5
Associate professor	32	45.88	2.57	21.37	60.41	4.8
Professor	60	94.22	3.39	12.59	64.46	5.8
Female	33	56.73	2.28	10.47	61.83	3.1
Associate professor	17	36.82	1.86	10.3	61.44	2.5
Professor	16	77.88	2.73	10.65	62.25	3.7
Overall	125	71.94	2.89	14.28	62.73	4.9

Table 4.

Average of publications and citation indicators for all three fields combined.

Authors’ attributes/indicators	N	Average no. of publications	Average publications per year	Average TC3	Median Hazen	Average P10
Male	208	101.85	4.15	13.78	63.50	7.4
Associate professor	66	62.55	3.42	16.19	61.73	6.3
Professor	142	120.12	4.49	12.65	64.32	8.0
Female	55	63.93	2.86	10.93	60.70	4.2
Associate professor	26	41.04	2.22	12.21	61.98	4.0
Professor	29	84.45	3.43	9.78	59.55	4.3
Overall	263	93.92	3.88	13.18	62.91	6.7

While male physicists published on average more than twice the average of female physicists, their citation impact is close to female physicists and even the average 3-year post-publication citations of females are better than their male counterparts. Female associate professors outperform female professors and even male counterparts in both ranks in some of the impact indicators despite their lower average number of publications. Likewise, female associate professors in chemistry outperform female professors and are very close to male counterparts of either rank despite their lower average number of publications.

Table 5 shows the averages and standard deviations for the authors’ publication counts/impact indicators. Physicists show a better output level, publishing on average more than the other two groups but each field has a different ranking for citation-based indicators. Biologists outperform the other two groups considering the average citations received after 3 years, while chemists outrank others considering Hazen percentiles, and physicists perform the best in terms of P10. Similarly, a clear citation-based impact distinction does not exist for rank, and associate professors have higher TC3 values despite having almost half the number of publications of professors on average. For the other two indicators (Hazen percentiles and P10) professors perform better than associate professors. There is a clear difference between the publication counts and impact of male and female researchers and male scientists outperform female scientists when combining all the fields.

Table 5.

Mean and SD of publication counts and impact indicators by discipline, gender and rank.

		Average publications count		Average TC3		Median Hazen		Average P10
		M	SD	M	SD	M	SD	M	SD
Discipline	Physics	120.61	114.78	12.7	8.3	62.8	13.87	10.0	15.4
	Chemistry	105.75	87.95	11.5	7.4	63.4	11.4	6.5	6.6
	Biology	71.94	49.99	14.3	18.8	62.7	14.8	4.9	5.4
Rank	Associate professor	56.47	61.56	15.1	21.7	61.8	15.2	5.6	11.0
Rank	Professor	114.07	88.90	12.2	7.3	63.5	12.9	7.3	9.1
Gender	Female	63.93	54.29	10.9	6.9	60.7	13.9	4.2	5.2
Gender	Male	101.85	89.69	13.8	15.5	63.5	13.6	7.4	10.6
Overall		93.92	84.86	13.2	14.1	62.9	13.7	6.7	9.8

SD: standard deviation.

Table 6 shows the distribution of publications for each of the three disciplines across all discipline fields (FoR codes). The dominant discipline fields are shown in bold for each of the three given disciplines. In physics, 64.77% of the publications were published in journals categorised in physical sciences and 22.21% were published in mathematical sciences. In chemistry, 79.64% were published in chemical sciences journals and 7.91% in physical sciences. In biology, 52.24% were published in biological sciences journals while the rest were published in other fields including 13.71% in environmental sciences. The percentage of publications in MD journals was highest in biology (11.01%) and lowest in chemistry (4.07%).

Table 6.

Percentage of publications in each FoR field by discipline.

FoR field	Physics	Chemistry	Biology
Multidisciplinary (MD)	5.26	4.07	11.01
Mathematical Sciences	22.21	0.49	1.36
Physical Sciences	64.77	7.91	1.51
Chemical Sciences	1.77	79.64	6.22
Earth Sciences	0.86	1.94	5.68
Environmental Sciences	0.28	0.37	13.71
Biological Sciences	0.76	2.30	52.24
Agricultural and Veterinary Sciences	0.01	0.49	1.60
Information and Computer Sciences	1.18	0.02	0.13
Engineering	1.85	1.44	0.71
Technology	0.01	0.00	0.01
Medical and Health Sciences	0.55	1.12	5.48
Other subjects	0.49	0.22	0.33
Total	100	100	100

FoR: Fields of Research.

4.2. Diversification by field

Table 7 shows the mean (M) and standard deviation (SD) of the four diversification measures by discipline, gender, rank and overall. Biology scholars appeared to have more diversity in their research compared with scholars in physics and chemistry in all indicators, while there was no clear ranking for chemistry and physics scholars. Physicists have more research diversity considering DR and MD ratio but lower diversity in ED and divergence. Analysing scholars’ rank for all the fields, publications by associate professors included more diversification compared with full professors. While women’s publications had higher ED and DR, men’s publications had larger divergence and a greater ratio of publications in MD journals.

Table 7.

Mean and SD of diversification indicators by discipline, gender and rank.

		Extent of diversification (ED)		Diversification ratio (DR)		Divergence^a		MD Ratio
		M	SD	M	SD	M	SD	M	SD
Discipline	Physics	0.100	0.08	0.269	0.19	4.659	5.53	0.058	0.08
	Chemistry	0.149	0.11	0.236	0.22	3.866	5.61	0.053	0.10
	Biology	0.209	0.11	0.398	0.17	7.834	5.75	0.104	0.09
Rank	Associate professor	0.227	0.13	0.360	0.19	6.451	6.02	0.083	0.09
Rank	Professor	0.129	0.09	0.302	0.20	5.724	5.86	0.076	0.10
Gender	Female	0.230	0.16	0.336	0.18	5.620	5.45	0.072	0.08
Gender	Male	0.146	0.09	0.319	0.20	6.073	6.04	0.081	0.10
Overall		0.163	0.11	0.322	0.20	5.978	5.91	0.079	0.09

SD: standard deviation; MD: multidisciplinary.

All Mean and SD values of divergence were divided by 1,000,000 to shorten the numbers.

4.3. Diversification, gender and rank

This section presents the results of OLS regressions for diversity indicators versus rank and gender (Tables 8 –11). The results show that rank and gender have a statistically significant effect on diversification (ED) for chemistry, biology and overall but not for physics. The coefficients are negative for both rank and gender which means full professors and men have smaller diversification (ED) values.

Table 8.

OLS regression of ED versus gender and rank.

Sub	Gender	Rank	R ²
Physics	0.012	−0.183	0.034
Chemistry	−0.387***	−0.456***	0.438
Biology	−0.206**	−0.429***	0.253
Overall	−0.251***	−0.379***	0.232

OLS: ordinary least squares; ED: extent of diversification.

p value < 0.05, ***p value < 0.01.

Table 9.

OLS regression of DR versus gender and rank.

Sub	Gender	Rank	R ²
Physics	0.177	−0.029	0.033
Chemistry	−0.267**	−0.214*	0.143
Biology	0.109	−0.096	0.018
Overall	−0.018	−0.137**	0.02

OLS: ordinary least squares; DR: diversification ratio.

p value < 0.10, **p value < 0.05.

Table 10.

OLS regression of divergence versus gender and rank.

Sub	Gender	Rank	R ²
Physics	0.152	0.194*	0.059
Chemistry	−0.089	−0.344**	0.140
Biology	0.141	−0.024	0.020
Overall	0.040	−0.064	0.005

OLS: ordinary least squares.

p value < 0.10, **p value < 0.05.

Table 11.

OLS regression of MD Ratio versus gender and rank.

Sub	Gender	Rank	R ²
Physics	0.153	0.198*	0.061
Chemistry	−0.057	−0.230*	0.062
Biology	0.142	−0.016	0.020
Overall	0.047	−0.043	0.004

OLS: ordinary least squares; MD: multidisciplinary.

p value < 0.10.

Rank is statistically significantly related to DR in chemistry and overall, but not in physics and biology. Gender also statistically significantly impacts on DR in chemistry. In the case of divergence, rank only has a statistically significant effect in chemistry.

4.4. Diversification and impact

This section shows the Spearman correlation coefficients (as the data were not normally distributed) for output, citation impact and diversification indicators. Figure 1 illustrates the scatter plot matrix and spearman correlation values. Pink/bold cells are significant at 0.01 and grey/bold/italics cells are significant at 0.05. The notable correlations are negative correlations between TC3 and ED, between DR and P10, between ED and P10 and between Hazen percentiles and ED. This generally indicates that those who have a larger ED and DR are less probably to have a larger number of highly cited papers. ED was also negatively correlated with the number of citations received in the first 3 years (TC3).

Figure 1.

Scatterplot and Spearman correlation between diversity indicators and output/impact indicators.N = 263; pink and bold cells: sig < 0.01, grey and bold and italics cells: sig < 0.05.

As some of the past studies [6] found curvilinear relationships between diversity and citation impact, we tested also for curvilinear relationships. We only found a statistically significant curvilinear relationship between ED and P10, while the statistically significant simple regression equation was F(1, 261) = 41.0, p = 0.000, R² = 0.136. Adding a quadratic component to the regression of ED and P10 produced a significant increase in fit, with 0.065 in R² change, F(2, 260) = 32.7, p = 0.000, R² = 0.201. This indicates that while ED has an initial negative effect on P10, a threshold is reached beyond which higher levels of ED might have positive impact on P10 (U shape). In other words, an initial increase in ED reduces P10 but after a certain threshold, an increase in ED results in an increase in P10.

There is also a very strong correlation between MD ratio and divergence which indicates those who have a higher percentage of articles in MD journals are also more probably to publish articles in journals that are farther apart in the FoR subject classification hierarchy.

5. Conclusion

This research investigated whether multi/inter-disciplinary research activities of scholars in chemistry, biology and physics are related to their research publication counts and (citation-based) impact. The study has implications for research evaluation and policies, but also it shed some light on some other aspects concerning the trends towards interdisciplinarity from the point of view of faculty age, gender and disciplinary origin. Although some studies have been done in the past in this area, the collective evidence was not conclusive and there was a need for more research. Understanding the relationship between diversity and impact has implications both for individual researchers in planning their career and for science policymakers.

The results showed disciplinary differences in the output and impact indicators. While physicists published more than chemists and biologists, the citation rates (TC3) were highest in biology and lowest in chemistry. On the other hand, the normalised citations (Hazen percentiles) were highest in chemistry and lowest in biology. P10 was highest in physics and lowest in chemistry. The distribution of publications in different FoR codes showed that biologists publish almost half (47.7%) of their papers in journals categorised outside biological sciences. This number was 35.23% for physics and 20.3% for chemistry. These findings are in line with those of past studies as physicists in the Netherlands published 36% of their publications outside physics journal [13]. A previous study in Australia [26] also showed similar results with, for example, 49% of biologists’ articles were published in journals outside biology. There were also disciplinary differences in various indicators of diversification. Biology had the highest values in all of the four aspects of diversification including extent and intensity of diversification, MD ratio and divergence. While chemistry had a larger ED, physics had a larger DR, MD ratio and divergence. In contrast to this study and Bourke and Butler [26], Abramo et al. [21] found that chemistry had the highest ED, however, their definition of ED was slightly different as in their calculation, the number of topics was not divided by the number of publications.

Although Abramo et al. [22] did not find gender as a related factor to diversification, our results showed that gender and rank of researchers have a statistically significant effect on aspects of research diversification in some disciplines. Women were more probably than men to have a greater ED in chemistry, biology and overall, and also more probably to have a greater DR in chemistry. In terms of rank, associate professors (compared with full professors) have a greater ED in chemistry, biology and overall, and have a larger DR overall. Furthermore, they have a larger divergence in chemistry. This is in contrast to the findings of Abramo et al. [22] who revealed a higher ED for full professors.

The analysis of the relationship between diversification indicators and output/impact indicators revealed that ED was negatively correlated with both output and impact. The larger the average number of FoR codes, the lower the number of publications and impact indicators (total citations, P10 and TC3). The DR also had a negative but weak correlation with the number of publications, P10 and TC3. MD Ratio also had a weak positive correlation with the number of citations received TC3. Comparing these results with those of past studies is problematic as there is no similar other study that has used similar approaches and indicators. Few studies have used SCs of published articles to study the relationship between diversity and citation impact at author level. Some past studies found positive correlations [20], some negative correlations [18] and some no effect [6].

Some of these differences are due to data and indicators used in the studies. For instance, both studies, Leahey et al. [20] and Yegros-Yegros et al. [6], used SCs of cited references of papers to measure diversity and they found contradicting findings. The former was conducted at author level while the latter was done at article level. A recent study [38] confirmed that different indicators (e.g. disparity vs variety) and methodologies (e.g. using references vs using authors’ affiliations) ‘can produce seriously inconsistent and even contradictory results’ (p. 271). Chakraborty et al. [1] used SCs of publishing journals but the field of their study was computer science which is different by nature from those included in our study. However, there is some similarity between our findings and those of Chakraborty et al. [1] as both studies reveal complexity of the relationship between citation impact and diversity. While Chakraborty et al’.s [1] study implied that diversity is overall good for impact only if it is spread over time and researchers should focus on few fields in each time window, our findings imply that positive or negative effects of diversity on citation impact might partly be influenced by the choice of fields and journals in which researchers contribute. This is because divergence and MD ratio seemed to have some positive effects on impact while other diversity indicators had negative effects on impact. Divergence in this study was measured with variance of FoR codes, but we should note that it is not a perfect measure. FoR numbers assigned to each discipline do not always reflect the cognitive distance between them, especially when one passes from the sciences to the social science, and from the latter to arts and humanities. It is difficult to measure the cognitive distance between different FoR codes (something that should be done in future studies), and some related fields might be farther from one another than expected. Therefore, the divergence values should be interpreted with caution. However, as this study only focused on three disciplines of physics (02), chemistry (03) and biology (06), the impact of the issue related to the cognitive distance on the calculation should not be big.

While the nature and dynamics of the relationship between diversity and impact merit further investigation, the reason for their interaction also needs investigation. We need to know why inter-disciplinary research is more or less cited than non-inter-disciplinary research. For example, Judge et al. [2] maintained that social network variables do a good job of explaining why some inter-disciplinary research is more cited than non-inter-disciplinary research. However, other variables such as quality and readership size of journals could play a role. That needs further research.

This study has some limitations that should be considered when trying to generalise any of the findings. Focusing on one country (Australia) and a lack of data on some variables that could improve the data analysis are among these limitations. Relying on journal classifications also has its own issues as such classifications may not precisely represent the scope of the journals. For instance, while PLoS One is a MD journal, it publishes more articles from life and health sciences than physical sciences. The focus only on three fields and the number of individuals included in the study are other issues that should be taken into account. We should also note that research activity in each university is influenced by the university’s policies, strategies and research culture and these may vary from one university to another.

Footnotes

Acknowledgements

Some of the bibliometric data used in this article are from an in-house database developed and maintained by the Max Planck Digital Library (MPDL, Munich) and derived from the Science Citation Index Expanded (SCI-E), Social Sciences Citation Index (SSCI) and Arts and Humanities Citation Index (AHCI) provided by Clarivate Analytics (Philadelphia, PA, USA).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Hamid R Jamali

References

Chakraborty

Tammana

Ganguly

, et al. Understanding and modeling diverse scientific careers of researchers. J Informet 2015; 9(1): 69–78.

Judge

Weber

Muller-Kahle

. What are the correlates of interdisciplinary research impact? The case of corporate governance research. Acad Manage Learn Educ 2012; 11(1): 82–98.

Wright

. Public administration as an interdisciplinary field: assessing its relationship with the fields of law, management, and political science. Public Administr Rev 2011; 71(1): 96–101.

Jacobs

Frickel

. Interdisciplinarity: a critical assessment. Annu Rev Sociol 2009; 35: 43–65.

Alexander

Bache

Chase

, et al. An exploratory study of interdisciplinarity and breakthrough ideas. In: 2013 Proceedings of technology management in the IT-driven services (PICMET’13), San Jose, CA, 28 July–1 August 2013, pp. 2130–2140. New York: IEEE.

Yegros-Yegros

Rafols

D’Este

. Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PLoS ONE 2015; 10(8): e0135095.

Klein

. Prospects for transdisciplinarity. Futures 2004; 36(4): 515–526.

Ledford

. How to solve the world’s biggest problems. Nature 2015; 525(7569): 308–311.

Carayol

Thi

. Why do academic scientists engage in interdisciplinary research? Res Eval 2005; 14(1): 70–79.

10.

Choi

Pak

. Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. Definitions, objectives, and evidence of effectiveness. Clin Invest Med 2006; 29(6): 351–364.

11.

Mutz

Bornmann

Daniel

. Cross-disciplinary research: what configurations of fields of science are found in grant proposals today? Res Eval 2014; 24(1): 30–36.

12.

Bromham

Dinnage

Hua

. Interdisciplinary research has consistently lower funding success. Nature 2016; 534(7609): 684–687.

13.

Rinia

Van Leeuwen

Van Vuren

, et al. Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research. Res Policy 2001; 30(3): 357–361.

14.

Levitt

Thelwall

. Is multidisciplinary research more highly cited? A macrolevel study. J Am Soc Informat Sci Tech 2008; 59(12): 1973–1984.

15.

Larivière

Gingras

. On the relationship between interdisciplinarity and scientific impact. J Am Soc Informat Sci Tech 2010; 61(1): 126–131.

16.

Wang

Thijs

Glänzel

. Interdisciplinarity and impact: distinct effects of variety, balance, and disparity. PLoS ONE 2015; 10(5): e0127298.

17.

Enduri

Reddy

Jolad

. Does diversity of papers affect their citations? Evidence from American Physical Society Journals. In: 2015 11th international conference on signal-image technology & Internet-based Systems (SITIS), Bangkok, Thailand, 23–27 November 2015, pp. 505–511. New York: IEEE.

18.

Pan

Katrenko

. A review of the UK’s interdisciplinary research using a citation-based approach: report to the UK HE funding bodies and MRC by Elsevier. Bristol: HEFCE.

19.

Chen

Arsenault

Larivière

. Are top-cited papers more interdisciplinary? J Informet 2015; 9(4): 1034–1046.

20.

Leahey

Beckman

Stanko

. Prominent but less productive: the impact of interdisciplinarity on scientists’ research. Administr Sci Quarter 2017; 62(1): 105–139.

21.

Abramo

D’Angelo

Di Costa

. Specialization versus diversification in research activities: the extent, intensity and relatedness of field diversification by individual scientists. Scientometrics 2017; 112(3): 1403–1418.

22.

Abramo

D’Angelo

Di Costa

. The effects of gender, age and academic rank on research diversification. Scientometrics 2018; 114(2): 373–387.

23.

Abramo

D’Angelo

Di Costa

. The effect of multidisciplinary collaborations on research diversification. Scientometrics 2018; 116(1): 423–433.

24.

Abramo

D’Angelo

Di Costa

. Do interdisciplinary research teams deliver higher gains to science? Scientometrics 2017; 111(1): 317–336.

25.

Abramo

D’Angelo

Di Costa

. Diversification versus specialization in scientific research: which strategy pays off? Technovation. Epub ahead of print 18 July 2018. DOI: 10.1016/j.technovation.2018.06.010.

26.

Bourke

Butler

. Institutions and the map of science: matching university departments and fields of research1. Res Policy 1998; 26(6): 711–718.

27.

Dranev

Kotsemir

Syomin

. Diversity of research publications: relation to agricultural productivity and possible implications for STI policy. Scientometrics 2018; 116(3): 1565–1587.

28.

Porter

Cohen

David Roessner

, et al. Measuring researcher interdisciplinarity. Scientometrics 2007; 72(1): 117–147.

29.

Rafols

Meyer

. Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience. Scientometrics 2009; 82(2): 263–287.

30.

Mugabushaka

Kyriakou

Papazoglou

. Bibliometric indicators of interdisciplinarity: the potential of the Leinster–Cobbold diversity indices to study disciplinary diversity. Scientometrics 2016; 107(2): 593–607.

31.

Van Raan

Van Leeuwen

. Assessment of the scientific basis of interdisciplinary, applied research: application of bibliometric methods in nutrition and food research. Res Policy 2002; 31(4): 611–632.

32.

Rousseau

. The Australian and New Zealand’s Fields of Research (FoR) codes. ISSI Newslett 2018; 14(3): 59–61.

33.

Leydesdorff

Carley

Rafols

. Global maps of science based on the new Web-of-Science categories. Scientometrics 2013; 94(2): 589–593.

34.

Abramo

D’Angelo

Felici

. Predicting long-term publication impact through a combination of early citations and journal impact factor. J Informet 2019; 13(1): 32–49.

35.

Council of Canadian Academies. Informing research choices: indicators and judgment: the expert panel on science performance and research funding. Ottawa, ON, Canada: Council of Canadian Academies, 2012.

36.

Bornmann

Leydesdorff

Mutz

. The use of percentiles and percentile rank classes in the analysis of bibliometric data: opportunities and limits. J Informet 2013; 7(1): 158–165.

37.

Hicks

Wouters

Waltman

, et al. Bibliometrics: the Leiden Manifesto for research metrics. Nature 2015; 520(7548): 429–431.

38.

Zhang

Sun

Chinchilla-Rodríguez

, et al. Interdisciplinarity and collaboration: on the relationship between disciplinary diversity in departmental affiliations and reference lists. Scientometrics 2018; 117(1): 271–291.