Review of research output of Australian and New Zealand colorectal surgeons over the past 20 years

Literature search

All current members of the CSSANZ were identified in April 2020 using the CSSANZ ‘find a surgeon’ website tool. A retrospective search of the Scopus database for medical peer-reviewed publications was conducted to collate, assess and define each surgeon’s research output. Scopus was the research database chosen as it employs an algorithm using name, location and affiliation to determine authorship and also provides detailed author profiles. Publications were deemed as belonging to the surgeon if the health service recorded on Scopus matched the health service of the surgeon, either current or previous. Only publications over the 20-year period between 1999 and 2020 were included.

Analysis

Data recorded included the total number of publications and citations per author, publication year, type of publication, topic of publication and the surgeons’ H-index. The type of study (systematic reviews, meta-analyses, randomised or non-randomised controlled trials, observational studies, etc.) was recorded, and publications were grouped according to the main topic covered (colorectal cancer, inflammatory bowel disease, diverticular disease, etc.). If a paper related to colorectal disease did not fit a main topic category (e.g. a rare colorectal pathology discussed in a case report), it was categorised as ‘other’. Papers unrelated to colorectal disease (surgical education, perioperative management, etc.) were classified as ‘non-colorectal’.

H-index was used to quantify each surgeon’s individual research contribution. It is an objective mathematical model, defined as the number (h) of publications which are cited at least h times. This allows adjustment for both the quantity and quality of an author’s work, and has been shown to be an accurate measure of an individual’s academic contribution. ⁷ All statistical measurements were performed using STATA v15 software. ⁸

What did we find?

Research output by Australasian colorectal surgeons has been increasing over the past 20 years. This finding is reflected in studies worldwide ⁹ and locally, such as that by Chua et al., who demonstrated that research within surgical oncology is rising in Australia. ⁶ These findings are encouraging given that van Rossum et al. reported that Australia is inferior to some European countries, the United States and the United Kingdom in terms of publications per 10⁶ inhabitants, total number of publications and publication mean impact factor. ¹⁰ We also found publication age to be directly associated with the number of publications produced and H-index. Hence, encouraging quality research in the generation of surgeons now would have a positive impact on research output in the future.

Colorectal cancer was the most highly represented research topic. This is not surprising given it is the second most common cancer diagnosed in both men and women and accounts for 9% of all cancer deaths in Australia. ¹¹ Such interest is reflected by ongoing research in new therapy options for colorectal cancer, such as immunotherapy and genetic therapy. ¹² To date, there has been no review of the research output of colorectal surgeons (or other general surgeons), and so comparing the output of Australasian surgeons to those from other regions is not possible.

Our study has chosen to include all forms of research output by Australasian surgeons, including opinion pieces and case reports. By focusing not only on high-impact factor journals or high-level evidence trials, our study has taken a broad look at colorectal research, providing a comprehensive review of what has been published. This may ultimately help guide directions for future research by gauging which topics need more input from the scientific community.

Despite that fact that Australasian colorectal surgeons have been involved in a substantial amount of publications over the past 20 years, the lack of higher-evidence publications is somewhat concerning and raises questions about the quality of research. A low prevalence of high-level clinical studies in the surgical literature is not unique to the cohort of clinicians analysed in this study and has been reported previously for other groups of surgeons.^13,14 This should, however, be seen as an opportunity for improvement, not only as a stimulus for increasing the number of clinical trials but also the quality of trials. Surgeons should strive to publish trials that procure a clinically relevant information gain ¹⁵ and are patient centred. ¹⁶ This lack of ‘useful’ research is a problem not facing only Australasian researchers, but is an issue that requires reform and improvement worldwide. ¹⁷ Possible reasons for the lack of randomised trials in surgery include the reluctance to randomise while a procedure is being developed, reluctance to randomise when a procedure is already being routinely practised, challenges in standardising techniques of operative procedures among surgeons, difficulties in accruing sufficient patient numbers and the lack of funding in comparison to drug trials that are often supported by pharmaceutical companies. ⁶

The top 10% (n = 23) of authors accounted for more than half of the manuscripts published. These authors had a significantly higher publication age when compared with the rest of the cohort (28 vs 12 years) suggesting that colorectal surgeons are better able to publish higher volumes in the latter part of their consultant career.

What are the limitations of this study?

One of the limitations of this study is the use of Scopus as the sole source of finding publications; while quick and largely reliable, ¹⁸ this resource may have omitted relevant publications or authors from the analysis. Furthermore, authors from the cohort of colorectal surgeons analysed in our study shared names with researchers in other fields, which had the potential to confuse findings. Hurdles due to homonyms were kept to a minimum because the Scopus identification number was applied for author identification, and all references were accurately checked and any incorrect references were removed.

The use of the H-index is a somewhat flawed tool and has been criticised as a measure of the impact of an individual researcher. For example, the H-index does not take in to account author placement in the authors’ list;^19,20 it may be manipulated through self-citations ²¹ and numbers cannot be compared across disciplines because citation habits differ. ²² However, a variety of studies have shown that the H-index by and large agrees with other objective and subjective measures of scientific quality in a variety of different disciplines and that the H-index is also effective in discriminating among scientists who will perform well and less well in the future; this makes it a useful indicator of scientific quality that can be used (together with other criteria) to assist in academic appointment processes and to allocate research resources. ⁷

What are the directions for future research?

Our study demonstrates a straightforward, rapid and accessible way of quantifying the research output of a group of surgeons over a long period of time. This study utilises freely available data gathered directly from online sources, and is thus free from bias. It is also easily reproducible for any other cohort of surgeons, and may be used as part of a wider study on publishing habits of all Australasian surgeons. Coupled with the use of the H-index as an objective marker of the impact of an individual surgeon’s research output, these data may help to stratify surgeons when allocation of research grants and funding is at stake. For example, a colorectal surgeon with a strong H-index and a track record of quality publications on pelvic floor disorders would be considered first for a major research grant on pelvic floor disorders, despite the fact that they may not be as senior or experienced as some of their other colleagues. Such a transparent process could help to negate biases that may exist with allocating researcher funding. Further work needs to be done to assess how these data may be utilised by policy-makers and those responsible for allocation of research funds.