Secular Trends in Impact Factor of Breastfeeding Research Publications over a 20-Year Period

Introduction

The concept of impact factor (IF) of a given scientific journal was developed by Garfield in order to provide a “measure reflecting the average number of citations to recent articles published in the journal.”^1,2 Although the IF does not necessarily reflect journal quality, a high IF is often a source of pride for the editor of a journal and is routinely reported on the home page of most medical journals' Web sites. ³ When scientists are considered for academic promotion, they are often asked to provide the IF of the journals in which their articles are published. ³ Presumably, if an article is published in a high IF journal, the article is considered as being of high quality. ⁴ However, Garfield ⁴ himself warned that, because of the “wide variation from article to article within a single journal,” IF might be misused in “evaluating individuals.”

Several systems have been proposed that allow determining the strength of scientific evidence that an article possesses. ⁵ Among them, evidence quality can be assessed based on the source type (from meta-analyses and systematic reviews at the top of the hierarchical classification) “down to conventional wisdom at the bottom.” ⁵

The aim of the current study was to determine whether secular changes have occurred in the breastfeeding literature, in terms of IF, as well as in terms of number and type of yearly published articles. We examined the following hypotheses: (1) the average IF of the journals in which breastfeeding-related articles are published has increased over the years, and (2) the percentage of articles of higher quality (in terms of degree of evidence) has increased over time.

Research Design and Setting

We used the Internet URL www.ncbi.nlm.nih.gov/entrez in order to evaluate all Medline articles recorded from January 1, 1992 through December 31, 2011. We focused on all articles in the field of breastfeeding. In order to do so, we limited the search, using Medline's own limits engine, to the keywords of “breast milk” or “breast feeding” or “human milk.” We limited the search to “humans only” and “English” language. We repeated the search year by year according to the total number per year of articles per type of article, for the 20 years of the specified period, analyzed year by year. The type of article was, according to Medline's own categorization, specifically labeled randomized controlled trial (RCT), clinical trial, case report, meta-analysis, letter to the editor, review, systematic review, practice guideline, and editorial. In order to verify that the categorization and tagging offered automatically by PubMed were accurate, we used a random sample of 10 studies each year, and in 100% of the cases, PubMed's categorization was found to be accurate.

The IF of each journal was determined for the year of publication of each RCT, based on the Thomson Reuters (ISI) Web of Knowledge. We recorded the sample size of each RCT as a potential confounder.

Results

Over the 20 year-period we identified 31,299 articles. By category studied, there were 2,022 clinical trials (among them 1,343 RCTs), 151 meta-analyses, 760 systematic reviews, 125 practice guidelines, 3,576 reviews, 936 case reports, 447 editorials, and 1,463 letters to the editor.

There was a linear increase over the years in all the relevant articles, recorded, from approximately 1,100/year in 1992 to 2,100 in 2011 (i.e., an approximately 1.9-fold increase [R²=0.995, p=0.001]), in clinical trials (a threefold increase from approximately 50 to 150 per year [R²=0.819, p=0.001]) (Fig. 1), in RCTs (a 3.6-fold increase from approximately 30 to 110 per year [R²=0.792, p<0.001]), in meta-analyses (a sharp increase from none to four in the years 1992–1995 to approximately 15 per year in 2011 [R²=0.639, p<0.001]), in systematic reviews (a sharp increase from none to 15 per year in 1992–1999 to approximately 80 per year in 2011 [R²=0.902, p<0.001]), in practice guidelines (a sharp increase from none to five per year in 1992–1995 to 11 per year in 2011 [R²=0.753, p<0.001]), in reviews (a 2.3-fold increase from approximately 110 per year in 1992 to 250 per year in 2011 [R²=0.669, p<0.001]), in case reports (a 1.5-fold increase from approximately 37 to 55 per year [R²=0.445, p=0.001]), and in editorials (a 1.7-fold increase from approximately 16 to 27 per year [R²=0.432, p=0.002]). There was a significant decrease in letters to the editor (a 1.3-fold decrease from approximately 85 to 65 per year [R²=0.425, p=0.002]).

OPEN IN VIEWER

FIG. 1.

Yearly number of clinical trials (y-axis) versus year of publication (x-axis) (R²=0.819, p=0.001).

When we considered the RCTs only in relation to the IF of the journal in which each was published, the following results were found: concomitant with the yearly increase in the number of published RCTs per year previously described, there was a linear increase in the number of journals publishing articles on breastfeeding per year (from approximately 20 in 1992 to >60 in 2011 [R²=0.811, p<0.001]) (Fig. 2) but an overall decrease in the average IF every year (R²=0.202, p=0.047) (Fig. 3). However, the ratio of high-quality articles (RCTs+meta-analyses+systematic reviews+CT)/low-quality articles (reviews+editorials+letters to the editor+case reports) increased significantly over time.

OPEN IN VIEWER

FIG. 2.

Yearly number of journals (y-axis) versus year of publication (x-axis) (R²=0.811, p<0.001).

OPEN IN VIEWER

FIG. 3.

Impact factor (y-axis) versus year of publication (X-axis) (R²=0.202, p=0.047).

Discussion

Our study showed that, as expected, there has been a sharp rise in the number of articles published in the field of human milk and breastfeeding. The clinician or investigator involved with this field of medicine who wanted to keep current with the literature needed to read over 1,000 articles per year in 1992 and over 2,000 articles in 2011. This increase is substantial and is probably due in part to the more than threefold increase in the number of journals that published articles related to this field of medicine. In fact, human milk- or breastfeeding-related articles were dispersed in >60 medical journals in 2011.

The yearly change in the number of published articles was not similar for all categories of articles. For instance, the yearly number of meta-analyses or systematic reviews increased enormously over time. This is not surprising, in view of the fact that prior to the 1990s these types of analyses were practically nonexistent. Also, in order to be conducted, meta-analyses or systematic reviews require a sufficient cumulative sample size (and study number). Meta-analyses were designed to let us use small studies that had insufficient power to stand alone. Meta-analyses use methods focused on contrasting and combining results from different studies, in order to detect patterns among study results and sources of disagreement among studies. ⁶ De facto, meta-analyses use a statistical approach to combine results from multiple studies. Thus, results are based on a much larger sample size (which increases power of the study). A major pitfall of meta-analyses is the presence of publication bias, whereby studies with negative results are less likely to be published than those with positive results. ⁷ Thus, for any given research area, one cannot know how many studies have gone unreported. The number of published clinical trials increased threefold over time, and it is noticeable that, among these trials, the rate of increase in RCTs (the “gold standard” of clinical trials) increased the most, with a 3.6-fold increase over the years considered. The number of reviews increased in a more modest fashion (a 2.3-fold increase) during this period, and that of editorials increased by 1.7-fold, whereas the number of letters to the editor even decreased over time. These articles represent the lowest level of evidence, and it can be said that from a quality standpoint that it is likely that medical articles in this field of medicine increased not only in numbers, but also in quality. Paradoxically, although the ratio of high-quality/low-quality articles improved over time, another measure of quality (i.e., IF) decreased over time. The reason(s) for this apparent paradox might relate to the fact that the IF is an estimate of quality of a journal, and not of a given article. Indeed, the IF is a measure reflecting the average number of citations of recent articles published in the journal. We speculate that when the number of journals related to a given field of medicine increases, articles of each journal are less often cited. Moreover, editors of journals place a limit on the number of articles that each author is allowed to cite in a given article. We speculate that these limitations might be more compulsively applied nowadays than in the past, leading to a reduction in the number of times that a given article, even of high quality, is cited.

We conclude that there has been a linear increase in the number of publications related to the field of breastfeeding and human milk. Although this represents a challenge to readers interested in the field, it appears that we might be witnessing an improvement in the quality of articles published. We noted that very few up-to-date guidelines have been published in the field (approximately 12/year), which might indicate the need for improved organization and coordination among professional societies related to breastfeeding, such as the Academy of Breastfeeding Medicine, the American Academy of Pediatrics, and the International Lactation Consultant Association.