Mapping the Contributions of the Review of Educational Research to Education,1931–2020

Data Collection

The Scopus search engine was selected to identify the full corpus of RER articles for two reasons. First, Scopus has the capability to export bibliographic metadata for uploading into bibliometric software. Second, Scopus has been shown to offer far more extensive coverage of education journals than the Web of Science (Hallinger & Kovačević, 2019; Mongeon & Paul-Hus, 2016). This latter advantage of Scopus was especially relevant in the current review that employed citation analysis to examine the impact of RER articles on other published documents.

The “source” function of the Scopus search engine was used in combination with the search term “Review of Educational Research” to identify all articles published since the launch of RER in 1931 through the end of 2021. The initial search yielded 3,174 documents. Scopus filters were used to delete documents published in the Open Review of Educational Research, documents published in 2022, as well as editorials, errata, and notes. The final corpus of RER articles consisted of 3,022 journal articles. Bibliographic metadata associated with the Scopus list of RER articles were then exported to Excel for use in data analysis.

Data Analysis

Before the data could be analyzed, it was necessary to clean the Excel file by “disambiguating” the bibliographic data (Strotmann & Zhao, 2012). In this process, terms with the same meaning, but expressed differently, are replaced by a common term. For example, in the RER database, the author Jere Brophy was cited alternately as “Brophy, J.” and “Brophy, J.E.”. Harvard Educational Review was sometimes alternatively cited as “The Harvard Educational Review.” In order to correct for these “ambiguities,” the author developed a thesaurus file consisting of instructions for the bibliometric software program (i.e., VOSviewer). These instructions, for example, would instruct the software to replace all instances of “Brophy, J.” with “Brophy, J.E.” or “The Harvard Educational Review” with “Harvard Educational Review” (van Eck & Waltman, 2014).

The two main approaches to data analysis used in this review were citation and co-citation analysis (Zupic & Čater, 2015). Citation analysis, conducted in VOSviewer software version 1.6.18, computed the frequency with which RER documents and authors had been cited in other Scopus-indexed documents (van Eck & Waltman, 2014). These analyses were used to identify key sources of RER’s impact on the broader Scopus-indexed education literature (Garfield & Merton, 1979; Price, 1965).

In contrast with citation analysis, co-citation analysis calculates the frequency with which authors or journals have been cited in the reference lists of the review documents. Thus, co-citation analysis has the capability to identify secondary literature that has influenced scholarship published in RER, regardless of whether the documents were published in RER or indexed in Scopus (Zupic & Čater, 2015). Thus, in this review, co-citation analysis was used to identify sources of influence on the ideas expressed in the documents published in RER.

Co-citation analysis has also been used to gain insights into the “relatedness” of a particular literature. When conducting co-citation analysis, the VOSviewer program also computes the frequency with which authors (or journals) have been “cited together in the reference lists of the documents” (McCain, 1991; Small, 1973; van Eck & Waltman, 2014).

In order to further clarify the methodology of co-citation analysis, assume that articles authored by Bandura (1977) and Pajares (1996) appear in the reference list of an RER article authored by Tschannen-Moran, Hoy, and Hoy (1998). In this instance, VOSviewer would assign one author “co-citation” each to Bandura and Pajares, as well as a “link” to each other. The relevance of co-citation lies in the observation that when two authors (or journals) are frequently cited together (i.e., co-cited) in the same documents, there is often an intellectual similarity in their published works (Small, 1973; White & McCain, 1998).

The VOSviewer software (van Eck & Waltman, 2014) can also be used to create “science maps” of the literature under review. It draws upon the bibliographic data to create co-citation matrices that are the basis for network maps that provide a “visualization of similarities” (VOS) among authors (author co-citation map) or journals (journal co-citation map). Co-citation analysis has been applied widely as a means of understanding paradigmatic trends in different disciplines and literatures (Ding et al., 2000; Liu, 2005; McCain, 1991; White & McCain, 1998; Zupic & Čater, 2015). Zupic and Čater (2015) describe the rationale for co-citation analysis as follows.

Co-citation connects documents, authors, or journals according to the way writers use them. This is a rigorous grouping principle repeatedly performed by subject-matter experts who cite publications they deem valuable and/or interesting. Because the publication process is time-consuming, the co-citation image reflects the state of the field some time before, not necessarily how it looks now or how it may look tomorrow. It is a dynamic measure that changes through time. When examined over time, co-citations are also helpful in detecting a shift in paradigms and schools of thought. (Pasadeos, Phelps, & Kim, 1998; Zupic & Čater, 2015, p. 431)

In this review, author co-citation analysis was to reveal the “intellectual structure” of the body of knowledge encompassed in RER’s publication history. Journal co-citation analysis was employed as a means of identifying the community of journals within which RER is located (Ding et al., 2000; Liu, 2005; McCain, 1991; Zupic & Čater, 2015). Together these analyses offer insights into the predominant theoretical orientations that are reflected in RER as a source of knowledge about education.

The Landscape of RER Publications

As noted above, this review identified a corpus comprised of 3,022 review articles published since the launch of RER in 1931. Longitudinal analysis of these articles suggested, however, that RER is comprised of a bifurcated corpus with the pivot point marked at 1970. The corpus accumulated during the pre-1970 era is comprised of relatively short, two- to eight-page, “invited reviews.” These offered practitioner-accessible reflections on educational research and practice (Franklin, 1999). During this era, RER published an annual volume that averaged 48 articles per year.

In contrast, during the post-1970 era, reviews published in RER have tended to adopt a social-science model of research review. Moreover, during this era, RER articles have generally ranged from 20 to 30 article pages in length, resulting in a smaller average annual volume of 22 articles per year. This explains why 1,747 articles accumulated over a 39-year period in the pre-1970 corpus, whereas the post-1970 era features 1,275 articles published over a 52-year period (Figure 2).

OPEN IN VIEWER

Figure 2.

Annual trend in the number of articles published in Review of Educational Research, 1931–2021 (N = 3,022).

In their historical review of RER, Grant and Graue (1999) attributed these changes to new editorial policies initiated by the incoming editor, Gene Glass, in 1970. Under Glass’s editorial leadership, and sustained during the subsequent tenure of Samuel Messick, RER adopted open submission, blind peer review, and a social-scientific orientation toward the review of educational research (Grant & Graue, 1999; Popkewitz, 1999). This led to the publication of longer review articles, often quantitatively oriented, and informed by social science theories drawn, for example, from psychology, sociology, or economics.

The geographic distribution of sources within a literature is also relevant due to the belief that educational processes are influenced by the cultural contextual (Biggs, 2001; Fuller & Clarke, 1994; Ladson-Billings, 1995). Analysis of the geographic distribution of the RER corpus found that scholars from 32 different nations have contributed articles as first author. Further analysis indicated that 94% of all RER articles were authored in the United States. This skewed distribution of RER articles reflects the American roots of the journal, which was founded in association with the National Education Association and later gained sponsorship from the American Educational Research Association (Grant & Graue, 1999).

These publication data were disaggregated and reanalyzed across several time periods in order to assess whether the geographic distribution of authorship has been stable or has varied significantly over time. The pre-1970 corpus was distributed across 11 countries with non-American authors accounting for 4% of RER articles. During the latter era, authorship has become more diverse, with authors from 32 different countries contributing articles to the journal. Authors from outside of the United States accounted for 21% of the RER corpus during this period.

This analysis was then repeated for the period 2000 to 2021. During this period, articles were published by authors from 29 countries with contributions from outside of the United States representing 35% of the corpus. The 2000–2021 corpus of 465 articles included 26 articles authored in Asia, 4 from Latin America, and none from Africa. Thus, although these longitudinal data suggest a trend of increasing geographical diversity in authorship over time, RER remains a fundamentally American journal.

Influential Authors and Documents Published in RER

The next analysis examined the Scopus citation impact of authors and documents published in RER (see Tables 1 and 2). When interpreting these tables, it is important to note that the citation counts are based on citations of RER authors/documents in other Scopus-indexed documents. Thus, these are labeled “Scopus citations.” Due to differences in the size of the respective document repositories, Scopus citation counts are usually larger than those obtained from the Web of Science and smaller than those obtained from Google Scholar.

OPEN IN VIEWER

Table 1

Top-cited authors of articles published in Review of Educational Research, 1931–2021.

Rank	Author	RER Topical Focus	Articles in RER	Scopus Citations	CPA 1
1	Hattie J.	Teaching and Learning	7	7255	1036
2	Pajares M. F.	Motivation	3	6382	2127
3	Timperley H.	Teaching and Learning	1	5013	5013
4	Fredricks J.	Motivation	1	4335	4335
5	Blumenfeld P.	Motivation	1	4335	4335
6	Paris A.	Motivation	1	4335	4335
7	Abrami P.	Teaching and Learning	9	3402	378
8	Shavelson R.	Motivation	3	3206	1069
9	Pintrich P.	Motivation	2	2941	1471
10	Tinto V.	Motivation	1	2858	2858
11	Slavin R.	Teaching and Learning	16	2856	179
12	Shute V. J.	Teaching and Learning	2	2376	1188
13	Bernard R.	Teaching and Learning	5	2375	475
14	Borokhovski E.	Teaching and Learning	5	2375	475
15	Kulik J. A.	Teaching and Learning	10	2190	219
16	Sirin S.	Learning	1	2189	2189
17	Cooper H. M.	Review Methods	10	2119	212
18	Hoy, W.	Teachers	2	2079	1040
19	Tschannen-Moran, M.	Teachers	2	2079	1040
20	Hubner J.	Motivation	1	2062	2062
20	Stanton G.	Motivation	1	2062	2062

1

CPA=citations per article.

OPEN IN VIEWER

Table 2

Top-cited documents published in Review of Educational Research, 1931–2021.

Rank	Author	Review Method	Topical Focus	Scopus Citations	Cites per Year
1	Hattie & Timperley. (2007). The power of feedback.	Meta-analysis	Feedback; Learning	5013	334
2	Fredricks, Blumenfeld, & Paris. (2004). School engagement: Potential of the concept, state of the evidence.	Conceptual Synthesis	Student Engagement	4335	289
3	Pajares M.F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct.	Conceptual	Teacher Beliefs	3701	247
4	Tinto. (1975). Dropout from higher education: A theoretical synthesis of recent research.	Conceptual	Student Engagement; Higher Ed	2858	191
5	Sirin. (2005). Socioeconomic status and academic achievement.	Meta-analysis	Learning	2189	146
6	Pajares M. F. (1996). Self-efficacy beliefs in academic settings.	Conceptual; Synthesis	Self-efficacy	2093	140
7	Shavelson, Hubner, & Stanton. (1976). Self-concept: Validation of construct interpretations.	Conceptual; Method	Self-efficacy	2062	137
8	Shute. (2008). Focus on formative feedback.	Synthesis	Feedback; Learning	1982	132
9	Hofer & Pintrich. (1997). The development of epistemological theories. . .	Conceptual; Method	Student Beliefs	1774	118
10	Butler & Winne. (1995). Feedback and self-regulated learning: A theoretical synthesis.	Conceptual	Feedback; Learning	1742	116
11	Tschannen-Moran, Hoy, & Hoy. (1998). Teacher efficacy: Its meaning and measure.	Conceptual; Synthesis	Self-efficacy	1685	112
12	Cummins. (1979). Linguistic interdependence and the educational development of bilingual children.	Conceptual	Bi-lingual Education	1469	98
13	Finn. (1989). Withdrawing from school.	Conceptual	Student Engagement	1346	90
14	Bus, Van Ijzendoorn, & Pellegrini. (1995). Joint book reading makes for success in learning to read . . .	Meta-analysis	Reading	1273	85
15	Jennings & Greenberg (2009). The prosocial classroom: Teacher social and emotional competence. . .	Conceptual	Teacher Attitudes	1235	82
16	Cohen, E. (1994). Restructuring the classroom: Conditions for productive small groups.	Conceptual	Learning; Classroom Context	1234	82
17	Witkin et al. (1977). Field-dependent and field-independent cognitive styles and their educational implications.	Synthesis	Cognition	1198	80
18	Topping. (1998). Peer assessment between students in colleges and universities.	Mixed Method	Assessment; Higher Ed.	1194	80
19	Pintrich, Marx, & Boyle. (1993). Beyond cold conceptual change: The role of motivational beliefs . . .	Conceptual	Motivation; Thinking; Classroom Context	1167	78
20	Springer et al. (1999). Effects of small-group learning on undergraduates . . .	Meta-analysis	Learning; Higher Ed; Context	1150	77

The significance of the data presented in Table 1 lies not only in the identification of key authors and the foci of their reviews, but also in the magnitude of the citation counts. For example, Table 1 indicates that John Hattie has accrued 7,255 Scopus citations from his seven research reviews published in RER (>23,000 Google Scholar citations). Moreover, as a group, the 20 top-cited RER authors average 1,802 Scopus citations per article published in RER. By way of comparison for readers who may be more familiar with Google Scholar citation metric, the single article authored by Fredricks et al. (2004) (4,335 Scopus citations) had gained over 11,500 Google Scholar citations during the equivalent time period.

These citation statistics should be interpreted as an extremely high level of impact for the individual scholars as well as for the journal. Comparison of these results with the top-cited articles published in the American Educational Research Journal (AERJ), another highly ranked, AERA-hosted education journal, reinforce this perception of extremely high citation impact. The top-cited article published in AERJ (Garet, Porter, Desimone Birman, & Yoon, 2001) had accrued 2,283 Scopus citations (>8,000 Google Scholar citations) through the end of 2021. Moreover, the average citation count for the 20 top-cited articles published in AERJ was 739 Scopus citations compared with the average of 1,802 citations across the 20 top-cited articles in RER.

Several other interesting patterns stand out in Table 1. Despite its 90-year publication history, none of the highly cited authors in Table 1 are primarily associated with RER’s pre-1970 period. Further analysis found that the top-cited authors of the pre-1970 era accumulated only 67 total citations (i.e., R. Bock, E. Cramer), and only 10 authors had accrued even 25 Scopus citations through 2021(not tabled). Finally, RER’s top-cited authors have tended to focus on issues related to teacher and student self-efficacy, teacher beliefs and expectations, learner motivation and engagement, and teaching and learning strategies. This pattern is elaborated in subsequent analyses.

The next analysis focused on the identification of RER’s most highly cited articles (see Table 2). Both the high magnitude of citation impact and distribution of topical foci reprise findings from the author citation analysis in Table 1.

Perhaps the most striking finding in Table 2 lies in the classification of articles by “type of review.” Based on the high citation impact of meta-analytic and systematic reviews in various subfields of education (e.g., Dochy, Segers, Van den Bossche, & Gijbels, 2003; Hattie, 2009; Issenberg, Mcgaghie, Petrusa, Gordon, & Scalese, 2005; Prince, 2004; Robinson, Lloyd, & Rowe, 2008), the author had expected meta-analytic reviews of research to dominate the list of highly cited reviews published in RER. Yet, only four meta-analytic reviews featured among the 20 top-cited RER articles (i.e., Bus, Van Ijzendoorn & Pellegrini, 1995; Hattie & Timperley, 2007; Sirin, 2005; Springer, Stanne & Donovan, 1999). Instead, the most highly cited reviews published in RER have tended to be conceptual syntheses (e.g., Fredricks, Blumenfeld, & Paris, 2004; Pajares, 1992, 1996; Shavelson, Hubner, & Stanton, 1976; Tinto, 1975). The implications of this finding will be discussed later in the article.

Another interesting finding in this table lies in the publication dates of the top-cited articles. The publication years of these 20 articles ranged from 1975 (Tinto, 1975) to 2008 (Shute, 2008). Yet, the median publication year was 1995. The interpretation of this finding is informed by Price’s (1965) conclusion that, “the chance that any one paper will be cited by any other, later paper decreases exponentially by about a factor of 2 every 13.5 years” (p. 513). This suggests that we might have expected the median publication year of the top-cited articles to be somewhat later. The sustained impact these articles continue to have on scholarship decades after their publication highlights the importance of emphasizing not only recent literature but also “classic papers” in the education of young scholars.

Intellectual Structure of the Research Corpus Published in RER

A signature feature of science mapping is the ability to reveal the “intellectual structure” or key research theoretical streams of scholarship that evolve within a discipline, line of research, or journal (Hallinger & Kovačević, 2019; White & McCain, 1998; Zupic & Čater, 2015). The first step in conducting author co-citation analysis (ACA) was to generate a list of authors most frequently cited in the reference lists of RER articles (see Table 3). The significance of these authors lies in their influence on shaping the intellectual content of RER. This contrasts with Scopus citation analysis, which examined the impact of RER articles on the broader literature (see Table 2).

OPEN IN VIEWER

Table 3

Authors most frequently cited in articles published in Review of Educational Research, 1931–2021.

Rank	Author	Topical Focus	School of Thought 1	Citations in Reference Lists
1	Hedges, L.	Quant Review Methods	Meta-Reviews	424
2	*Slavin, R.	Teaching/Learning	Meta-Reviews	379
3	*Walberg, H.	Teaching/Learning	Teaching & Learning	297
4	Glaser, R.	Cognitive science; Learning	Cognition	296
5	Anderson, R.	Teaching/Learning	Cognition	293
6	Johnson, D.	Teaching/Learning	Meta-Reviews	260
7	Glass, G.	Quant Review Methods	Meta-Reviews	249
8	Marsh, H.	Quant Methods; Self-concept	Meta-Reviews	246
9	Brown, A.	Teaching/Learning	Cognition	238
10	*Cooper, H.	Review Methods; Expectations	Meta-Reviews	223
11	Bandura, A.	Self-concept/efficacy	Motivation	197
12	Cronbach, L.	Quantitative Methods	Teaching & Learning	193
13	Johnson, R.	Teaching/Learning	Meta-Reviews	192
14	Rosenthal, R.	Review Methods	Meta-Reviews	188
15	Levin, J.	Reading; Quantitative Methods	Cognition	185
16	Eccles, J.	Gender; Expectations	Motivation	184
17	Fuchs, L.	Teaching/Learning	Meta-Reviews	182
18	Cohen, J.	Quantitative Methods	Meta-Reviews	174
19	Fuchs, D.	Teaching/Learning	Meta-Reviews	161
20	Piaget, J.	Child Development	Cognition	158

1

These schools of thought listed in this column are keyed to the four schools displayed in Figure 2 *Indicates that the author also appeared in Table 1, which analyzed Scopus citation impact.

The added value of author co-citation analysis is immediately discerned by the limited overlap between the authors listed in Tables 1 and 3. Indeed, only three authors feature on both lists (i.e., Slavin, Walberg, Cooper). The reason for this pattern of results is suggested by data in the “topical focus” column. These highlight the influence of scholars noted for publications on quantitative review methods (e.g., L. Hedges, G. Glass, H. Cooper, H. Marsh, R. Rosenthal, J. Cohen) and psychological theories that RER authors have applied to research on cognition, learner motivation, teaching and learning (e.g., R. Anderson, R. Glaser, A. Brown, A. Bandura, J. Levin, J., Piaget). Notably, although several of these influential scholars never published in RER, their research published in other sources has shaped the reviews published in RER (e.g., Bandura, 1982; Brown et al., 1989; Cohen, 2013; Piaget, 2003).

Next, VOSviewer was used to create an author co-citation map aimed at revealing the “schools of thought” that cohere to represent the “intellectual structure” of RER’s publication corpus. As explained earlier, this analysis exploits the “relational” feature of co-citation analysis that can be used to visualize, “authors and author clusters within the two- or three-dimensional ‘intellectual space’ of a mapped display” (McCain, 1990, p. 433). Because the software bases author locations in the map on similarities derived from patterns of author co-citation, scholars who are frequently co-cited in the reference lists of RER articles tend to be clustered together. The clustering patterns can be interpreted to represent key research streams or schools of thought that comprise the body of knowledge represented by the RER corpus (McCain, 1990; White & McCain, 1998; Zupic & Čater, 2015).

Interpretation of the map in Figure 3 follows several guidelines. First, the size of an author’s node reflects the frequency with which the scholar has been cited in the reference lists of RER articles. Because this ACA map was produced using a threshold of 62 co-citations, the smallest nodes on the map highlight scholars that reflect this threshold. In contrast, the largest nodes are associated with authors who gained upward of 200 co-citations (see partial details in Table 3). Also of significance is the proximity of author nodes that suggest the degree of “intellectual affiliation” among scholars. For example, in Figure 3, Larry Hedges, Ingram Olkin, and Henry Cooper are located in close proximity to one another due to their common focus on quantitative review methods. Scholars with a “weak co-citation relationship” are located farther apart on the map (e.g., Larry Hedges and Herbert Simon). “Links” connecting pairs of authors offer a visual representation of their co-citation by other authors. The software also groups authors into colored clusters based on patterns of similarity derived from their co-citation relationships. We interpret the colored clusters as “schools of thought” (van Eck & Waltman, 2014; White & McCain, 1998).

OPEN IN VIEWER

Figure 3.

Author co-citation map showing four schools of thought comprising the published content in Review of Educational Research, 1931–2021 (threshold 62 citations of authors in the reference lists of RER articles, display 136 authors).

Four “schools of thought” emerged from the author co-citation analysis presented in Figure 3. Interpretation of the “meaning” of each of the schools is derived from the researcher’s understanding of the scholarly works published by the authors comprising the author cluster. On this map, the four clusters are largely self-contained and coherent in form, occupying different physical spaces on the map. This suggests “distinctiveness” in the respective intellectual foci of the four “schools of thought.” Note, however, that even within a cluster the principle of “proximity” remains relevant. Thus, for example, in the green cluster the scholarship of authors such as Gage, Shavelson, and Cronbach features a stronger similarity with one another than with scholars located at the lower end of the cluster (e.g., Levin, Hanushek).

The central region of the map is anchored by a school of thought comprised of authors associated with research on Teaching and Learning Strategies and Effects. Notably, this is the only school whose authors have substantial “links” to authors in each of the other three schools. Leading members of this school include Herbert Walberg, Jere Brophy, Thomas Good, Lee Schulman, Penelope Peterson, James Coleman, Henry Levin, Nate Gage, and Richard Shavelson. Some of these scholars are known for theoretically informed contributions that focus on how the organization of schools influences teaching and learning (e.g., Coleman, Levin, Walberg), whereas others have focused on teaching and learning practices (e.g., Brophy, Good, Schulman). For example, Henry Levin’s publications on educational effectiveness (Bowles & Levin, 1968; Levin & McEwan, 2000) and accelerated schools (Levin, 1987) represent distinctive theory-informed, research-based applications to education policy and practice related to teaching and learning. Similarly, Nate Gage is known for his applications of psychological theory to understanding teaching processes (Gage, 1963, 1978).

The largest school of thought, located on the right-hand side of the map, is concerned with Cognition, Teaching, and Learning. Scholars located in this school have linked cognitive and social psychological theories to the study of teaching and learning (Bransford, Brown & Cocking, 2000; Brown, Collins, & Duguid, 1989; Lave & Wenger, 1991; Vygotsky, 1978). Leading members of this school include Robert Glaser, Richard Anderson, Ann Brown, J. R. Levin, Jean Piaget, John Bransford, Loren Resnick, and Herbert Simon. Scholars in this school are responsible for advancing our understanding of how information processing influences learning (e.g., Craik & Lockhart, 1972; Simon, 1962, 1979), how children and adults develop capacities for thinking and problem-solving (Barron et al., 1998; Bransford et al., 1986; Glaser, 1984; Piaget, 2003), and the social construction of learning (e.g., Anderson, Reder, & Simon, 1996; Brown et al., 1989; Lave & Wenger, 1991; Vygotsky, 1978).

Authors comprising the blue cluster are noted for Meta-analytic Reviews and Methods. This school is led by Larry Hedges, Robert Slavin, David and Robert Johnson, Gene Glass, Henry Cooper, Herb Marsh, Richard Rosenthal, and Lynn and Doug Fuchs. Note that although Herbert Walberg is physically located in the Teaching and Learning school, his node is located very close to this school, thereby reflecting his publication of meta-analytic reviews on teaching and learning processes and effects. Walberg’s location also reflects the fact that an author’s cited publications are not always restricted to a single line of inquiry.

The frequent citation of scholars in this school by RER authors can be traced to two distinct sources of influence. First, one group of scholars in this school has specialized in publishing meta-analytic and other quantitative reviews of research on teaching, learning, and schooling. For example C. Kulik and J. Kulik have published meta-analytic reviews of research on the effects of ability grouping, computer-based instruction, and mastery learning (Cohen, Kulik, & Kulik, 1982; Kulick et al., 1980, 1985). David and Robert Johnson were responsible for evaluating and synthesizing results from a long-term program of research on cooperative learning (Johnson & Johnson, 1974; Johnson et al., 1981). Robert Slavin is noted for his reviews of different teaching and learning strategies (Slavin, 1980, 1990), as is John Hattie (Hattie, 2009; Hattie & Timperley, 2007).

A second line of scholarship within this school has detailed the “methodologies” used in conducting systematic reviews of research. This subset of authors pioneered and has continued to refine the systematic method of research review that has become a mainstay of RER and research review more generally (e.g., Glass, 1977; Glass, Smith, & McGaw, 1981; Hedges & Olkin, 1982, 2014; Hedges, Tipton, & Johnson, 2010; Lipsey & Wilson, 2001; Rosenthal, 1979; Slavin, 1984). Scholars specializing in statistical methods and research design are included in this school due to frequent citation of their technical guidelines for quantitative analysis (e.g., Campbell & Stanley, 2015; Cohen, 2013; Rosenthal, 1979). More broadly, scholars within this subset have been instrumental in defining and elaborating methods associated with systematic review methods (e.g., Cooper, 1982, 1988; Cooper & Rosenthal, 1980; Light & Pillemer, 1984; Slavin, 1986). Thus, these scholars are frequently cited by RER authors to support their methodological approaches to research review.

The fourth school of thought is concerned with Motivation and Learning. The smallest of the four schools, it is led by scholars such as Albert Bandura, Jacquelynne Eccles, Edward Deci, Bernard Weiner, and Paul Pintrich. Strongly influenced by the seminal construct of “self-efficacy” (Bandura, 1977, 1982; Pajares, 1996; Shavelson et al., 1976), scholars in this school have theorized on the impact of beliefs and expectations held by students (Pintrich et al., 1993) and teachers (Pajares, 1992; Tschannen-Moran, Hoy, & Hoy, 1998). These motivational constructs have been studied in relation to student engagement in learning (Deci, 1971; Dweck, 1986; Dweck & Leggett, 1988; Pintrich, 2003; Pintrich et al., 1993) as well as schooling (Fredricks et al., 2004; Maehr & Midgley, 1991).

Locating RER’s Place in the Constellation of Academic Journals

Next, co-citation analysis was used to map the community of journals in which RER is located (Ding et al., 2000). This analysis begins with an inspection of the “journals most frequently cited” in the reference lists of RER articles (see Table 4). Journals associated with the discipline of psychology account for half of the 20 journals most frequently cited by RER authors. This emphasizes the outsized influence that psychology has had in shaping the content and direction of RER.

OPEN IN VIEWER

Table 4

Journals most frequently cited in articles published in Review of Educational Research, 1931–2021.

Rank	Source	School of Thought 1	Citations in References Lists	Scopus Quartile	SJR Quartile 2
1	Journal of Educational Psychology	Ed Psych	2794	Q1	Q1
2	Review of Educational Research	Ed Research	2187	Q1	Q1
3	American Educational Research Jnl	Ed Research	1299	Q1	Q1
4	Diss. Abstracts International	Psychology	1152	—	—
5	Child Development	Ed Psych	964	Q1	Q1
6	Ed Researcher	Ed Research	865	Q1	Q1
7	Psychological Bulletin	Psychology	761	Q1	Q1
8	Jnl of Personality & Social Psych	Psychology	684	Q1	Q1
9	American Psychologist	Psychology	667	Q1	Q1
10	Jnl of Research in Science Teach	Cog Tch & Lrng	630	Q1	Q1
11	Journal of Educational Research	Ed Research	599	Q1	Q2
12	Harvard Educational Review	Ed Research	548	Q1	Q1
13	Reading Research Quarterly	Ed Psych	474	Q1	Q1
14	Sociology of Education	Ed Research	470	Q1	Q1
15	Developmental Psychology	Ed Psych	469	Q1	Q1
16	Teachers College Record	Ed Research	436	Q1	Q1
17	Elementary School Journal	Ed Research	429	Q1	Q1
18	Journal of Learning Disabilities	Ed Psych	416	Q1	Q1
19	Educational & Psych Measurement	Psychology	413	Q1	Q1
20	Psychological Review	Cog Tch & Lrng	384	Q1	Q1

1

Refers to school of thought labeled in Figure 3.

2

Scimago Journal Rank.

The sources most frequently cited by RER authors have been the Journal of Educational Psychology (2,794 citations) and RER (2,187). Note also that every journal listed in Table 4 ranks in the top Scopus quartile; indeed each of these journals has achieved the 96th percentile ranking or higher in their respective fields (not tabled). This implies that content published in RER has been informed by the highest quality academic research in education and psychology. Therefore, although this review did not directly evaluate the quality of individual articles published in RER, this finding offers a second-order indicator of high academic quality.

Next, VOSviewer was used to produce a journal co-citation map (see Figure 4) that visualizes the “community of journals” that have most significantly influenced the RER corpus (Ding et al., 2000; van Eck & Waltman, 2014; Zupic & Čater, 2015). McCain (1991) earlier demonstrated how, “journal cocitation data [can be used to] highlight subject relatedness, research specializations, and important dimensions of scholarship” (p. 295). Using a threshold of 100 citations in the reference lists of RER articles, VOSviewer generated a display of 87 journals. This implies that journals with the smallest nodes on the map have been cited at least 100 times in RER articles.

OPEN IN VIEWER

Figure 4.

Journal co-citation map showing the constellation of journals cited in the reference lists of articles published in Review of Educational Research, 1931–2021 (threshold 100 citations, display 87 journals).

The community of journals most frequently cited in RER are located in four conceptual fields (see Figure 4). These journal clusters are balanced in size and largely distinct from one another. The dominant cluster, based on the number of affiliated journals and citation frequency, is comprised of Educational Research journals. Leading journals in this cluster include RER, AERJ, Educational Researcher, Journal of Educational Research, Harvard Educational Review, Sociology of Education, Elementary School Journal, and Teachers College Record (see also Table 4). RER’s inclusion in this cluster reflects the frequent citation of previous RER articles by authors publishing in the journal. Journals located in this cluster examine issues and report findings that relate directly to educational research, policy, and practice. As such, this cluster tends to encompass scholarship reported by authors who are located in the Teaching and Learning and Meta-analysis schools of thought in the ACA map in Figure 3.

The next most influential cluster is comprised of Educational Psychology journals. This cluster’s influence on RER content is anchored by the Journal of Educational Psychology. However, articles published in Child Development, Journal of Learning Disabilities, Reading Research Quarterly, Developmental Psychology, Exceptional Children, and the Journal of Applied Behavior Analysis have also made significant contributions toward shaping the content of RER. The titles of these journals suggest the range of theoretical and educational content on which RER authors have drawn. These include child development, reading, counseling, and special needs education (e.g., Bus et al., 1995; Fuchs & Fuchs, 1986; Mayer & Greeno, 1972; Wittrock, 1974).

The third cluster consists of Psychology journals. Leading journals include Psychological Bulletin, Journal of Personality and Social Psychology, American Psychologist, Educational and Psychological Measurement, Journal of Educational Measurement, and the Journal of Counseling Psychology. Cited articles published in these journals have detailed empirical findings of research on learning (e.g., Deci, 1971; Johnson et al., 1981), offered theoretical support for the analysis of teaching and learning (e.g., Bandura, 1982; Bransford, Sherwood, Vye, & Rieser, 1986; Dweck & Leggett, 1988; Glaser, 1984), and supplied methodological resources for conducting systematic reviews of research (e.g., Cooper & Rosenthal, 1980; Rosenthal, 1979).

The smallest cluster is comprised of journals focusing on Cognition, Teaching, and Learning. Key journals in this cluster include the Journal of Research in Science Teaching, Psychological Review, Science Education, Journal of Experimental Education, and the Journal of Verbal Learning and Verbal Behavior. This cluster of journals has disseminated scholarship authored by scholars located in the Cognitive Science and Learning school of thought (Figure 3) (e.g., Bandura, 1977; Barron et al., 1998; Bransford & Johnson, 1972; Craik & Lockhart, 1972; Dweck, 1986).

Journal co-citation analysis has also been used to locate a particular journal’s conceptual residence within its community of journals. McCain (1991) proposed that, “a journal’s placement (central or peripheral) in the [journal co-citation] map should reflect the congruence between the content of that journal’s articles and the research interests of authors publishing (and citing) in the field represented by the set of journals studied” (p. 292). Because RER is a “generalist” education journal, this visualization also highlights the conceptual organization of education as a field of study (see Ding et al., 2000; Liu, 2005; McCain, 1991).

Limitations

Although science mapping offers an empirical means of analyzing a large corpus of publications, this method of review neither directly evaluates the quality of scholarship, nor does it synthesize research findings. This contrasts with the typical form of research review published in RER. Thus, the findings from this review should be interpreted in concert with the results of reviews that use complementary methods of research review.

A second limitation lies in the tendency for bibliometric analysis to emphasize “dominant trends” in the published literature. Although this serves a useful purpose, it can lead to the under-representation of nondominant voices in the literature (Apple, 1999; Baker, 1999). This was, for example, apparent in the citation analyses that surfaced few authors or documents from the pre-1970 RER era.

Third, as a form of secondary analysis, bibliometric review is limited by the data available for extraction from the source data repository. Unfortunately, at this time, several relevant kinds of data were either unavailable (e.g., gender of authors) or of insufficient quality for inclusion in the review. For example, inconsistency in the inclusion of keywords during the publication history of RER made it impractical to employ keyword analysis to identify the evolution of topical trends in the journal over time.

Fourth, the bibliometric employed in this review did not allow for an in-depth or critical analysis of the sociohistorical context in which RER has evolved. Thus, some readers may feel the need for a more critical, contextualized analysis that makes sense of the trends identified in this review from other perspectives. Complementary reviews that undertake this type of analysis would be welcome.

A final limitation follows from the author’s decision to limit the review to the analysis of a single journal, RER. Despite RER’s status in the field of education, the findings identified in this review cannot be assumed to represent the broader field of “educational research.” Therefore, although the author proposes that the analysis offers some insights into broad trends in educational research, the primary aim of the review has been to gain a more refined understanding of RER’s contribution to the field of education.

Interpretation of the Findings

The 3,022 research reviews published in RER represent a substantial corpus of research-based, theory-informed knowledge about education. The size of this corpus takes on added significance when we consider the broad subdisciplinary coverage featured in the journal (Franklin, 1999; Grant & Graue, 1999). From its earliest days, reviews published in RER have encompassed curriculum, instruction, learning, educational psychology, facilities, finance, administration, policy, research methods, health education, sciences, social studies, learning technology, language learning, and more. Moreover, because articles published in RER seek to synthesize bodies of related research, the RER corpus captures an important portion of the theory-informed, empirically researched knowledge base on education.

Like Grant and Graue (1999), this review found that the RER corpus consists of two distinctive subsets compiled during the pre- and post-1970 eras. This assessment was supported first by descriptive content analysis of the documents, and then by citation analyses. For example, 73% of the pre-1970 reviews have never been cited, compared with less than 1% of the latter corpus. Similarly, no pre-1970 reviews featured among RER’s topic-cited articles, the earliest of which was authored by Tinto in 1975. Indeed, the top-cited pre-1970 document, a review of multivariate methods (Cramer & Bock, 1966), had accrued just 67 citations through 2021. This contrasts with Hattie and Timperley’s (2007) top-cited, post-1970 review which has accrued 5,013 Scopus citations.

Multiple factors could explain this demarcation in the RER corpus as well as the differences in citation impact. The shift in form and content of reviews published in RER coincided with the journal’s change in affiliation from the National Education Association to the American Educational Research Association (Di Vesta & Grinder, 1968). The new direction taken in RER was consistent with the normative focus of its new host organization. Subsequent decades witnessed the growth of educational research as an industry as well as a steadily increasing number of active researchers in the United States and internationally. Also significant in making sense of the citation results is the increasing digitization of educational publication and the launch of many more journals. This has greatly facilitated scholars’ access to journals and resulted in substantial increases in citation frequency (Cope & Kalantzis, 2014; Cope & Phillips, 2014).

Grant and Graue’s (1999) analysis of RER’s journey highlighted the historically American roots of the journal. This conclusion was confirmed empirically in this review, which found that 94% of the RER’s articles have been authored in the United States. This suggests a discrepancy between RER’s worldwide readership and the geographical distribution of its authorship base. Fortunately, the data do suggest a trend of increasing diversification over the past two decades. During this period, articles authored outside of the United States accounted for 35% of RER’s articles. This change was, no doubt, influenced by the 21st-century globalization of education and related expansion of higher education beyond traditional centers of academic research and publication. A geographically diverse set of authors is needed for the field of education to progress in developing a “global knowledge base.”

Nonetheless, despite this progress, important gaps remain. More specifically, even during the past two decades only 6% of RER articles were published outside of Anglo-American-European societies. It is widely accepted that education is a “contextually situated activity” (Brown et al., 1989; Lave & Wenger, 1991). This extends to the “cultural context” in which education is practiced (Biggs, 2001; Frambach, Driessen, Chan, & van der Vleuten, 2012; Fuller & Clarke, 1994; Hallinger, 2018, 2020; Ladson-Billings, 1995; Mertkan et al., 2017). Thus, neither perspectives toward education nor the efficacy of educational practices can be assumed to be universal. This suggests the need to further diversify the geographies of RER’s authorship and integrate research findings from a broader cross-section of societies.

Two decades ago, participants in the RER symposium critiqued the positivist, quantitative ethos that had become increasingly prevalent in the journal’s content since 1970 (Apple, 1999; Baker, 1999; Franklin, 1999; Grant & Graue, 1999; Popkewitz, 1999). The data presented in this review suggest that the trend observed by Grant and Graue (1999) has further intensified during the first two decades of the 21st century. Moreover, no data examined in this review suggest the likelihood of divergence from this trend in the near future.

Aligned with this finding is the broad and deep disciplinary influence of psychology on educational research published in the RER corpus. This was evident in both the author and journal co-citation maps. The author co-citation map featured two schools of thought explicitly grounded in psychology, whereas the journal co-citation map highlighted three psychology-oriented journal clusters. It is not a given that psychology either would or should exert such a profound influence on the research that features in RER. Alternative perspectives and methods drawn from other social sciences deserve more explicit encouragement.

The influence of these trends on the content of RER is also evident in the publication frequency of meta-analytic reviews. The first meta-analytic review published in RER appeared in 1980 (Kulik et al., 1980). Disaggregation of the growth trajectory of meta-analytic reviews published in RER highlights their growing prominence in the journal: 1931–1979 (0%), 1980–1989 (15%), 1990–1999 (17%), 2000–2009 (19%), 2010–2021 (35%). These data affirm that the trend of publishing meta-analytic reviews of research has gathered momentum, even in the wake of the critiques leveled by participants in the 1997 RER symposium and others (Evans & Benefield, 2001; Hammersley, 2001). This was no doubt reinforced by the increasing demand for “evidence-based” research findings, as well as by a broader trend toward the publication of systematic reviews in education, social sciences, and health sciences (Elbourne, Oakley, & Gough, 2001; Gough & Richardson, 2018)

That said, the results of this review suggest a more differentiated story than “the triumph of positivism.” Analysis of the most highly-cited articles published in RER highlights the long-term, sustained contributions of conceptually oriented reviews of research (e.g., Fredricks & Blumenfeld, 2004; Pajares, 1992, 1996; Shavelson et al., 1976; Tinto, 1975). Indeed, the 20 top-cited RER articles contained only 4 meta-analytic reviews. Thus, despite the increased prominence of meta-analytic reviews in RER, it would be a mistake to assume that they represent the sole approach to high-impact research review.

Both citation and co-citation analyses provided empirical support for the conclusion that RER has evolved into a high-impact journal. The citation metrics of RER authors and documents are extremely high, thereby suggesting significant impact on broader scholarship in education. Journal co-citation analysis elaborated on this conclusion by highlighting the constellation of journals whose scholarship has been referenced in RER articles. The 20 journals most frequently cited in RER are all top-tier journals specializing in education and psychology.

At the outset of this review, the author asserted that reviews of research tend to appear only after a topic of inquiry has gained a critical mass of publications (Hallinger, 2014). Therefore, it was proposed that the analysis of content published in RER can offer insight into important streams of research that have evolved in educational research over time. Co-citation analysis of authors and journals provided mutually reinforcing perspectives on the intellectual structure of the knowledge base embedded in the RER corpus. Author co-citation analysis visualized Teaching and Learning Strategies and Effects at the center of the map.

This visualization of the RER corpus further highlights the significance of research on student (and teacher) self-concept, self-efficacy, and expectations conceptualized by scholars in the Motivation school (Bandura, 1977, 1982; Deci, 1971; Dweck, 1986; Maehr & Midgley, 1991; Pajares, 1996; Pintrich, 2003). In turn, these conceptualizations have been studied and validated by scholars in the Teaching and Learning and Meta-analytic schools (Hattie & Timperley, 2007; Huang, 2011; Johnson et al., 1981; Marsh & Martin, 2011; O’Mara, Marsh, Craven, & Debus, 2006). Only if the teacher is successful at motivating and engaging students can learners organize and process information for deeper understanding (e.g., Bransford et al., 2000; Brown et al., 1989; Butler & Winne, 1995; Glaser, 1984; Hattie & Timperley, 2007). Thus, strategies studied by scholars in the Teaching and Learning Strategies and Effects school, which anchors the map, are informed by theories of motivation and cognition. This conception of “education” has evolved into a guiding theme for the field.

Implications

These findings yield several potentially useful implications for RER’s editors and editorial board. First, empirical validation of the bifurcated collection of articles comprising the RER corpus suggests that the pre- and post-1970 articles should be approached as distinct subsets. Because of the citation-based nature of the bibliometric method, the pre-1970 “knowledge base” contained in the RER corpus seldom rose to the fore in this review. Nonetheless, the pre-1970 corpus represents a database of historically relevant articles that is well-suited to historical, narrative, and content analysis methods of review (Franklin, 1999). Such reviews would make a useful contribution toward understanding the evolution of educational research during the 20th century.

A signature feature of bibliometric review lies in the ability to identify “canonical authors and documents” through empirical analysis. This term refers to authors and documents that have achieved high levels of citation impact sustained over a period of several or more decades (Price, 1965; White & McCain, 1998). Given the reputational status of RER and the breadth of citation/co-citation data presented in this review, the author is confident that the authors and documents highlighted in Tables 1 to 3 have produced some of the most valuable syntheses of research-informed knowledge about education (e.g., Brown et al., 1989; Fredricks & Blumenfeld, 2004; Hattie & Timperley, 2007; Johnson et al., 1981; Pajares, 1992, 1996; Shavelson et al., 1976; Sirin, 2005; Tinto, 1975) and research review methods (e.g., Cooper, 1982, 1988, 1998; Hedges & Olkin, 2014; Light & Pillemer, 1984; Lipsey & Wilson, 2001; Slavin, 1986). The fact that these authors and articles continue to be cited frequently, 30 and 40 years after publication, testifies to their enduring influence on scholarship in education.

This review also demonstrated how, over the course of the past five decades, quantitative and meta-analytic reviews have gained increasingly prominent status in RER. This trend can be interpreted, at least in part, as a response to the demands of government, foundations, funding agencies, policymakers, and practitioners. These consumers of education research increasingly seek the consolidation of empirical research findings in the form of conclusions on “what works in schools and classrooms, and under what conditions” (Feuer et al., 2002; Glass, 2000; Mann, 1994). Nonetheless, results reported in this review also highlight the long-term impact of reviews that emphasize conceptual trends and breakthroughs and that employ other review methods (e.g., Cohen, 1994; Pajares, 1992, 1996; Pintrich et al., 1993; Tinto, 1975).

These findings support Grant and Graue’s (1999) plea for diversity in the perspectives and methods used to review research and that are legitimated by publication in RER (Cope & Kalantzis, 2014; Post, 2014). Empirical findings reported in this review suggest that continuing to increase the proportion of meta-analytic reviews published in RER could yield a diminished return, in terms of scholarly influence. Conceptually oriented reviews that critique and integrate past research play a complementary, but no less important, role in supporting the consolidation and accumulation of knowledge. Indeed, these reviews often chart the theoretical and methodological directions for subsequent empirical studies that become the grist for the meta-analytic mill. This should give food for thought not only to future contributors to the journal, but also to the RER’s editors and manuscript referees.

The imbalanced geographical distribution of the RER corpus implies a need for actions aimed at maintaining the journal’s long-term relevance in an increasingly global educational enterprise. The imperative for geographical-cultural diversity in the production (and synthesis) of knowledge reprises arguments about “voice” and the “social production of knowledge” asserted by participants in the 1997 RER symposium (e.g., Apple, 1999; Baker, 1999; Popkewitz, 1999). Indeed, the implications posed by the “cultural construction of knowledge” are no less pertinent when approaching the synthesis of educational research than those of race and gender (Frambach et al., 2012; Hyland, 2008; Kempner & Makino, 1993; Mertkan et al., 2017). This leads the author to ask: “What should RER’s mission be in contributing toward the development of a globally validated knowledge base in education, and what does this imply with respect to the journal’s publication strategies?”

A similar implication arises from the dominance of conceptual orientations and research methods associated with the discipline of psychology in the pages of RER. In the absence of self-critical reflection, there will be a tendency for the journal to reproduce this past orientation. For example, the assignment of reviewers implicitly establishes the norms of what constitutes “high impact” and “legitimate” scholarship. Should the journal offer more room for paradigmatic contributions outside of psychology and educational psychology? Changes of this sort will require a formal commitment from the editors and the editorial board.

Although the journal maintains an open submission policy, the publication process has several points of leverage where editors can exercise discretionary wisdom (Cope & Kalantzis, 2014). These offer degrees of freedom that might foster a more culturally diverse set of voices in the pages of RER. For example, editors have the discretion to establish theme issues or prioritize particular topics. This was the case in 2008–2009 when additional articles were published in RER to memorialize Asa Hilliard’s contributions to the field of education.

Additionally, the editors could consider expanding the proportion of “international” members on RER’s editorial board. Currently, only 7 of the 58 members (12%) of the journal’s editorial board are located outside of the United States, and only 5 (9%) from outside of North America. No current members of RER’s editorial board are located in Asia or Latin America, and only 1 in Africa. The absence of any editorial board members from China is particularly surprising as Chinese scholars are poised to become a principal source of educational research in the coming decade.

Expanding the geographical diversity of the board could have several positive knock-on effects. First, the voices of international board members would contribute to shaping the journal’s direction at the strategic level. Second, an internationally diverse editorial board would send a message to education researchers that the journal actively seeks and values manuscripts from a global base of scholars. Third, their inclusion could lead to the emergence of new conceptual perspectives embedded in manuscripts submitted to RER. Fourth, a more globally representative editorial board would have the practical effect of expanding the reviewer base for the journal. This would implicitly reshape perceptions of topical importance when reviewers evaluate manuscripts and make publication recommendations. The author asserts that addressing this imbalance now through these and other measures will place the journal in a stronger position at a time when educational research, as field of study, is transitioning away from an Anglo-American center. For example, it was recently reported that between 2018 and 2022 China overtook the United States in the publication of both scientific research output and “high impact” studies (Lu, 2022). What role should RER play in this changing global education landscape?

This review also has implications for the way we think about the possibilities and constraints of editorial leadership. In the 1997 symposium, participants linked RER’s shift toward a positivist, empirically oriented model of research review to decisions taken during the earlier editorial tenures of Gene Glass and Samuel Messick during the 1970s and 1980s. Yet, the editors who convened the symposium, Carl Grant and Elizabeth Graue (1999), lamented their own inability to reorient the journal’s direction. Their experiences as editors were shaped by the publication environments in which they operated and highlight the constraints on editorial leadership (see Figure 1).

The editorial changes initiated by Glass and sustained by Messick rode a sea change in the broader direction of mid-20th-century education and social policy (Mann, 1994; Popkewitz, 1999). This period marked the growth of “funded research” in concert with a belief among government policymakers that education research “should” be applied scientifically toward the solution of identified problems (Glass, 1976, 1977; Mann, 1994). Changes in RER’s approach also reflected the normative influence of the American Educational Research Association, the journal’s new host. These environmental forces reshaped the editor’s and reviewers’ perceptions of priority topics as well as valid methods of research review (Cope & Kalantzis, 2014; Mann, 2004; Post, 2014). In contrast, the social justice perspective advocated by Grant and Graue (1999) ran up against a strengthening tide of technocratic, accountability-oriented forces in the education environment at the turn of the millennium (Au, 2011; Feuer et al., 2002; Schoen & Fusarelli, 2008).

The author suggests that today all journals are operating in a dynamically changing publication landscape. Whereas published scholarship in education has traditionally been dominated by authors located in Anglo-American societies, the past decade has witnessed dramatic growth in both readership and scholarship in other parts of the world. While data presented in this review suggest that RER, as a journal, has begun to grasp this change, proactive leadership will be needed to sustain the journal’s relevance and status in the next five decades.