Edifying the Body: Troubling Notions of Impact When Discussing EAQ's Intellectual Structure

“Impact” is Complex: Considering Gender, Race, and Internationality

Bibliometrics as a collection of methods is purported to help understand the impact of scholarly work within a field and outline a thematic overview of any given field. An Empirical Reflection on Educational Administration Quarterly's Distinctive Contributions to the Field, utilizes bibliometric methods to analyze trends in EAQ's publication history to highlight the prevalent schools of thought that permeate the field by way of the journal. One of the major arguments presented in Hallinger et al. centers on the impact articles published in EAQ has had on the field of educational administration. This argument hinges on using citation as a proxy for scientific quality, with the primary assumption being that an author's citation number directly corresponds with impact in their field. In comparison, it is true that in academic spaces, citation number functions as an implicit currency that is used to approximate the value of an idea or concept; it is also known that much like systems of actual currency, there is a long history of complex social structures that trouble notion of value.

One of the first issues is that citation metrics are inherently biased. This bias is rooted in a systemic, historical web of marginalization that has implicitly privileged certain voices and certain narratives. Davies et al. (2021) argue that the lack of diversity in “most cited” articles within a given field can partially be attributed to the historical demographics of faculty and those in academic leadership positions. This phenomenon is apparent when observing the extant literature on differential citation behaviors based on race and gender. A growing body of literature details the significant gender-based citation gaps across multiple fields, reaching 30% or more in some fields (Dion et al., 2018; Maliniak et al., 2013; Odic & Wojcik, 2020). Sexism in citation and research dissemination is pervasive and well documented (Davies et al., 2021). Often women, (through this section, I use the term “women” to acknowledge and be inclusive of the experiences of all who identify as women and also recognize that similar challenges exist for nonbinary individuals or any other gender diverse populations) are regularly cited less, receive more manuscript rejections, are invited to provide commentary less, self-cite less, are less likely to be published in prestigious journals, less likely to be published with sole authorship, and are less likely to be given representation at conferences and conventions (Dworkin et al. 2020; Fulvio et al., 2021; Odic & Wojcik, 2020). This occurrence creates a network of exclusions that may negatively impact the citation counts of women in the field, making bibliometrics less reliable for comprehensively addressing impact.

This issue of bias is further compounded when race is taken into consideration. Research on this phenomenon has shown systemic racial discrimination in publication rates, citation rates, and editorial positions (Chakravartty et al., 2018). The metrics used to understand publication patterns reveal that racially and ethnically diverse research teams frequently experience 5% lower acceptance rates than less diverse research teams (Lerback et al., 2020). These factors, when taken together, can lead to citational segregation. In this occurrence, authors prefer citing from the same racial/ethnic group, as shown by white authors who tend to more frequently cite other white authors (Chakravartty et al., 2018; Davies et al., 2021). When bias in citation is operationalized, it helps to reduce the circulation and intellectual recognition of the contributions of nonwhite scholars and the diverse perspectives they provide.

Additionally, we must consider how race and gender simultaneously complicate the citation count metrics, and intersectionality is the proper framing for this. Intersectionality is a critique of the societal and structural obfuscation of the unique experiences of individuals who exist at the interstices of marginalized identities (Collins & Bilge, 2020; Cooper, 2015). Intersectionality calls for an examination and a deconstruction of how structures of domination “interact and intertwine” to influence identity-specific experiences (Harris & Patton, 2019). Our understanding of the machinations of intersectionality in academia helps paint a picture of why citation count as an estimation of value, quality, or impact, detached from structural and historical context, could inadvertently undermine the contributions of women of color scholars (Cordova & Knecht, 2019; Shields, 2012).

The paper highlights the structural systems affecting citation count regarding international entries. The authors identify that EAQ in the last couple of decades has had an increasingly international readership, while most of the published articles remain US-centric. Here the paper posits the reasons for this, citing the demographics of the editorial staff and the reviewers (all most all being from the US) and the way these practices “implicitly perpetuates a US-centric perception of which issues are high impact and of interest to our readers”. This assertion of a US-centric bias in citation is supported by much of the literature in various academic fields (Davies et al., 2021; Harris et al., 2017; Ramírez-Castañeda, 2020). Most of these authors make similar arguments; that the construction of editorial boards, the social dynamics around reviewer-ship, and the perceived capital, both economic and socio-cultural (proximity “western” ideals) of countries where manuscripts originate from, all contribute to an implicit erasure of other knowledges. Hallinger et al. argue that the editorial board must consider addressing the gap if the journal is serious about expanding to international audiences.

The authors' awareness of the international bias and the structures maintaining it is an appropriate and necessary critique of EAQ. What is challenging is that this critique is levied in an article that (a) discusses the impact and implies quality using citation count and (b) does not acknowledge that the same biases may exist for many types of marginalized academic communities. While the goal of this specific paper was to identify dominant schools of thought within EAQ and trace their origin in other fields, the findings open the door for an overall critique of the field at large. I believe this complicates the notion of impact being defined strictly through citation count. While bibliometrics can accurately capture trends, which can be used to construct a narrative, there still need to be accurate when sharing the narrative. To this end, Rousseau and Rousseau (2021) offer various methods and terminology that could more accurately describe what is measured when engaged in bibliometric analysis. The authors position evaluative inquiry to acquire a more nuanced understanding of all factors that situation researchers work in the field. Fochler and de Rijcke (2017), when speaking about an evaluative inquiry from a departmental framing, suggested treating different research activities and output (going beyond just publication) as “punctualizations” of ongoing processes or “nodes” rather than just numerical input. They state, “an inquiry inspired by such relational concepts would not be grounded in the vocabulary of ‘impact,’ but would try to visualize the relations to actors enrolled in the work and describe how the department's research came to matter in the network.” (p. 34). There is a deliberate choice to eschew the language of impact instead of focusing on the entire network where the work is situated. What is encouraging is that the Hallinger et al. piece already does a version of this by trying to map out significant concepts. The problem is just emphasis on impact and highlighting/ranking specific individuals’ contributions reifies the existing structures that perpetuate bias in the field.

Defining Social Justice and Its Role in Educational Administration

The authors identify leadership for learning, teacher development and school change, school culture and organization, and social justice as the dominant schools of thought for material published in EAQ. While highlighting these themes is accurate and helpful for understanding the journal's direction, there are some specifications issues. Particularly with the discussion of social justice, this work is both limiting in the parameters it outlines for social justice work in the educational administration field and reductive in valuing the impact of social justice work in the field. The authors define social justice scholars as those who “have focused on how social inequities are perpetrated and enacted by systems of schooling “(p. 12)—highlighting work with an emphasis on the social inequities around “gender, race and gender orientation” (p.12). The authors also note that equity-oriented scholarship in educational administration “only evolved into a coherent school of thought in the early 2000s” (p.12). To discuss this, they cite many scholars whose work is pivotal for these specific topics and name scholars who are seen as the “leaders” (as determined by the citing metrics). The problem with this practice is twofold: (a) it implicitly distills the broad spectrum of equity-based research in educational administration down to a few papers, ignoring the work that has been built on those papers. (b) It perpetuates the same practice critiqued earlier in this paper—highlighting a few thought leaders (who are mainly white) and generating an exclusionary citational feedback loop. This practice is paradoxical, given that the whole point of social justice is dismantling the very systems that would exclude the viewpoints of diverse scholars. The works of Murphy, Scheurich, Young, Capper, Shields, Lopez, De Leon, Brunner, Marshal, McCarthy, Theoharis, and Anderson are all essential in our understanding of the operationalization of social justice in education administration. However, all those scholars would likely name the origin of the ideas and concepts presented in their papers as rooted in work outside education and outside of academe. Additionally, these scholars would likely acknowledge that their work was not meant to be the end of all social justice work. Instead, they would accept the sprawling body of work that came after and improved upon ideas since social justice work is iterative and highlighting an ideological centering is antithetical.

I will provide some examples of impactful work to demonstrate just how vast equity-oriented work is in EAQ. I completely understand that the act of naming enacts some of the things I argued against earlier. However, this is not meant to be an exhaustive or definitive catalog of work, just some examples. Papers in EAQ have utilized socially frames to discuss; the experiences of diverse school and district administrators (Bass, 2020; Gooden, 2005; Liou & Liang, 2021; Murtadha & Watts, 2005; Rodela & Rodriguez-Mojica, 2020), practices that promote change and transformation in schools and communities (Cooper 2009; DeMatthews et al., 2021; Green, 2017; Ishimaru, 2013; Khalifa, 2012; Rivera-McCutchen, 2021; Santamaria, 2014; Tillman, 2005; Wang, 2019), critiques of law and policy (Gooden & Thompson Dorsey, 2014; Horsford, 2010; Irby, 2014; Lopez & Burciaga, 2014), and research methodology (Bertrand, 2018). All the above papers explore some nuanced dimensions of social justice and equity work, and as stated before, this is not even an exhaustive list of all the work printed in EAQ. For the newer articles, some may argue that a time bias makes it difficult to estimate the impact. However, utilizing the evaluation analysis method described earlier, the effect could be measured by situating these articles within the social justice network in EAQ. Additionally, the older articles cited above endorse the idea of citation bias discussed earlier in this article. My citing of all these scholars is meant to represent that social justice research, much like social justice itself, not only emphasizes the “leaders” but also gives voice to the composite parts that make up the work as a whole.

My second observation An Empirical Reflection on Educational Administration Quarterly’s Distinctive Contributions to the Field regarding social justice is that it reduces the importance of social justice work. In outlining the four schools of thought, the paper describes the social justice cluster as “small” and disconnected from the “main streams” of work in EAQ. The language used here pushes social justice work to the periphery. One exception is the position of the work of Joseph Murphy, as he had publications that reached across all four clusters in the model. The issue with this is that increasingly we understand that social justice work should be embedded across the administrative process. That is to say, that leadership for learning, teacher development, school change, and school culture and organization should integrate equity and social justice in some form when done correctly. Galloway and Ishimaru (2015) argued this in a paper that offered to restructure the existing Interstate School Leader Licensure Consortium (ISLLC) Standards used to shape leadership preparation. Galloway and Ishimaru proposed standards centered on equity as a call to transform leadership preparation to equip school administrators with the tools necessary to improve the lives of traditionally marginalized students. Their standards articulated school improvement, climate, and organization culture as byproducts aligned to equity, in that every aspect of the school (i.e., instructional practice, family engagement, discipline) would be conducted through a lens of equity. While radical at its time, this paper was designed to push the conversation of leadership preparation standards toward acknowledging the importance of equity and social justice work.

Eventually, the ISLLC was replaced by two standards that created space to include equity-centered work. The National Policy Board for Educational Administration (NPBEA) developed the Professional Standards for Educational Leaders (PSEL) in 2015. The Council for the Accreditation of Educator Preparation (CAEP) created the National Educational Leadership Preparation Standards (NELP) in 2018. Both standards explicitly mentioned equity as a goal for school operations (NELP Standard 3—Equity, Inclusivity and Cultural Responsiveness and PSEL Standard 3—Equity and Cultural Responsiveness). The document that covers the PSEL makes a few claims to support the inclusion of equity as a part of effective school leadership. It starts with an evidence-based exploration of the importance of good school leaders. It then positions equity as one of the components working in concert to impact student learning.

Equity is not treated as a tangential or peripheral sub-goal but as an essential part that directly interacts with student learning and with every other component of the school organism. The NELP and PSEL standards have been used to shape curriculum and practice at several leadership preparation programs across the country. I invoke the standards here to make the point that social justice and equity work are crucial for our understanding of leadership preparation. Programs that adopt the standards will likely utilize high-quality articles from EAQ to help shape their structure and curriculum. Concepts like cultural responsiveness and equity audits are used in these spaces because the standards are a marker of impact. Pushing social justice and equity work to the margins and considering it a “small” part of the overall EAQ corpus undermines the value of the work and the ways it may be shaping leadership preparation. This marginalization of the work also conceals its utility and interactions with the other “main streams” of EAQ's intellectual thought. Some may argue that the metrics dictated the positioning of social justice work. However, this specific argument is more about the language used to describe social justice work in relationship to the other clusters and how that creates a hierarchy or canon in educational administration that ignores the vital inclusion of social justice and equity.

A Physical Examination for Edification

Given everything said throughout this paper, I think it is essential to step back and discuss why I chose to respond. Drawing on the imagery of the body (about the corpus of work), I saw this as an opportunity to engage for edification, not in the sense of moral rebuke, but as an exchange of ideas to strengthen the entire body. Fundamentally, the Hallinger et al. article does an excellent job of highlighting many essential things about EAQ. My critique is not intended to bash bibliometrics or any of the hard work done to produce these results, but instead of the conclusions at the end of the paper. It felt like the paper was not highlighting the “who” in terms of impact in EAQ and instead emphasized “what” ideas are valued without critiquing the structures that engineers value. If the work in EAQ is a body, all the parts must work together to strengthen and support its functioning. Thus, I hope my arguments help make the work more potent and push us toward the deconstruction of the systems of power and marginalization in the publishing economy. In critiquing my response, I offer this: (a) as a nascent academic, my arguments likely contain some blind spots to the overall goals of the original paper, and (b) my arguments leverage publications from multiple non-education fields to try and make general comments about citation behaviors which may affect the applicability of my statements. These weaknesses or shortcomings make it possible that my response can be interpreted as a series of considerations instead of some form of reproach.