Automated Text Data Mining Analysis of Five Decades of Educational Leadership Research Literature

Abstract

Purpose: The purpose of this study is to describe the underlying topics and the topic evolution in the 50-year history of educational leadership research literature. Method: We used automated text data mining with probabilistic latent topic models to examine the full text of the entire publication history of all 1,539 articles published in Educational Administration Quarterly (EAQ) from 1965 to 2014. Given the computationally intensive data analysis required by probabilistic topic models, relying on high-performance computing, we used a 10-fold cross-validation to estimate the model in which we categorized each article in each year into one of 19 latent topics and illustrated the rise and fall of topics over the EAQ’s 50-year history. Findings: Our model identified a total of 19 topics from the 1965 to 2014 EAQ corpus. Among them, five topics—inequity and social justice, female leadership, school leadership preparation and development, trust, and teaching and instructional leadership—gained research attention over the 50-year time period, whereas the research interest appears to have declined for the topic of epistemology of educational leadership since the 2000s. Other topics waxed and waned over the past five decades. Implications: This study maps the temporal terrain of topics in the educational leadership field over the past 50 years and sheds new light on the development and current status of the central topics in educational leadership research literature. More important, the panoramic view of topical landscape provides a unique backdrop as scholars contemplate the future of educational leadership research.

Keywords

text data mining educational leadership probabilistic latent topic modeling literature review high-performance computing

Introduction

Since its inception, educational leadership has been an interdisciplinary field, closely interacting with other fields such as sociology, psychology, and economics, as well as the subfields of education such as urban education and teacher education (Bates, 1980; Haller, 1968; Oplatka, 2009; Wang & Bowers, 2016). Over the past 50 years, scholars have surveyed the landscape of educational leadership through examining its research literature (see Bates, 1980; Bridges, 1982; Campbell, 1979; Gunter & Ribbins, 2003; Haller, 1968; Hallinger & Chen, 2015; Hoy, 1994; Murphy, Vriesenga, & Storey, 2007; Oplatka, 2009, 2014; Riehl, 2015; Wang & Bowers, 2016) to understand the amorphous nature of educational leadership, as termed by Bates (1980). Since the 1960s, educational leadership has evolved into a field with diverse conceptual lenses (Boyan, 1981; Donmoyer, 2001; Willower, 1975), various philosophical propositions (Evers & Lakomski, 2012; Willower, 1981), debatable epistemological issues (Oplatka, 2009, 2012), and diverse methods (Briggs, Coleman, & Morrison, 2011; Heck & Hallinger, 2005). Moreover, some researchers have noted that the education leadership field is subject to shifts and fads as politics and policies come and go and come back again with changes in the political and policy environment (Labaree, 2011; Tyack & Cuban, 1995). As a result, the diversification of the field has left scholars grappling with identifying the topic structure and knowledge base that could help inform not only the field of research in educational leadership but also leadership practitioners in schools, policymakers, and the future development of the field (Heck, 2015; Oplatka, 2012). The continual development of educational leadership research pivots on an enriched understanding of the topics and topic evolution in the field, as researchers continue to search the field’s epistemological identity and contemplate the future of the field.

To date, the common practice to identify the topics in the educational leadership literature is to allow the topics to emerge as researchers examine, code, and classify the literature, consisting mostly of journal articles (e.g., Campbell, 1979; Hallinger, 2013; Murphy et al., 2007; Oplatka, 2009). However, despite these valuable efforts, the understanding of all topics studied in the educational leadership research literature and the extent to which they have been studied have been weak to date. This has led to a lack of understanding of what the research literature in educational leadership has focused on, and has led to multiple critiques of the knowledge base as to the interdisciplinary nature of the field (Haller, 1968; Levine, 2005; Wang & Bowers, 2016). In response to Hallinger’s (2014) call for the improvement in the methodology of conducting systematic reviews of research in education leadership, in this study we build on recent innovations in the field of automated text data mining and machine learning to apply probabilistic topic modeling—a suite of automated text mining algorithms that computationally detect latent topic structures from a corpus of documents such as journal articles—to investigate the nature of topics in the educational leadership research literature. As Educational Administration Quarterly (EAQ) has been consistently regarded as the most prestigious research journal in the field (Campbell, 1979; Cherkowski, Currie, & Hilton, 2011; Haas et al., 2007; Murphy et al., 2007; Richardson & McLeod, 2009; Wang & Bowers, 2016), we use probabilistic topic modeling to empirically derive the latent topics discussed by the research literature across the entire history of EAQ starting with Volume 1, Issue 1, in 1965 up through Volume 50, Issue 5, in 2014, as a means to build on the past work of narrative reviews (e.g., Campbell, 1979; Haller, 1968; Murphy et al., 2007). We specifically seek to answer two research questions:

Research Question 1: What are the topics in the educational leadership research literature over the past 50 years in EAQ from 1965 to 2014?

Research Question 2: How do the topics in EAQ change over time, which topics come and go, and which are consistently researched in EAQ from 1965 to 2014?

In the subsequent pages, we begin with a literature review on the history of the epistemological identity of educational leadership research, followed by the methodological challenges of research reviews in the field to date. We then introduce probabilistic topic modeling as an innovative approach to analyze and empirically describe latent topics across articles, a technique developed in the field of text data mining and machine learning, which has been used in a wide variety of fields across the social sciences. Next, we provide details of how we applied the probabilistic topic modeling in mining the research literature in educational leadership. The study concludes with a discussion on how the findings of this study contribute to the educational leadership field.

Literature Review

So, one might ask, does the history of the scholarship of education leadership represent incremental change or . . . punctuated equilibrium? As Donald Willower noted, “the spirit of the times is a slippery concept” (1996, p. 346). Still, looking back, it is hard to characterize education leadership research, theory, and practice over the past several decades in terms of “equilibrium.” In numerous assessments of the field, scholars have portrayed a fairly constant churn of development, disagreement, outright conflict, or contestation (e.g., Willower & Forsyth, 1999). The situation has been stable: flux is constant (Riehl, 2015, p. 225-226).

An Epistemological Identity of Educational Leadership

What is the epistemological identity (the field’s nature, purpose, knowledge base, interdisciplinary boundaries, etc.) of the field of educational leadership? Over the past 50 years, a stream of studies has been devoted to answering this question, as summarized in Table 1. Here we review in detail these prior studies that examined the topics and interdisciplinary nature in EAQ and other educational leadership journals. Haller’s (1968) study might be the first study that presented substantial evidence of the field’s interdisciplinary ideology. To apply citation analysis techniques to the field of educational administration, Haller examined article citations in the first three volumes of EAQ and found that 34.2% of citations referred to education, followed by sociology 32.1%, psychology and social psychology 12.8%, political science 7.0%, economics 2.9%, and anthropology 1.2%.

Table 1.

Summary of Topics or Interdisciplinary Nature in the Educational Leadership Research Literature.

	Literature Sources	Literature Timeframe	Topics or Fields
Haller (1968)	EAQ	1965-1967	Fields: education and sociology, psychology and social psychology, political science, economics, and anthropology
Campbell (1979)	EAQ	1965-1978	10 Topics: politics/policymaking, school finance, decision making, motivation/satisfaction, preparation programs, leadership, administrative behavior, authority/bureaucracy, collective bargaining, and organizational structure
Bridges (1982)	EAQ, six other journals, and Dissertation Abstracts International	1967-1980	7 Topics: attitudes, traits, behavior, impact, expectations, power, and effectiveness
Hoy (1994)	N/A	N/A	7 Topics: societal and cultural influences on schooling, teaching and learning processes, organizational studies, leadership and management processes, policy and political studies, legal and ethical dimensions of schooling, and economic and financial dimensions of schooling
Murphy et al. (2007)	EAQ	1979-2003	12 Topics: organizational theory, profession of school administration, politics, school reform, core technology, gender and race, personnel, law, economics and finance, work of school leaders, philosophy and ethics, and psychology
Oplatka (2009)	EAQ, JEA, and EMAL	First volume: 2007	Diverse topics
Wang and Bowers (2016)	EAQ and 29 other journals	2009-2013	Fields: urban education, teacher education, economics, human resources, sociology, psychology, etc.

Note. EAQ = Educational Administration Quarterly; JEA = Journal of Educational Administration; EMAL = Educational Management Administration & Leadership.

Along with the interdisciplinary nature of educational leadership, the scholarship in the field is characterized by diverse topics (Haller & Knapp, 1985; Hoy, 1978, 1982; Oplatka, 2009). Over the past 50 years, scholars have surveyed the research literature of educational leadership (see Bates, 1980; Campbell, 1979; Gunter & Ribbins, 2003; Haller, 1968; Hallinger & Chen, 2015; Murphy et al., 2007; Riehl, 2015; Wang & Bowers, 2016) to grasp the amorphous landscape of educational leadership, as termed by Bates (1980). As Hoy noted, the research in educational administration “is fragmented and lacks a systemic attack on a series of related problems” (Hoy, 1978, p. 5), and “systematic and cumulative knowledge building is conspicuously absent” (Hoy, 1982, p. 4). Writing more recently, Oplatka (2009) described the overdiversification and the seemingly limitless expansion of the field as “the big bang” (p. 15). However, not all scholars hold such a pessimistic view. Rather, the diversity in scholarly inquiry in educational leadership “is all to the good. New and varied ideas and approaches, or old ones newly applied, and constructive controversy give a field vitality” (Willower, 1981, p. 115). McCarthy (1986) later echoed that the diversity in the field’s intellectual inquiry is “a positive development that reduces parochialism in the field as a whole” (p. 11). While whether topic diversification is instrumental to the educational leadership field is subject to debate, what scholars have agreed on is the premise of the debate: The research in the field has been characterized by diverse topics. Below, we not only review the studies that examined the research topics in educational leadership but also present the methodological challenges these studies encountered as the topics continue to diversify across the 50-year history of educational leadership research literature.

At the 15th anniversary of the founding of EAQ, Campbell (1979), the first editor of EAQ, categorized the topics in the first 14 volumes of the journal (1965-1978). Campbell (1979) detailed his three attempts to categorize the broad topics addressed in EAQ articles, and concluded that developing the categories of topics was “a difficult task and relatively unrewarding” (p. 2). Ten topics (in the descending order of the percentage of EAQ articles addressing the topics) identified by Campbell were politics/policymaking, school finance, decision making, motivation/satisfaction, preparation programs, leadership, administrative behavior, authority/bureaucracy, collective bargaining, and organizational structure.

To examine the state of the art of the research on school administration, Bridges (1982) analyzed 322 research reports published in Dissertation Abstracts International (the Humanities and Social Sciences) and seven journals in the field of educational administration (i.e., Educational Administration Quarterly, Journal of Educational Administration, Administrative Science Quarterly, the Canadian Administrator, Administrator’s Notebook, NASSP Bulletin, and the Alberta Journal of Educational Research) from 1967 to 1980. Bridges (1982) outlined seven topics that emerged: attitudes, traits, behavior, impact, expectations, power, and effectiveness. Bridges harshly criticized that the “studies of school administrators are intellectual random events” (p. 22) and “the more things change, the more they remain the same. The state-of-the-art is scarcely different from what seemed to be in place nearly 15 years ago” (p. 24). Given the field’s excessive obsession on the topics of administrators’ traits and attitudes, Bridges proposed that “studies that merely describe the traits or attitudes should be discontinued unless they shed light on a problem of practical, social, or theoretical significance” (p. 26).

Amid “a great uncertainty and lack of consensus about content—the relevant knowledge for practice and inquiry in educational administration” (Hoy, 1994, p. 178), the University Council for Educational Administration (UCEA) Executive Committee appointed seven study teams, each of which had three to six scholars, to “map the essential knowledge for educational leaders” (p. 179). The appointed 30 scholars worked for 2 years to define the knowledge base in the field. Despite the lack of unanimous agreement by the scholars in seven teams, seven domains of knowledge were identified as the essential knowledge for educational leadership: societal and cultural influences on schooling, teaching and learning processes, organizational studies, leadership and management processes, policy and political studies, legal and ethical dimensions of schooling, and economic and financial dimensions of schooling. Furthermore, Hoy outlined three common themes across the seven knowledge domains in the educational leadership field: (a) goals of education; (b) the analytical paradigm tensions among the traditional methods to test and verify generalizations, theories of positivists sciences, and emerging procedures and perspectives of interpretivists, constructivists, radical humanists, structuralists, and feminists; and (c) the elusive links between causes and effects.

Murphy et al. (2007) built on Campbell’s (1979) work to identify the topics in the field. Using the articles published in EAQ from 1979 to 2003, Murphy et al. (2007) coded the topics as they emerged across time. By doing so, 12 topics were identified: organizational theory, profession of school administration, politics, school reform, core technology, gender and race, personnel, law, economics and finance, work of school leaders, philosophy and ethics, and psychology. Murphy et al. also shared Campbell’s (1979) lamentation that categorizing EAQ articles by topics was “exceedingly arduous” (Murphy et al., 2007, p. 619).

Most recently, Oplatka (2009) outlined the field’s historical landscape and epistemological boundaries in his historical overview of educational leadership scholarship. Drawing on the literature in the three oldest and prestigious journals in the educational leadership field (Educational Administration Quarterly, Journal of Educational Administration, and Educational Management Administration & Leadership), Oplatka used open coding to analyze the study purposes, arguments, epistemological questions, criticism, findings, and insights of the articles that addressed the philosophical, epistemological, and methodological issues in the field. Oplatka found that topics in the field evolved over time: from social science–based topics such as rational planning, democratic leadership, and school finance in the 1960s and the 1970s; education-oriented topics such as value and equity, motivation, organizational culture and climate, and management/policy interface in the 1980s; public and political pressure–driven topics such as policy studies, principal preparation programs, and the quality of research and programs in the 1990s; to the critical reflections on the field between 2000 and 2007 within the context of globalization. Despite Oplatka’s comprehensive and exhaustive review, he noted that it was impossible to adequately identify the tremendous variety of the topics that have been studied in the field, because of the continuously extended knowledge boundaries of educational leadership.

Building on Campbell’s (1979) and Murphy et al.’s (2007) work, Wang and Bowers (2016) constructed and visualized an educational administration journal citation network by extracting 157,372 citations from 5,359 journal articles in 30 educational administration journals from 2009 to 2013. The eight journal clusters detected in the citation network suggest that educational administration is a porous, outward-facing field, intimately interacting with the subfields of education (e.g., urban education and teacher education), other disciplines (e.g., economics, human resources, sociology, and psychology), and the research internationally.

Overall, across this line of literature, there appears to be a consensus that the fuzzy epistemological boundaries are a major concern of the field, which has been inadequately addressed by the research community to date (Oplatka, 2009). Moreover, none of the previous studies longitudinally depicted the entire history of the topic evolution from the 1960s to this day, despite Oplatka’s (2009) remarkable review from 1960s to 2007. This is probably because the ever-growing body of research literature poses enormous methodological challenges in conducting research reviews. Recently, Hallinger (2014) stated that reviews of research are critical for the educational leadership field in terms of knowledge generation and accumulation. However, this critical role of reviews of research has been underappreciated. After conducting a “methodological review of the reviews of research in educational leadership and management” (p. 539), Hallinger called for the improvement in the methodology of conducting systematic reviews of research in educational leadership. However, thus far no methodological approach in educational leadership and in education has surmounted the challenge of analyzing the sheer massive amounts of text data represented across a large and complex body of research journal articles. Therefore, in the present study, we look to the emerging fields of text data mining and machine learning, because these fields are driven by the “Big Data” analytical and technical demands of the influx of high volume, high velocity, and high variety of data generated from digital devices, sensors, and social media (boyd & Crawford, 2012; Kitchin, 2014). Here, we first introduce probabilistic topic modeling, which is a scalable, algorithmic approach to analyze text data (Blei, 2012), and then discuss how we apply it to the educational leadership field as we traced the field’s roots back to the 1960s to examine the topic structure of the field.

Correlated Topic Modeling

Correlated topic modelling (Blei & Lafferty, 2007), a subdomain of latent semantic analysis, is a statistical model that automates text analysis, which we used in this study to empirically identify the latent topic structure across EAQ articles. The goal of topic modeling is to use the observed documents (in this case journal articles) to automatically infer a latent topic structure—the topics, per-document topic distributions, and the per-document per-word topic assignments (Blei, 2012). Traditionally, one does not know the topic of a journal article until we read and cognitively process the article. The distinguishing characteristics of topic modeling is a departure from a researcher’s cognitive process to a suite of automated algorithms, which computationally detects the topic structure without any prior annotation or labeling of the documents. Topic modeling analyzes the correlations and variance–covariance information between the sparse words by documents matrix, and then through matrix algebraic calculations, generates a matrix of terms by latent topics and a matrix of topics by documents. This technique is colloquially referred to in the latent semantic analysis literature as a “bag of words” model (Steyvers & Griffiths, 2007), as each document is considered to be a bag holding a jumbled set of words, as the frequency of words found in each document is correlated to each of the other documents in the data set, with no assumptions implied about word order or phrase linkages. More specifically, probabilistic topic modeling is a statistical modeling technique that automates the analysis of detecting latent topics across documents in a large corpus of documents, such as across newspaper articles (see Yang, Torget, & Mihalcea, 2011) or journal articles (see Blei & Lafferty, 2007). Topic modeling is predicated on the assumption that each document exhibits multiple topics; each topic is a distribution of words in the corpus; and the topics can be generated inductively based on the probability of co-occurrence of words (Blei, 2012).

In topic modeling, a topic is defined as a distribution over the words used in the corpus (Blei, 2012). These words are the only observed variables. For any given topic, there is a corresponding distribution over all the words in the corpus, and the topic modeling provides the most likely words with high probability with respect to each topic. For example, the high-probability terms, such as “life,” “abort*,” “babi*,” “life,” and “ban,” describe the abortion topic identified in Quinn, Monroe, Colaresi, Crespin, and Radev’s (2010) topic model of over 118,000 speeches in the U.S. Senate; the high-probability terms, such as “human,” “genome,” “dna,” “genetics,” and “genes,” describe the genetic topic identified in Blei’s topic model of approximately 17,000 articles from the journal Science (Blei & Lafferty, 2007); the high-probability terms, such as “facilities,” “acquiring,” “refurnishing,” “constructing,” and “equipping,” describe the school facility equipment topic identified in Bowers and Chen’s (2015) topic model of 1,210 school district capital facility bond election ballot proposals in Michigan. In this technique, the topics emerge from the high-probability terms.

Probabilistic topic modeling is different from conventional text data analysis in a couple ways. First, the scalable, algorithmic approach in topic modeling frees up human labor and enables us to analyze large corpora at a scale that would be infeasible by human annotations, such as analyzing tens of thousands of journal articles (see Blei & Lafferty, 2007). Second, topic modeling prevents implicit manual coding bias. Instead of being predefined by researchers, the topics are generated inductively by the topic-modeling algorithms; thus, the latent topic identification is not subject to the constraints of researcher knowledge. To this end, topic modeling is well-suited to analyze large amount of text data, and has proved valuable in many fields in the social sciences. It was used to extract the latent topics from over 73 million words in 118,065 speech documents in the U.S. Senate as recorded from the 1995 to 2004 Congressional Record (Quinn et al., 2010), to study the history of topics in the field of computational linguistics from 1978 to 2006 (Hall, Jurafsky, & Manning, 2008), to identify the topics in 16,351 Science articles from 1990 to 1999 (Blei & Lafferty, 2007), and to detect public opinion in the field of public policy via the text data acquired from blogs and social networking sites such as Twitter and Facebook (Zhang & Ma, 2015).

While topic modeling is a rapidly developing field (Blei, 2011, 2012; Grün & Hornik, 2011; Hofmann, 1999; Junqué de Fortuny, Martens, & Provost, 2013; Lee, Song, & Kim, 2010; Ponweiser, 2012; Steyvers & Griffiths, 2007), its scalable, algorithmic approach has received little attention among educational leadership scholars. The increasing applications of topic modeling in a variety of fields in the social sciences suggests the promising potential of applying topic modeling to the field of educational leadership. As a means to build on the past research reviews of educational leadership research literature (e.g., Bridges, 1982; Campbell, 1979; Haller, 1968; Hoy, 1994; Murphy et al., 2007; Oplatka, 2009; Wang & Bowers, 2016) and to address the methodological limitations of conventional text data analysis, we apply topic modeling to identify the latent topics in the research literature across the entire history of EAQ for the first time, starting with Volume 1, Issue 1, in 1965.

Method

The journal Educational Administration Quarterly was chosen because it has been consistently considered as the most prestigious journal in educational leadership research (Campbell, 1979; Cherkowski et al., 2011; Haas et al., 2007; Murphy et al., 2007; Richardson & McLeod, 2009; Wang & Bowers, 2016). Below, we present in detail how we applied topic modeling to all articles published in EAQ from 1965 to 2014. We illustrate an overview of the research design in Figure 1.

Figure 1.

Overview of research design.

Data Collection

The collection of all EAQ text data itself was a nontrivial task. First, we assembled all EAQ articles. With the permission of the EAQ publisher Sage, the EAQ editor, and the UCEA Executive Director, we downloaded from the EAQ website (journals.sagepub.com/home/eaq) all articles from Volume 1, Issue 1, published in January 1965 to Volume 50, Issue 5, published in December 2014. To acquire EAQ text data in an efficient manner, we used Casper JS (Perriault, 2011)—an open source Java script—to automate the navigation of the EAQ website and the download of all EAQ articles. Next, we excluded articles that were announcements and calls for proposals, because the text data in these documents did not provide substantive content on research studies. We did so by identifying whether the EAQ article titles contained the texts of “William J. Davis Memorial Award,” “Acknowledgement of Reviewers,” or “Call for Proposals.” It is worth noting that “The Editor’s Desk” and “Editor’s Foreword” were included in this study, because the text data in these documents were considered closely related to the published articles in EAQ. “The Authors” were also included in this study’s data collection. This is because unlike the articles in 2003 and thereafter that include the authors’ biography in the same documents as the text of the article, EAQ compiled author biographies and published them in separate documents titled “The Authors” from 1987 to 2002 (i.e., from EAQ Volumes 23 to 38). To ensure the consistency of the data inclusion, we thereby included the “The Authors” in the EAQ corpus. Moreover, “Essay Reviews” and “Book Reviews” were also included, because a recent study suggested that books and reports have been the major source of the knowledge base in the field of educational leadership (Wang & Bowers, 2016).

Last, we used optical character recognition technology (Smith, 2015) to convert all of the text data in EAQ articles from PDF format to .txt (i.e., text) format. We used optical character recognition to extract text data because not all EAQ articles were digitized, particularly the articles from Volumes 8 to 24 (see Fikis, Wang, & Bowers, 2015, for the evidence of digitized and undigitized EAQ articles according to the file size in bytes of the PDF text import). As a result, every word in each EAQ article was retrieved, including article title, author, main manuscript, in-text citations, and references. While some text mining studies in other disciplines only analyzed the text data in article abstracts (e.g., Evangelopoulos, Zhang, & Prybutok, 2012; Sidorova, Evangelopoulos, Valacich, & Ramakrishnan, 2008), we followed the current recommendations from the journal text mining literature (Blei & Lafferty, 2007) and included the full text of all articles, because the data from full-text articles provide a deeper analysis of each article than just the abstracts alone. Furthermore, given that the citation patterns in research literature manifest the knowledge structure of a field (Narin, Carpenter, & Berlt, 1972; Price, 1965), we included the text data regarding the in-text citations and references in our study. All text data in the 1,539 EAQ articles are termed as the EAQ corpus from here onward.

Data Processing

To prepare the EAQ corpus for probabilistic topic modeling, we performed several necessary text data processing operations, including word segmentation (Peña, Bonatti, Nespor, & Mehler, 2002; Yang, 2002), stemming (Hull, 1996; Porter, 1980), and stop-word removal (Baeza-Yates & Ribeiro-Neto, 1999; Manning, Raghavan, & Schütze, 2008). Thereafter, each EAQ article’s text data were represented by a vector of words (also called terms). We then generated the EAQ document–term matrix, in which each row represents an EAQ article and each column represents a unique word in the EAQ corpus. The dimension of the EAQ document–term sparse matrix is 1,539 × 119,303, with each of 1,539 EAQ articles as a row and each of 119,303 words in the EAQ corpus as a column.

Next, following the recommendations for probabilistic topic modeling (Grün & Hornik, 2011; Steyvers & Griffiths, 2007), we prepared the text data for analysis by removing extremely low frequency words as well as extremely high frequency words—such as “and,” “the,” and “for” that contain little topical content by calculating an optimal tf-idf = 0.0022, which is the product of the overall term frequency (tf) by the inverse document frequency (idf). Setting tf-idf slightly lower than the median tf-idf = 0.002295 allows us to exclude from the analysis both high- and low-frequency words in the EAQ corpus. Thus, the EAQ document–term matrix generated from the cleaned corpus for probabilistic topic modeling is 1,539 × 100,766, meaning 1,539 documents and 100,766 words are included for the analysis.

Data Analysis and Model Fitting

The data analysis and model fitting for the probabilistic EAQ topic modeling was performed using the open source statistical computing software R 3.1.3 (R Development Core Team, 2015). The R code for the data analysis and model fitting is given in the online Appendix. As noted earlier, a topic is defined as a distribution over the words used in the corpus (Blei, 2012). In topic modeling, the words are the only observed variables (W); the latent variables, on the other hand, are the topic structure—the per-document topic proportions (θ), the per-document per-word topic assignments (Z), and the topics (β; Blei, 2012). To generate the most likely words with high probability with respect to each topic, first, assuming each document (EAQ articles in this case) exhibits its topics in different proportions, a distribution over topics is randomly chosen; thus, the topic proportions for the dth document are $θ_{d}$ , where $θ_{d, k}$ is the topic proportion for topic k in document d. Then in document d, a topic is randomly assigned for the nth word (W). Therefore, the joint distribution of the hidden variables (θ, Z, and β) and observed variables (W) is

p (β_{1 : K}, θ_{1 : D}, Z_{1 : D}, W_{1 : D}) = \prod_{i = 1}^{K} p (β_{i}) \prod_{d = 1}^{D} p (θ_{d}) \prod_{n = 1}^{N} p (Z_{d, n} | θ_{d}) p (W_{d, n} | β_{1 : K}, Z_{d, n})

Note that the topic assignment $Z_{d, n}$ depends on the per-document topic proportions $θ_{d}$ ; the observed word $w_{d, n}$ depends on the topic assignment $Z_{d, n}$ and all of the topics $β_{1 : K}$ . This process thus generates the high-probability terms that define a topic in the corpus (see Blei, 2012, for a thorough explication of probabilistic topic modeling).

Following the recommendations of the topic modeling literature (Arlot & Celisse, 2010; Blei, Ng, & Jordan, 2003; Grün & Hornik, 2011), we applied a 10-fold cross validation to the data set and examined the topic by perplexity plots, identifying k = 20 as the optimal number of latent topics, as this model had the lowest perplexity out of models from k = 2 to k = 41. However, this type of computation across this large data set is highly computationally intensive, making it infeasible to run models on currently available individual computers. To address this issue, we conducted the data analysis using high-performance computing (HPC; Fikis et al., 2015), through the Batch Linux Computer VELA, which is composed of 10 mutually exclusive servers run in parallel (University Research Services & Administration, 2015). Thus, through leveraging HPC, the data analysis and model fitting in this study could be completed 40 times faster than a single desktop computer. Still, it took over 12 hours to finish the entire data analysis and model fitting using HPC, which is a nontrivial concern for future research in this area.

Results

This study is the first text data mining analysis of the research literature in the field of educational leadership. The purpose of this study is to describe the underlying topics and topic evolution over the past 50 years. Here, we first present our findings from the probabilistic topic models, describing 20 topics identified across the 1965 to 2014 EAQ corpus, and detail the longitudinal change of topics as a means to understand the development of the field since the 1960s. We then turn to a discussion of the findings and the potential applications of text data mining and probabilistic topic modeling in educational leadership research.

Major Topics

The topic model fit the data well. Our topic model identifies many of the topics previously suggested in the past literature (see Table 1), as well as some that have not been explicitly stated in the past. In addition to identifying the total number of latent topics, we also identified the percentages of articles categorized to each topic and were able to plot the topic changes over time. Table 2 presents the high-probability terms from the k = 20 topic model for each of 20 topics in the 1965 to 2014 EAQ corpus. For each topic, we list the 10 high-probability terms that best distinguish the topics from one another. Overall, across the 50-year time period, we found 20 topics emerged from the EAQ corpus: inequities and social justice, epistemology of educational administration, district collective bargaining, female leadership, organizational studies, international context, research reviews and reflections, policymaking and government, faculty of educational leadership and higher education, profession of school leaders, school leadership preparation and development, trust, legal perspective and accountability, education finance, teacher recruitment and retention, qualitative method, teaching and instructional leadership, quantitative method, school effectiveness, and nonspecific. Note that while our topic model identified Topic 20, labeled as the nonspecific topic with the high-probability terms (teachers, leader, principal, pupil, achievement, etc.), no EAQ article was classified as this nonspecific topic because none of EAQ article’s probability for this topic exceeded the rest of 19 topics. We thus, from here onward, focused on the 19 topics, each of which has a certain number (n > 0) of EAQ articles classified to a specific topic.

Table 2.

Topics in the 1965 to 2014 Educational Administration Quarterly Corpus.

	Topic Labels	% of Articles	High-Probability Terms
1	Inequities and social justice	5.7	Inequity, justice, race, disability, bilingual, deficit, oppress, multicultural, cross-cultural, sociocultural
2	Epistemology of educational leadership	12.7	Epistemology, philosophy, scientific, scholarship, paradigm, positivist, Dewey, University Council for Educational Administration, Foucault, administration
3	District collective bargaining	4.8	Bargain, negotiate, union, contract, strike, district, schools, dropout, consolidate, reform
4	Female leadership	5.4	Women, gender, superintendent, career, female, male, feminist, mother, sexism, masculine
5	Organizational studies	6.8	Teacher, innovate, bureaucracy, in-service, turnover, authority, reword, stressor, collaborate, change
6	International context	3.7	Israeli, Asia, Hispanic, Mexico, region, Schechter, migrant, score, ministry, decentralization
7	Research reviews and reflections	10.7	Campbell, dialogue, scholarship, manuscript, reader, community, collaboration, Murphy, Sergiovanni, Michael
8	Policymaking and government	3.9	Federal, legislature, policymaking, governor, Senate, Congress, vote, tax, lobby, coalition
9	Faculty of educational leadership and higher education	4.1	Faculty, rank, department, discipline, supervisor, instructor, online, bureaucracy, deficit, culture
10	Profession of school leaders	2.5	Principal, superintendent, district, teacher, mentor, reform, restructure, collaborate, culture, problem-solving
11	School leadership preparation and development	2.7	Leader, candidate, portfolio, licensure, internship, proficiency, accredit, specialist, redesign, Wallace
12	Trust	2.6	Principal, trust, leadership, teacher, climate, Hallinger, Hoy, Heck, Bryk, Tschannen-Moran
13	Legal perspective and accountability	5.3	Court, supreme, judicial, amendment, statute, regulatory, oversight, charter, punish, board
14	Education finance	4.4	District, cost, finance, expenditure, bond, county, revenue, wealth, equity, fiscal
15	Teacher recruitment and retention	4.6	Teacher, salary, recruit, market, compensate, certificate, retention, wage, vacancy, shortage
16	Qualitative method	5.0	Cognitive, metaphor, symbol, trait, artifact, naturalist, speech, vignette, ethnography, actor
17	Teaching and instructional leadership	4.5	Leadership, teacher, efficacy, learning, school-wide, self-efficacy, Leithwood, coach, collaborate, innovate
18	Quantitative method	6.8	Variable, correlate, hypothesis, interdependent, predictor, beta, Cronbach, alpha, questionnaire, coefficient
19	School effectiveness	3.7	Output, bureaucrat, simulate, variable, efficiency, employee, zone, subsystem, leader, focal
20	Nonspecific	0	Teacher, reform, leader, principal, interpersonal, achievement, pupil, subgroup, score, collaboration

Together, a quarter (23.4%) of all articles published in EAQ are classified as either the epistemology of educational leadership topic (Topic 2, 12.7%) with high-probability terms such as “epistemology,” “philosophy,” “scientific,” “scholarship,” “paradigm,” and “positivist” or the research reviews and reflections topic (Topic 7, 10.7%) with high-probability terms such as “Campbell” (the first editor of EAQ), “dialogue,” “scholarship,” “manuscript,” and “reader.” To provide a close look at the topics, we present example articles with high probability in the EAQ corpus in the online Table 3. For instance, regarding the topic of epistemology of the field (Topic 2), we found the articles in EAQ’s 1983 special issue on the subject of “Educational Administration 1959-1981: A Profession in Evolution” (e.g., Culbertson, 1983; Fogarty, 1983). Regarding the topic of research reviews and reflections, we found the articles in which EAQ’s past, present, and future were reflected (see Pounder & Johnson, 2007); EAQ’s influence on the field was assessed (see Haas et al., 2007); and the research in educational leadership was empirically reviewed (see Hallinger, 2013).

Research methods topics are also a highly represented set of topics across the documents. Specifically, the topics that are related to research methods make up 11.8% of the articles in the entire EAQ corpus: quantitative method (Topic 18, 6.8%) with high-probability terms such as “variable,” “correlate,” “hypothesis,” “interdependent,” and “predictor,” and qualitative method (Topic 16, 5.0%) with high-probability terms such as “cognitive,” “metaphor,” “symbol,” “trait,” and “artifact.” In the articles classified as the topic of quantitative method (Topic 18), for instance, the methodological concerns in the study of school principals’ job satisfaction were discussed (see Friesen, Holdaway, & Rice, 1983); the scale of group-level organizational citizenship behavior was reconstructed and validated (see Vigoda-Gadot, Beeri, Birman-Shemesh, & Somech, 2007); the structural equation modeling was used to examine the influence of school leaders on student achievement (see Bruggencate, Luyten, Scheerens, & Sleegers, 2012). In the articles classified as the topic of qualitative method, for instance, the observational studies were critiqued (see Gronn, 1982) and defended (see Thomas, 1986); a method for sampling in case study research on administrative behavior was presented (see McClintock, 1985); and metaphor and reflective coaching were used to explore the principals’ thinking process (see Dana & Pitts, 1993). These findings suggest that EAQ, as the most prestigious research journal in the field, has been continuously scanning the landscape of the field and has been devoted to publishing methodologically rigorous research as well as discussing and confronting a diversity of methodological perspectives.

In addition, the topics of social justice (Topic 1, 5.7%) and female leadership (Topic 4, 5.4%) have attracted much scholarly interest. In the EAQ corpus, a total of 11.1% of articles addressed these two topics. The social justice topic is described by the high-probability terms such as “inequity,” “justice,” “race,” “disability,” and “bilingual,” and the female leadership topic is described by the high-probability terms such as “women,” “gender,” “superintendent,” “career,” and “female.” As the example articles in the online Table 3 show, social justice has been examined from a critical ecological perspective (see Furman & Gruenewald, 2004) and transformative leadership framework (see Cooper, 2009); female leadership was probed from the lens of androcentric bias (see Epps, Sackney, & Kustaski, 1994) and resilience (see Christman & McClellan, 2008).

Evolution of Topics

Moving beyond a snapshot view, we probed into the entire EAQ corpus from Volume 1 in 1965 to the recent Volume 50 in 2014. To provide a panoramic view of the rise and fall of the topics in the corpus over time, we sorted the number of articles in each topic per year from 1965 to 2014, and illustrated the topic evolution in Figure 2 in which the x-axis represents the year and y-axis the number of articles addressing the corresponding topic in a given year. As EAQ has transitioned over the past 50 years, the number of issues per year in the journal has shifted over time, to interpret the results of the topic evolutions illustrated in Figure 2, we need to take into account the change in the number of issue in EAQ per volume. That is, EAQ published only three issues per yearly volume from 1965 to 1980, four issues per yearly volume from 1981 to 1999, and five issues per yearly volume from 2000 to the present. With this caveat in mind, Figure 2 depicts the trend of how each of the 19 topics evolved over the past five decades.

Figure 2.

Evolution of topics in the EAQ corpus from 1965 to 2014.

Across these 19 topics, 5 topics have gained attention from researchers over time. These topics include inequities and social justice (Topic 1), female leadership (Topic 4), school leadership preparation and development (Topic 11), trust (Topic 12), and teaching and instructional leadership (Topic 17). An increasing number of articles attended to these topics over 50 years, defining the emerging forefront in the terrain of the educational leadership field. Of particular note, for many of these topics that rose over time, there were special issues of EAQ devoted to these topics, such as the special issues on school leadership preparation in 2011 (Volume 47, Issue 1), social justice in 2004 (Volume 40, Issue 1), as well as the concentration of articles on the topics regarding trust in 2009 (Volume 45, Issue 2) and instructional leadership in 2012 (Volume 48, Issue 2). As a means of promoting attention to special topics that had not received adequate attention in the past, this finding shows that publishing special issues appears to be a useful strategy by journal editors to grow the diversity of topics in a field and bring attention to underattended topics.

In comparison, the topic of epistemology of educational leadership (Topic 2) attracted high attention in the research community in the first four decades of EAQ, but then, the attention began to decline after 2005. In the early era of EAQ, our findings suggest that this line of inquiry appeared to be of vital importance to define the epistemological boundaries of educational leadership as a field, and thus to establish and retain a foothold among other fields in academia. Then, over the past decade, the epistemology topic has attracted less attention in the educational leadership research community as represented by the small number of articles in EAQ. This finding suggests multiple, and perhaps competing hypotheses: Was the recent decline because epistemology has become a well-established, settled topic in the field of educational leadership after 40 years’ inquiry and development? There is emerging literature on the temporal nature of the sociology of research findings, which has found that over time in a research domain topics can become “settled” and the field moves on (Shwed & Bearman, 2010). Alternatively, a competing hypothesis may be that the change is due to the research community becoming distracted by other intellectual inquiries. A third explanation could be due to the influence of different journal editors and editorial boards over time, and a shift in purpose and attention for the journal as different editors and institutions take the helm.

The remainder of the topics waxed and waned over the five decades between 1965 and 2014, as evidenced by large variations in the number of articles without a consistent pattern, as displayed in Figure 2. The legal perspective and accountability topic (Topic 13) provides a good example of this pattern. In the 1960s, the articles on this topic mostly discussed an array of issues, namely, the neutrality principle in the U.S. Supreme Court cases dealing with religion and education (see Katz, 1965), church–state relations (see Reutter, 1965), the legal definition of disruptive student behavior (see Ladd, 1971), the significance of the Fourteenth Amendment on public school educators (see La Morte, 1974), school desegregation/integration litigation (see Brown, 1979), and teachers’ religious freedom (see Beezer, 1982). The topic of legal perspective and accountability then tapered off in the 1980s and the 1990s, and then rose again in 2003 with a special issue on law and education. This again highlights the powerful role of special issues of the journal. The detection of special issues also suggests the robustness and validity of our topic models.

Discussion

The purpose of this study was to describe the underlying topics and the topic evolution over the 50-year history of the educational leadership field in EAQ. Applying automated text data mining, we provide the first evidence to date demonstrating the total number of topics in the entire 1965 to 2014 EAQ corpus, what each topic represents, the distribution of articles across the topics, and how these topics have changed and evolved over the 50 years since the first issue of EAQ. We turn next to discuss how the findings of this study map the temporal terrain of topics in the field of educational leadership, shed light on the future of the field, and the potential of applying text data mining and topic modeling to educational leadership research.

Temporal Terrain of Topics of the Field

Tracking the historical development of the field, our findings reveal the unfolding landscape of 19 topics in the educational leadership field. The breadth of topics, along with the rise and fall of the topics, have charted the intellectual progress in educational leadership research since the 1960s. The 19 topics’ interdisciplinary boundaries are, to a large extent, congruent with the fields that closely align with educational leadership (Haller, 1968; Wang & Bowers, 2016). For instance, teacher recruitment and retention (Topic 15) and teaching and instructional leadership (Topic 17) are related to the fields of teacher education and human resources; education finance (Topic 14) is related to economics; trust (Topic 12) is related to psychology.

How did the topics in EAQ evolve over time? In comparison with the topics identified in prior studies summarized in Table 1, the 19 topics identified in our study provide us with a more panoramic view of the temporal terrain of the field. This is partly because the literature examined in our study stretches 50 years spanning from 1965 to 2014, and partly because the probabilistic topic modeling used in our study is not subject to the cognitive and labor constraints imposed by reviewing over 1,500 articles. The 19 topics identified in this study surpass the number of topics in the prior literature (see Table 1). Some topics—such as decision making, administrative behavior, authority/bureaucracy, and power (Bridges, 1982; Campbell, 1979)—did not gain sustained attention from the research community. Yet all the topics identified by Hoy (1994) and Murphy et al. (2007) were present in our findings. In addition to the topics that have already been identified in the prior literature, we found some topics that were not noted explicitly in prior literature, such as inequities and social justice (Topic 1), epistemology of educational leadership (Topic 2), international context (Topic 6), research reviews and reflections (Topic 7), trust (Topic 12), qualitative method (Topic 16), and quantitative method (Topic 18).

These 19 topics across the 1965 to 2014 EAQ corpus provide a uniquely valuable backdrop to examine the current status of educational leadership, to reflect on the development of the field over the past 50 years, and more important, to contemplate advances and next steps. Our findings on the topics and topic evolution not only validate the statement made by Willower (1981) three decades ago that “the intellectual scene in educational administration is characterized by diversity and change” (p. 115) but also suggest that the trend of topic diversification has remained strong since the recent comprehensive review by Oplatka (2009). The continued diversification of the topics is likely because education is inherently political (Labaree, 2011). That is, the scholarly inquiry in educational leadership has been “more affected by changes in politics and societal values (e.g., efficiency, equity, accountability)” (Heck, 2015, p. 58). For instance, instead of a sustained research focus on “a set of well-defined disciplinary problems” (Heck, 2015, p. 58), the external policies that governed the educational leadership field include the National Defense Education Act of 1958, the Elementary and Secondary Education Act of 1965, the Education for All Handicapped Children Act of 1975, which was later replaced by the Individuals with Disabilities Education Act of 1990, A Nation at Risk of 1983, the No Child Left Behind Act of 2001, the Race to the Top of 2011, and the Every Student Succeeds Act of 2015, among many others. In such a policy and political environment where education problems and solutions are constantly changing (Berliner & Biddle, 1995), scholars have faced challenges in producing sustained empirical inquiry into educational leadership.

An intriguing, if not concerning, finding is the conspicuously diminishing research interest in the epistemology of the field (Topic 2) in recent years. Specifically, between the 1960s and the 2000s, the topic of epistemology remained relatively active. It might be explained that in EAQ’s 1965 to 1978 “formative years” and 1979 to 2003 “adolescent years” (Murphy et al., 2007, p. 626), the scholars had been searching for and defining the epistemological identity of educational leadership research. However, these epistemological efforts did not persist into the present after 2000. Instead, the educational leadership research community showed a heightened awareness of inequities and social justice (Topic 1), drawing attention to historically disadvantaged community and schooling issues. Moreover, a changing social and international context in the United States and globally (Heck & Hallinger, 2005; Leithwood & Hallinger, 2002; Leithwood, Chapman, Corson, Hallinger, & Hart, 1996) was likely a driver for the growing interest in several topics, namely, school leadership preparation and development (Topic 11), trust (Topic 12), teaching and instructional leadership (Topic 17), and international context (Topic 6). These topics are indeed valuable to advance the field; however, we posit the decent of the epistemological topic merits attention from the educational leadership research community. For a porous field of education leadership (Wang & Bowers, 2016), it is particularly important for the field to stay focused by a continual examination of the fields’ nature, purpose, knowledge base, topical contours, and interdisciplinary boundaries.

How do the topics identified in this study demonstrate EAQ’s mission? As stated on its website, EAQ “presents prominent empirical and conceptual articles focused on timely and critical leadership and policy issues facing educational organizations” (Sage, n.d., para 1). EAQ’s mission, to a large extent, is manifested by the scope of the topics in Figure 2, and is attested by the rise of the topics in the evolving social and political context, including inequities and social justice (Topic 1), international context (Topic 6), legal perspective and accountability (Topic 13), and school effectiveness (Topic 19). Considering the intertwined relationship between the field and the journals—“the journal defines the field, the field defines the journal” (Fitz, 1999, p. 316), EAQ, as the most prestigious journal in a field that has been persistently criticized for lacking a focused research agenda (Bridges, 1982; Erickson, 1979; Hoy, 1982; Oplatka, 2009; Rowan, 1995), could play a pivotal role to shape a focused, robust agenda for the field. The topical landscape identified in this study presents ample opportunities for researchers to establish individual research agendas, and more important, for EAQ and perhaps other related journals to shape a collective, focused research agenda for the field. A common perception is the tendency for research to be siloed into a given topic, creating an echo chamber as researchers are left talking to themselves, which then may hinder synergy among the topics. We posit here that a focused research agenda for the field of educational leadership may perhaps be better shaped by encouraging synergy and collaboration across the 19 topics identified in this study rather than the promotion of a few siloed topics.

Educational Leadership Research: Where to Go From Here?

Our findings pose important questions for the future research not only in EAQ but also in educational leadership: Where should the field of educational leadership go from here? What is/are the field’s vision(s) for the future? And what are the “settled” topics, versus what are the emerging, disputed, or conflicting topics? As noted by past authors in educational leadership, advances in any field need a concrete cumulative knowledge base to attend to the field’s future direction; otherwise, we run the risk of allowing the field to decline and demise (Heck, 2015; Heck & Hallinger, 2005; Tschannen-Moran, Firestone, Hoy, & Moore-Johnson, 2000; Oplatka, 2009, 2010). The continued and strengthened topic diversification trend may warrant that the educational leadership research community focuses on sustained research efforts to resolve a set of well-defined problems in the field, as proposed by Heck (2015). By bringing a tighter focus while marshalling resources to tackle the main unsolved questions that the educational leadership field has historically wrestled with, scholarship in the field might be able to depart from “intellectual random events” (Bridges, 1982, p. 22) or continually debating reoccurring issues. As noted by Oplatka (2009), “the field is typically embedded with debates over similar ideas, assumptions, and insights about educational administration as a field of study throughout the last five decades” (p. 26). As a result, a sustained focus on a set of research questions in the field would be able to create cumulative knowledge that practitioners can directly use to improve school leadership (Heck, 2015). Indeed, some scholars have already taken on sustained efforts to unpack the relationship between school leadership and student achievement: Hallinger and Heck (1996) examined the theoretical and empirical models; Witziers, Bosker, and Kruger (2003) conducted a meta-analysis; Leithwood and Mascall (2008) examined the relationship from a distributed leadership perspective; Dumay (2009) tested the relationship between school leadership, school culture homogeneity, and student achievement; Bruggencate et al. (2012) found an indirect school leadership–student achievement relationship, which is mediated by development-oriented school organizational and classroom practices. As we can see in this line of research, scholars’ sustained research focus on the same problem helped advance our understanding of the relationship between school leadership and student achievement.

Furthermore, sustained, focused research efforts will only speed up knowledge accumulation in conjunction with being open to diverse ideas. As we take a sanguine view on the continued and strengthened topic diversification in educational leadership research, we found that the current state of the art in the educational leadership field is far from what Bridges (1982) described, “the state-of-the-art is scarcely different from what seemed to be in place nearly 15 years ago” (p. 24). Instead, the topic evolution illustrated in Figure 2 fits precisely the description by Boyan (1981) that the current state of the art in the field is “several freeways which run through the territory, with only a few crossover and intersections available” (p. 8). Educational leadership, as a porous, outward-facing field (Wang & Bowers, 2016), embraces diverse philosophical paradigms, pluralistic theoretical underpinnings, and emerging methodological approaches. It is thus pivotal for the field to strike a balance between being open to new ideas and the rigorous scrutiny of all ideas (Makel & Plucker, 2014; Sagan, 1997).

Yet how do we know our field’s topic diversification is not too diverse for an outward-facing field as porous as educational leadership (Oplatka, 2010; Wang & Bowers, 2016)? Said another way, a critique of the present study is to ask how this analysis and the number of topics compare with other research fields and journals in which text data mining has been used. Are 19 topics a lot, a little, or normative for the field of educational leadership? While of interest, however, we envision that these questions can be answered in the near future as the field of journal text data mining is just beginning to mature, and the present study provides the first evidence in education leadership using this type of technique. We also posit that it would be impractical to set an arbitrary cutoff point of the number of topics based on other fields’ topical landscapes, as each research field has a unique epistemological identity, especially across social sciences. However, in comparison with the typology of 7, 10, and 12 topics postulated in the past research from the manually coded typologies of topics in EAQ from Table 1, our finding of 19 topics appears reasonable, as it is not less than these previous studies and neither is it more than double, demonstrating the strong value of the triangulation, hard work of the previous authors in this domain, and the validity of our findings. Additionally, perhaps a more important question lies not with topic diversification or overdiversification, but whether the topics address the core of the applied field of educational leadership—the leadership that promotes teaching and learning, whether the topics bridge the disconnect between leadership research and practice, and whether the topics accomplish EAQ’s goal to foster the dialogue among scholars and practitioners (Sage, n.d.) and the UCEA’s (n.d., para 1) mission of “advancing the preparation and practice of educational leaders for the benefit of schools and children.”

An additional critique of the findings in this study is that they might have overly focused on the concerns of researchers rather than those of practitioners in educational leadership. Given the inherent nature of educational leadership as an applied field (Campbell, 1981; Glatter, 1987; Riffel, 1986; Rowan, 1995), it is of great importance to engage leadership practitioners, who are knowledge consumers, to evaluate the progress of knowledge accumulation in educational leadership. We acknowledge that as a research journal, the findings from the present study of EAQ are necessarily focused on issues of concern to the research community, and we see this method as providing an exciting opportunity to explore the topics in other journals, including practitioner oriented journals, such as Educational Leadership, published by the Association for Supervision and Curriculum Development, to examine the intersection of topics over time between research and practice.

Text Data Mining and Topic Modeling in Educational Leadership Research

Using an innovative methodological approach of probabilistic topic modeling, we mapped the terrain of topics in EAQ. To our knowledge, this is the first work that captures topics, along with topic evolution over time in the entire 1965 to 2014 EAQ corpus spanning 50 years. To overcome the limit of labor-intensive manual coding in traditional analyses, we mobilized a repertoire of analytical and technological resources in the fast-growing fields of text data mining and machine learning. Without manual coding or prior annotation, we identified the latent topics in the EAQ corpus based on high-probability terms and the probability of each EAQ article to the 19 topics. These analytic techniques of topic modeling provide an alternative approach to analyze text data in educational leadership research. Since the topic modeling algorithms are highly scalable, topic modeling has been increasingly used to analyze massive amounts of text data in the social sciences. Thus, there are tantalizing possibilities of applying topic modeling to educational leadership research. In addition to identifying the latent topics in the ever-growing volumes of scholarly literature, topic modeling is a viable analytic tool to help extract latent topics in text data in real time. This is of particular value to extract real-time information from the vast trove of digital text data such as blogs, microblogs (also called Tweets), and online comments, which are generated by millions of Internet users (Wang & Fikis, 2016). By doing so, topic modeling holds great potential to bolster the research potential in the field of educational leadership research, allowing us to detect topics that would otherwise be difficult to identify in the raw data. In our increasingly digitized educational environment, applying topic modeling in educational leadership and policy research is certainly a growth area to watch for in the future.

Limitations and Suggestions for Future Inquiry

One limitation of this study is that we only examined a single journal’s entire corpus of articles. We acknowledge that the focus on a single journal limits the scope of the results. As the first application of automated text mining to identify topics in an educational leadership journal, this study lays the foundation for text mining of a larger set of journals. We therefore recommend future research to apply topic modeling to explore the topical landscape across multiple journals. Furthermore, we also acknowledge the practical limitations of text mining, which are nontrivial, as researchers must first gain copyright approval from each journal and editor, and then download and convert the entire full text of all articles from a journal. Also, as noted in the Method section, we relied on a HPC solution (also known as a “super computer”) to analyze the data, because current desktop computers would have taken multiple days or weeks to run the model. Even so, our analysis of just a single journal took 12 hours to complete on an HPC. Considering very few examples in the research literature of collaborations between educational leadership researchers and HPC centers, we provide the R code in the online Appendix to help provide a strong starting point for future researchers who wish to take on such a highly computationally intensive modeling tasks.

Additionally, as Hallinger (2013) noted in his proposed conceptual framework for systematic reviews of research in educational leadership, whether using a single source type or mixed sources in review studies “depends largely on the density and quality of relevant literature identified in the domain” (p. 133). For instance, using a single data source was well-justified in Hallinger’s (2011) review of doctoral studies using the Principal Instructional Management Rating Scale, as well as Leithwood and Sun’s (2012) review of unpublished research on transformational school leadership. In our study, we consider that all the EAQ articles published over the past 50 years provide the sufficient density of educational leadership research literature. Furthermore, regarding the quality of relevant literature, EAQ has been consistently considered as the most prestigious research journal in our field of educational leadership (Campbell, 1979; Cherkowski et al., 2011; Haas et al., 2007; Murphy et al., 2007; Richardson & McLeod, 2009; Wang & Bowers, 2016), in which prior literature justifies the “most prestigious” moniker through journal citation analyses, impact factors, as well as surveys and interviews of researchers in the field. Nevertheless, we acknowledge that the literature published in EAQ does not demonstrate an exhaustive knowledge base of educational leadership research. For instance, recent evidence suggests that despite EAQ being consistently considered as the most prestigious research journal in educational leadership, others leading journals—such as the Journal of Educational Administration, Educational Management Administration & Leadership, and the Journal of School Leadership—are more interdisciplinary than EAQ (Wang & Bowers, 2016). More important, the majority (54.71%) of citations in educational leadership research literature rely on nonjournal sources such as books and reports (Wang & Bowers, 2016). We therefore encourage future researchers to apply topic modeling to the text data from extensive and diverse literature sources across the field of educational leadership.

Another limitation of our study is that, as with other latent variable methods, such as factor analysis and latent class analysis, the topic-labeling process is subject to the authors’ interpretation of the results. The results of topic modeling provide high-probability terms of each topic, along with the probability of each article to the topics in the corpus, but it takes the authors’ interpretation to label the thematic topics from these results. Data in the social sciences are socially constructed; thus, the data lose their meaning and value when they are taken out of context (boyd & Crawford, 2012). Despite the automated data mining process, researchers must carefully unpack the meaning behind the data in a given context. Therefore, like all social science studies, the subjectivity in the sensemaking of the topics in this study needs to be taken into consideration as the readers interpret the findings.

The third limitation is that the findings of this study provide a coarse-grained projection of the field’s topic landscape. One of the assumptions of topic modeling is that each document is assumed to be described as multiple topic probabilities, and the document is assigned to a given topic according to a posterior probability (Blei, 2011; Chang Gerrish, Wang, Boyd-Graber, & Blei, 2009). In this study, we followed the recommendations of the research methods in text mining, such that the topic with the highest posterior probability was assigned to the document. For instance, if an EAQ article has 18% of chance to be assigned to the trust topic, and has only a 4% chance to be assigned to the rest of topics, our model assigned the article to the trust topic. Across the 19 topics, articles have about a 5% chance of being assigned to any random topic if all topics were equally probable, which they were not as evidenced by the result that none of EAQ articles was classified as the nonspecific topic (Topic 20). Indeed, the field of automated text mining is a rapidly developing methodological subfield within the broader text data mining literature (Blei, 2012; Lee, et al., 2010; Steyvers & Griffiths, 2007). As the techniques and models in automated text mining continue to advance, we look forward to future research in which documents may be assigned across multiple topics and discussed, as well as the hierarchical structure of the topics are explored.

And finally, while the techniques in topic modeling—as an exploratory tool to infer topics from large corpora (Blei, 2011; Blei & Lafferty, 2007; Chang et al., 2009)—are innovative and sufficiently useful to extract the topics that are otherwise hard to uncover, the probabilistic approach clearly loses rich information contained in text data from which the traditional qualitative methods can enable researchers to generate a fine-grained depiction. Hence, we encourage future inquiry to continue to examine the rich social, philosophical, economic, and political context in which the field of educational leadership is embedded.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

The Appendix and Table 3 are available in the online version of the article.

Author Biographies

Yinying Wang is an assistant professor of educational leadership in the Department of Educational Policy Studies at College of Education and Human Development, Georgia State University. Her research interests include social network analysis, social media in education policy making and organizational communication, policy informatics in education, and digital inequities in education.

Alex J. Bowers is an associate professor of education leadership at Teachers College, Columbia University. His research interests include organizational-level data analytics, organizational behavior, school and district leadership, data-driven decision making, high school dropouts and completion, educational assessment and accountability, education technology, and school facilities financing. ORCID: 0000-0002-5140-6428.

David J. Fikis is a doctoral candidate of research, measurement, and statistics in the Department of Educational Policy Studies at College of Education and Human Development, Georgia State University. His research focuses on score equating, psychometrics, item response theory, high-performance computing, and pedagogical advancements in quantitative methods instruction.

References

Arlot

Celisse

(2010). A survey of cross-validation procedures for model selection. Statistical Surveys, 4, 40-79.

Baeza-Yates

Ribeiro-Neto

(1999). Modern information retrieval. New York, NY: Addison-Wesley.

Bates

(1980). Educational administration, the sociology of science, and the management of knowledge. Educational Administration Quarterly, 16(2), 1-20.

Beezer

(1982). Religion and employment: How extensive is a teacher’s religions freedom? Educational Administration Quarterly, 18(2), 95-113.

Berliner

D. C.

Biddle

B. J.

(1995). The manufactured crisis: Myths, fraud, and the attack on American schools. New York, NY: Addison-Wesley.

Blei

D. M.

(2011). Introduction to probabilistic topic models. Retrieved from https://www.cs.princeton.edu/~blei/papers/Blei2011.pdf

Blei

D. M.

(2012). Probabilistic topic models. Communication of the ACM, 55(4), 77-84.

Blei

D. M.

Lafferty

J. D.

(2007). A correlated topic model of science. Annals of Applied Statistics, 1(1), 17-35.

Blei

D. M.

A. Y.

Jordan

M. I.

(2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993-1022.

10.

Bowers

A. J.

Chen

(2015). Ask and ye shall receive? Automated text mining of Michigan capital facility finance bond election proposals to identify which topics are associated with bond passage and voter. Journal of Education Finance, 41(2), 164-196.

11.

Boyan

N. J.

(1981). Follow the leader: Commentary on research in educational administration. Educational Researcher, 10(2), 6-21.

12.

boyd

Crawford

(2012). Critical questions for Big Data: Provocations for a cultural, technological, and scholarly phenomenon. Information, Communication & Society, 15(5), 662-679.

13.

Bridges

(1982). Research on the school administrator: The state-of-the-art, 1967-1980. Educational Administration Quarterly, 18(3), 12-33.

14.

Briggs

A. R. J.

Coleman

Morrison

(Eds.). (2011). Research methods in educational leadership and management (3rd ed.). London, England: Sage.

15.

Brown

(1979). Major changes in school integration litigation, 1954-1979. Educational Administration Quarterly, 15(2), 76-97.

16.

Bruggencate

Luyten

Scheerens

Sleegers

(2012). Modeling the influence of school leaders on student achievement: How can school leaders make a difference? Educational Administration Quarterly, 48(4), 699-732.

17.

Campbell

R. F.

(1979). Fifteenth anniversary perspective: A critique of the Educational Administration Quarterly. Educational Administration Quarterly, 15(3), 1-19.

18.

Campbell

R. F.

(1981). The professorship in educational administration: A personal view. Educational Administration Quarterly, 17(1), 1-24.

19.

Chang

Gerrish

Wang

Boyd-Graber

J. L.

Blei

D. M.

(2009). Reading tea leaves: How humans interpret topic models. In Proceedings from the conference on advances in neural information processing systems (pp. 288-296). Retrieved from https://papers.nips.cc/paper/3700-reading-tea-leaves-how-humans-interpret-topic-models

20.

Cherkowski

Currie

Hilton

(2011). Who should rank our journals . . . and based on what? Journal of Educational Administration, 50(2), 206-230.

21.

Christman

McClellan

(2008). “Living on barbed wire”: Resilient women administrators in educational leadership programs. Educational Administration Quarterly, 44(1), 3-29.

22.

Cooper

C. W.

(2009). Performing cultural work in demographically changing schools: Implications for expanding transformative leadership frameworks. Educational Administration Quarterly, 45(5), 694-724.

23.

Culbertson

J. A.

(1983). Leadership horizon in education. Educational Administration Quarterly, 19(3), 273-296.

24.

Dana

N. F.

Pitts

J. H.

Jr. (1993). The use of metaphor and reflective coaching in the exploration of principal thinking: A case study of principal change. Educational Administration Quarterly, 29(3), 328-338.

25.

Donmoyer

(2001). Evers and Lakomoski’s search for leadership’s Holy Grail (and the intriguing ideas they encountered along the along). Journal of Educational Administration, 39(6), 554-572.

26.

Dumay

(2009). Origins and consequences of schools’ organizational culture for student achievement. Educational Administration Quarterly, 45(4), 523-555.

27.

Epps

J. R.

Sackney

L. E.

Kustaski

J. M.

(1994). Reassessing levels of androcentric bias in Educational Administration Quarterly. Educational Administration Quarterly, 30(4), 451-471.

28.

Erickson

D. A.

(1979). Research on educational administration: The state of the art. Educational Researcher, 8(3), 9-14.

29.

Evangelopoulos

Zhang

Prybutok

V. R.

(2012). Latent semantic analysis: Five methodological recommendations. European Journal of Information Systems, 21, 70-86.

30.

Evers

C. W.

Lakomski

(2012). Science, systems, and theoretical alternatives in educational administration: The road less travelled. Journal of Educational Administration, 50(1), 57-75.

31.

Fikis

Wang

Bowers

(2015, September). An application of multithread data mining in educational leadership research. Poster presented at the 2015 Scientific Computing Day Conference at Georgia State University, Atlanta, GA. doi:10.6084/m9.figshare.1558319

32.

Fitz

(1999). Reflections on the field of educational management studies. Educational Management & Administration, 27(3), 313-321.

33.

Fogarty

B. M.

(1983). Educational administration 1959-1981: A profession in evolution. Educational Administration Quarterly, 19(3), 141-152.

34.

Friesen

Holdaway

E. A.

Rice

A. W.

(1983). Satisfaction of school principals with their work. Educational Administration Quarterly, 19(4), 35-58.

35.

Furman

G. C.

Gruenewald

D. A.

(2004). Expanding the landscape of social justice: A critical ecological analysis. Educational Administration Quarterly, 40(1), 47-76.

36.

Glatter

(1987). Towards an agenda for education management. Educational Management & Administration, 15(1), 5-12.

37.

Gronn

P. C.

(1982). Neo-Taylorism in educational administration? Educational Administration Quarterly, 18(4), 17-35.

38.

Grün

Hornik

(2011). Topicmodels: An R package for fitting topic models. Journal of Statistical Software, 40(13), 1-30.

39.

Gunter

Ribbins

(2003). The field of educational leadership: Studying maps and mapping studies. British Journal of Educational Studies, 51(3), 254-281.

40.

Haas

Wilson

G. Y.

Cobb

C. D.

Hyle

A. E.

Jordan

Keamey

(2007). Assessing influence on the field: An analysis of citations to Educational Administration Quarterly, 1979-2003. Educational Administration Quarterly, 43(4), 494-512.

41.

Hall

Jurafsky

Manning

C. D.

(2008). Studying the history of ideas using topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 363-371). Stroudsburg, PA: Association for Computational Linguistics.

42.

Haller

E. J.

(1968). The interdisciplinary ideology in educational administration: Some preliminary notes on the sociology of knowledge. Educational Administration Quarterly, 4(2), 61-77.

43.

Haller

E. J.

Knapp

T. R.

(1985). Problems and methodology in educational administration. Educational Administration Quarterly, 21(3), 157-168.

44.

Hallinger

(2011). A review of three decades of doctoral studies using the principal instructional management rating scale: A lens on methodological progress in educational leadership. Educational Administration Quarterly, 47(2), 271-306.

45.

Hallinger

(2013). A conceptual framework for systematic reviewers of research in educational leadership and management. Journal of Educational Administration, 51(2), 126-149.

46.

Hallinger

(2014). Reviewing reviews of research in educational leadership: An empirical assessment. Educational Administration Quarterly, 50(4), 539-576.

47.

Hallinger

Chen

(2015). Review of research on educational leadership and management in Asia: A comparative analysis of research topics and methods, 1995-2012. Educational Management Administration & Leadership, 43(1), 5-27.

48.

Hallinger

Heck

R. H.

(1996). Reassessing the principal’s role in school effectives: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32(1), 5-44.

49.

Heck

R. H.

(2015). Organizational theory and the study of educational leadership and school improvement: Some reflections. Journal of Organizational Theory in Education, 1(1), 58-67.

50.

Heck

R. H.

Hallinger

(2005). The study of educational leadership and management: Where does the field stand today? Educational Management Administration & Leadership, 33(2), 229-244.

51.

Hofmann

(1999). Probabilistic latent semantic indexing. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Berkeley, CA: ACM.

52.

Hoy

W. K.

(1978). Scientific research in educational administration. Educational Administration Quarterly, 14(3), 1-12.

53.

Hoy

W. K.

(1982). Recent development in theory and research in educational administration. Educational Administration Quarterly, 18(3), 1-11.

54.

Hoy

W. K.

(1994). Foundations of educational administration: Traditional and emerging perspectives. Educational Administration Quarterly, 30(2), 178-198.

55.

Hull

D. A.

(1996). Stemming algorithms: A case study for detailed evaluation. Journal of the American Society for Information Science, 47(1), 70-84.

56.

Junqué de Fortuny

Martens

Provost

(2013). Predictive modeling with Big Data: Is bigger really better? Big Data, 1(4), 215-226.

57.

Katz

W. G.

(1965). Religion, education, and the Constitution: The doctrine of neutrality. Educational Administration Quarterly, 1(2), 1-11.

58.

Kitchin

(2014). Big Data, new epistemologies and paradigm shifts. Big Data & Society. doi:10.1177/2053951714528481

59.

Labaree

D. F.

(2011). Consuming the public school. Educational Theory, 61(4), 381-394.

60.

Ladd

E. T.

(1971). Toward an educationally appropriate legal definition of disruptive student behavior. Educational Administration Quarterly, 7(3), 1-25.

61.

La Morte

M. W.

(1974). The Fourteenth Amendment: Its significance for public school educators. Educational Administration Quarterly, 10(3), 1-19.

62.

Lee

Song

Kim

(2010). An empirical comparison of four text mining methods. Journal of Computer Information Systems, 51(1), 1-10.

63.

Leithwood

Chapman

Corson

Hallinger

Hart

(Eds.). (1996). International handbook of educational leadership and administration. Dordrecht, Netherlands: Kluwer Academic.

64.

Leithwood

Hallinger

(Eds.). (2002). Second international handbook of educational leadership and administration. London, England: Kluwer Academic.

65.

Leithwood

Mascall

(2008). Collective leadership effects on student achievement. Educational Administration Quarterly, 44(4), 529-561.

66.

Leithwood

Sun

J. P.

(2012). The nature and effects of transformational school leadership: A meta-analytic review of unpublished research. Educational Administration Quarterly, 48(3), 387-423.

67.

Levine

(2005). Educating schools leaders. New York, NY: Education Schools Project.

68.

Makel

M. C.

Plucker

J. A.

(2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304-316.

69.

Manning

C. D.

Raghavan

Schütze

(2008). Introduction to information retrieval. Cambridge, England: Cambridge University Press.

70.

McCarthy

M. M.

(1986). Research in educational administration: Promising signs for the future. Educational Administration Quarterly, 22(3), 3-20.

71.

McClintock

(1985). Process sampling: A method for case study research on administrative behavior. Educational Administration Quarterly, 21(3), 205-222.

72.

Murphy

Vriesenga

Storey

(2007). Educational Administration Quarterly, 1979-2003: An analysis of types of works, methods of investigation, and influences. Educational Administration Quarterly, 43(5), 612-628.

73.

Narin

Carpenter

Berlt

N. C.

(1972). Interrelationships of scientific journals. Journal of the American Society for Information Science, 23(5), 323-331.

74.

Oplatka

(2009). The field of educational administration: A historical overview of scholarly attempts to recognize epistemological identities, meanings and boundaries from 1960s onwards. Journal of Educational Administration, 47(1), 8-35.

75.

Oplatka

(2010). The legacy of educational administration: A historical analysis of an academic field. Berlin, Germany: Peter Lang.

76.

Oplatka

(2012). Fifty years of publication: Pondering the legacies of the Journal of Educational Administration. Journal of Educational Administration, 50(1), 34-56.

77.

Oplatka

(2014). Differentiating the scholarly identity of educational administration: An epistemological comparison of two neighbouring fields of study. Journal of Educational Administration, 52(1), 116-136.

78.

Peña

Bonatti

L. L.

Nespor

Mehler

(2002). Signal-driven computations in speech processing. Science, 298(5593), 604-607.

79.

Perriault

(2011). CasperJS [Program code]. Retrieved from GitHub website: https://github.com/n1k0/casperjs

80.

Ponweiser

(2012). Latent Dirichlet Allocation in R. Vienna, Austria: Institute for Statistics and Mathematics, Vienna University of Business and Economics.

81.

Porter

M. F.

(1980). An algorithm for suffix stripping. Program, 14(3), 130-137.

82.

Pounder

D. G.

Johnson

B. L.

Jr. (2007). Reflections on EAQ’s past, present, and future. Educational Administration Quarterly, 43(2), 259-272.

83.

Price

D. J. D.

(1965). Networks of scientific papers. Science, 149(3683), 510-515.

84.

Quinn

K. M.

Monroe

B. L.

Colaresi

Crespin

M. H.

Radev

D. R.

(2010). How to analyze political attention with minimal assumptions and costs. American Journal of Political Science, 54(1), 209-228.

85.

R Development Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org

86.

Reutter

Jr. (1965). Church-state-education: Another man’s view. Educational Administration Quarterly, 1(2), 12-18.

87.

Richardson

J. W.

McLeod

(2009). Where should educational leadership authors publish to get noticed by the top journals in the discipline? Educational Administration Quarterly, 45(4), 631-639.

88.

Riehl

C. J.

(2015). Mostly unpunctuated disequilibrium: A commentary on new directions in research and practice in education leadership. In Bowers

A. J.

Shoho

A. R.

Barnett

B. G.

(Eds.), Challenges and opportunities of educational leadership research and practice: the state of the field and its multiple futures (Vol. 6, pp. 225-248). Charlotte, NC: Information Age.

89.

Riffel

J. A.

(1986). The study of educational administration: A developmental point of view. Journal of Educational Administration, 24(2), 152-172.

90.

Rowan

(1995). Learning, teaching and educational administration: Toward a research agenda. Educational Administration Quarterly, 31(3), 344-354.

91.

Sagan

(1997). The demon-haunted world: Science as a candle in the dark. New York, NY: Ballantine Books.

92.

Sage. (n.d.). Educational Administration Quarterly: Aims and scope. Retrieved from https://us.sagepub.com/en-us/nam/educational-administration-quarterly/journal200854#description

93.

Shwed

Bearman

P. S.

(2010). The temporal structure of scientific consensus formation. American Sociology Review, 75(6), 817-840.

94.

Sidorova

Evangelopoulos

Valacich

J. S.

Ramakrishnan

(2008). Uncovering the intellectual core of the information systems discipline. MIS Quarterly, 32(3), 467-482.

95.

Smith

(2015). Tesseract [Open source OCR engine]. Retrieved from https://code.google.com/p/tesseract-ocr/

96.

Steyvers

Griffiths

(2007). Probabilistic topic models. In Landauer

T. K.

McNamara

D. S.

Dennis

Kintsch

(Eds.), Handbook of latent semantic analysis (pp. 427-448). Mahwah, NJ: Lawrence Erlbaum.

97.

Thomas

A. R.

(1986). Seeing isn’t believing? Neither is hearing! In defense of observational studies. Educational Administration Quarterly, 22(1), 29-48.

98.

Tschannen-Moran

Firestone

W. A.

Hoy

W. K.

Moore-Johnson

(2000). The write stuff: A study of productive scholars in educational administration. Educational Administration Quarterly, 36(3), 358-390.

99.

Tyack

Cuban

(1995). Tinkering toward utopia: A century of public school reform. Cambridge, MA: Harvard University Press.

100.

University Council for Educational Administration. (n.d.). About UCEA: Improving leadership and policy. Retrieved from http://www.ucea.org/about-ucea/

101.

University Research Services & Administration. (2015). High performance computing. Retrieved from http://ursa.research.gsu.edu/research-computing/high-performance-computing/

102.

Vigoda-Gadot

Beeri

Birman-Shemesh

Somech

(2007). Group-level organizational citizenship behavior in the education system: A scale reconstruction and validation. Educational Administration Quarterly, 43(4), 462-493.

103.

Wang

Bowers

A. J.

(2016). Mapping the field of educational administration research: A journal citation network analysis of the discipline. Journal of Educational Administration, 54(3), 242-269.

104.

Wang

Fikis

(2016, May). How do the Common Core State Standards unfold on Twitter: Mining opinions and identifying opinion leaders. Paper presented at the 2016 annual meeting of American Educational Research Association, Washington, DC.

105.

Willower

D. J.

(1975). Theory in educational administration. Journal of Educational Administration, 13(1), 77-91.

106.

Willower

D. J.

(1981). Educational administration: Some philosophical and other considerations. Journal of Educational Administration, 19(2), 115-139.

107.

Willower

D. J.

(1996). Inquiry in educational administration and the spirit of the times. Educational Administration Quarterly, 32(3), 344-365.

108.

Willower

D. J.

Forsyth

P. B.

(1999). A brief history of scholarship on educational administration. In Murphy

Seashore Louis

(Eds.), Handbook of research on educational administration (2nd ed., pp. 1-23). San Francisco, CA: Jossey-Bass.

109.

Witziers

Bosker

R. J.

Kruger

M. L.

(2003). Educational leadership and student achievement: The elusive search for an association. Educational Administration Quarterly, 39(3), 398-425.

110.

Yang

C. D.

(2002). Knowledge and learning in natural language. New York, NY: Oxford University Press.

111.

Yang

Torget

A. J.

Mihalcea

(2011). Topic modeling on historical newspapers. In Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (pp. 96-104). Stroudsburg, PA: Association for Computational Linguistics.

112.

Zhang

(2015). Constructing a methodology toward policy analysts for understanding online public opinions: A probabilistic topic modeling approach. In Janssen

. (Eds.), Electronic government and electronic participation (pp. 72-79). Amsterdam, Netherlands: Inter Ocean Services.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.02 MB

0.00 MB