Abstract
Over the past two decades, educational research in Germany has undergone unprecedented changes. Following large-scale assessments such as the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA), and a political interest in evidence-based policy-making, quality assessment and internationalization, direct involvement of national decision-makers has led to the establishment of new organizations, programs, funding structures, professorships, and training programs. Thus, a markedly different educational research field has emerged in contrast to the traditional philosophy-rooted, hermeneutics-trained and humanities-based German pedagogy or educational science. Instead, the new paradigm refers to itself as empirical educational research (EER). Thus, we trace institutionalization processes of EER from 1995 through the foundation of the new Empirical Educational Research Association (GEBF), which rivals the long-standing German Educational Research Association (DGfE). Official documents shed light on policymakers’ and funding agencies’ motivations and rationales as they successfully engage in building new research infrastructure. Expert interviews conducted with (inter)national representatives illuminate perceptions of crucial actors involved in the organizational field’s institutionalization. What are the causes and consequences of the emergent EER field in Germany? Extending the neoinstitutionalist organizational field literature, particularly about incipient stages of such fields, we show that a new division of labor transcends national and international as well as governmental and non-governmental borders.
Keywords
Introduction: German educational research within global shifts
Education has received massive policy attention around the world, increasingly so over the past several decades. This political interest has not only substantially changed educational systems and their governance. It has also profoundly transformed the perceived goals and functions of educational research and the kinds of research to be promoted. Prominently, international large-scale assessments have grown in significance worldwide as aggregate educational performance is understood to be fundamental to reach social, political, and economic goals. Heyneman and Lykins (2008) count 33 international comparative studies spanning the entire life course of students and their achievements. A considerable body of literature has emerged tracing both causes and effects of international large-scale assessments (ILSAs) in a globalized educational environment that is marked by a “metrological mood” in general (Power, 2004: 766) and a “comparative turn” in education in particular (Martens, 2007: 42; Sellar and Lingard, 2014). As the “sociology of measurement in education policy” (Gorur, 2014) takes shape, it pays particular attention to international organizations (IOs) and their diverse roles in national policy making (Fenwick et al., 2014). Yet, countries demand and pay for educational knowledge, assessment and policy advice, even as the Organisation for Economic Cooperation and Development (OECD) and other IOs develop the mechanisms of governance and incentivize participation in large-scale assessments (see Martens and Jakobi, 2010; Meyer and Benavot, 2013).
Whether national policy-makers and educational administrators are genuinely interested in scientifically buttressing their decisions is difficult to unravel, yet the notion of evidence-based policy-making has spread quickly around the world. In “Modernising Government” (1999) the United Kingdom’s Labour government, for example, called for more evidence in educational research in order to facilitate crucial policy decision-making beyond ideological divides. Similar trends can be found in the United States and in Canada (Howlett, 2009), Australia (Banks, 2009) and, with a slight delay, in Germany (Aljets, 2015; Dedering, 2009).
Evidence-based policy-making in education is strongly anchored in specific kinds of educational research. Often, this research is called empirical, quantitative, interdisciplinary or multidisciplinary. Much of this research is reasoned causally as well as oriented to problems and their solutions. Such a description of educational research may not seem particularly novel. After all, many (especially Anglophone) countries have developed this kind of empirical educational research (EER) for decades. In contrast, educational research communities in other countries build on very different traditions. Germany represents the paragon of a humanities-based pedagogy or educational scholarship with origins in 19th century philosophy of education. This approach sits diametrically opposed to the “empirical” education research growing rapidly in recent years (Biesta, 2015).
Have global trends and international influences, visible in the rise of ILSAs, challenged Germany to transform the size, shape, and content of education research? We argue that German educational research has indeed experienced fundamental changes, marking the end of German ‘exceptionalism’ as the country experiences an “‘empirical’ turn in education policy and research” (Buchhaas-Birkholz, 2009). This turn occurred due to growing evidence-based claims in policy-making and ILSAs, especially the OECD’s Programme for International Student Assessment (PISA) (see Tillmann et al., 2008).
Germany’s tradition in educational studies and scholarship has been rivalled by an emerging empirical research infrastructure originating, as we will show, in direct government intervention. Policy actors have always been interested in harnessing the political, social and economic benefits of education through attempts to steer educational research. Yet, recent developments have, we argue, profoundly transformed education research conducted in Germany.
Because of the scope of these interventions, policy-makers’ intentions have not brought about a simple complement to the older research landscape. Rather, Germany has witnessed the emergence of an entirely new organizational field of educational knowledge production. This field adheres to a different “belief system” (Scott, 1995: 56). It distinguishes itself from the older educational research community, yet operates in close collaboration with (non-)governmental and (inter)national actors. It is, therefore, strongly embedded in expanding structures of a globalized organizational field (Chabbott, 2003) or international educational regime (Parreira do Amaral, 2011). Its backbone has been and continues to be ILSAs, while growing its roots deeper into the national research landscape.
To explain the emergence and contemporary changes in this evolving organizational field, we utilize neoinstitutionalist concepts. The complex structuration of this organizational field and the various building blocks needed in its initial construction involve both considerable “institutional work” and “institutional agency” (Lawrence and Suddaby, 2006). Therefore, we contribute to the understanding of mechanisms necessary to construct new organizational fields, a question relatively neglected compared to explanations of change in existing fields that are predominant in relevant research.
In the first part of the paper, we introduce our approach. Then, we present the results of our analysis according to chosen conceptual building blocks. Finally, we discuss and embed our findings for Germany within wider historical and international transformations in educational research and policy-making.
How do new organizational fields emerge?
Conceptually, the organizational field has become a prominent tool in neoinstitutionalist organization studies. The virtue of the field notion is that it bridges “organizational and societal levels in the study of social and community change” (DiMaggio, 1986: 337). The original definition of organizational fields revolves around “those organizations that, in the aggregate, constitute a recognized area of institutional life: key suppliers, resource and product consumers, regulatory agencies, and other organizations that produce similar services and products” (DiMaggio and Powell, 1983: 143). As communities of organizations, organizational fields reflect the interrelationships of diverse organizations sharing an environment (Aldrich and Ruef, 1999).
Research using such perspectives has long concentrated on phenomena taking place within extant fields. Therefore, we know much about the types, stages and effects of institutional change generally (Dacin et al., 2002; DiMaggio and Powell, 1983; Greenwood and Hinings, 1996) and see applications in various social domains such as art (DiMaggio, 1991), law (Dezalay and Garth, 1996), health care (Scott et al., 2000), international human development and education (Chabbott, 2003) as well as academic research (Cruz-Castro and Sanz-Menéndez, 2007).
By contrast, theorization and empirical investigation of the emergence of fields is scarce. This is surprising since the construction of new fields is logically and historically prior to inter-organizational processes in an established field. Here, we are primarily interested in these incipient stages of organizational field construction.
Powell (1999: 45) claims that “[n]ew fields emerge out of a felicitous combination of resources, technical know-how, and supportive organizations” and asserts that such “clustering is rarely serendipitous – it is a socially structured process.” In the case of EER, we explore these factors adding specific elements needed to do justice to both the particularities of our individual national case (Germany) and those of the social domain in which the field emerges – education and educational research.
Organizational fields are considered to emerge around problem domains or issues often too complex to be dealt with by any single organization, but they require collective engagement (Hoffman, 1999; Trist, 1983). Sometimes such a conceptualization adds distinct functionalist thrust to understanding fields by introducing strategic sense-making and attention to divisions of labor. Acknowledging such functional relevance, we nonetheless prioritize field members’ shared adherence to a common “meaning system” or “belief system” (Scott, 1995: 56).
Initially, within such domains, fields may unfold around a groundbreaking innovation as in Powell’s (1999) account of the dynamics sparked in the domain of biotechnology following new drug development. Alternately, innovation might be replaced by the occurrence of a disruptive event, such as regulatory transformations (Edelman, 1990) or environmental catastrophes (Hoffman and Ocasio, 2001). In both accounts, as an equilibrium tumbles (whether inadvertently or through intervention), new entrants obtain access.
In these incipient stages, participants pursue strategies of “institutional work” done to consolidate an emergent field or change an extant one (Lawrence and Suddaby, 2006). It is often paralleled by identity projects and “boundary work” (Greenwood et al., 2002; Zietsma and Lawrence, 2010). These concepts are discussed in relation to EER and traditional educational studies below.
Concomitant with such deliberate actions is the inherent logic of continuous consolidation. DiMaggio and Powell (1983) propose four features of growing densification in emerging organizational fields: first, an increase in the extent of interaction among participants; second, the emergence of well-defined patterns of coalition and domination; third, an increase in the information load with which participants in a field must contend; and fourth, the development of mutual awareness among participants involved in common activities.
These features are all contingent on time as an independent variable. Fortunately, our case of EER in Germany has reached the end of its second decade, having matured into a field that spans multiple types of actors, sites and levels.
Increase in interaction, information exchange or awareness can occur gradually or abruptly, it may stagnate or even reverse. Alongside time, other structuration principles are similarly crucial. Thus, we account for time as an overarching ordering principle, but propose an original circular analytical framework that revolves around distinct building blocks needed in field construction (Figure 1).
Building blocks and processes in constructing organizational fields (source: authors’ own depiction).
The first building block in this model acknowledges that most organizational fields emerge neither in institutional voids, nor insular seclusion. Prior structures within the wider environment can take the form of organizational actors that may have remained outsiders, populating the fringes of previously legitimated fields, with difficulties in entering the field due to high social, economic or cultural barriers. Alternately, we can think of key actors in already existing fields whose adaptability enables them to jump on the latest bandwagon, participating in the establishment of a new field.
Environmental factors have been given considerable attention in classical neoinstitutionalist writings (Powell and DiMaggio, 1991). Related concepts of isomorphism and mimesis have been repeatedly highlighted in the context of both national and worldwide institutional change, especially relating to education (Meyer et al., 1997).
Next, critical shocks or innovations often are precipitating forces, with these events creating contradictions in organizations’ environment. Similarly powerful ideas, often enshrined in rationales of official policy papers and political speeches (Campbell, 2002), may force organizations into processes of collective sense-making. In the building of EER, the “PISA shock” and the core idea of the advantage of numerical evidence and its relevance for policy-making loom large.
In the field of EER, during its earliest moments, institutional work refers primarily to the political will and resources provided by policy-makers and particular individual agents, less to professional associations as in the case of DiMaggio’s (1991) museum field or particular organizations as in Powell’s (1999) account of biotechnology. Only later, in the institutionalization of the field, do we find a wider set of policy or institutional entrepreneurs defined as actors “who leverage resources to create new institutions or to transform existing ones” (Maguire et al., 2004: 657; also DiMaggio, 1988). Importantly, however, we propose a balanced view of field construction in which the relationship between agency and structure is decidedly recursive (Lawrence et al., 2009).
A fourth strand draws on Scott’s (1994) concept of belief systems that bind together the actors within organizational fields. Homogeneity of belief systems can vary from very high (one single dominant belief) to very low (two or more competing strong beliefs). We can easily relate this idea to another strategy of institutional work. Here, identity formation is crucial in the creation of new institutions in which “the relationship between an actor and the field in which that actor operates” is (re)defined (Lawrence and Suddaby, 2006). Similarly, the changing of normative associations and construction of normative networks (such as professional associations) represent outcomes of shared belief and meaning systems.
We assume that – over time – these intertwined factors may lead to routinized interaction patterns that nurture trust among participants. The latter aspect is not to be underestimated, as many research results produced in the field of EER represent politically-sensitive subjects requiring substantial cooperative efforts from both political and scientific partners to be legitimated.
This framework will structure our analysis. Next, we present our methodology and sketch the field’s internal structure.
Methodology: Reconstructing the evolution of organizational fields
Case selection
The evolution of EER in Germany is a rich case to trace processes of institutional change and organizational field construction. Firstly, change was radical. EER in the period prior to our observation looked markedly different from its contemporary shape. Secondly, two decades provide a sufficiently long span of time to trace the field’s growth, which was at times gradual, at times abrupt, burgeoning and maturing, without any imminent slowdown. Thirdly, the process involves a wide range of scientific, political, and administrative actors at sub-national, national and international levels. The full complexity of networks involved in field construction evolved gradually. We argue that EER, in its current state, is best understood as an organizational field. Indeed, EER is markedly different from what is generally thought to be an academic discipline (Ben-David and Collins, 1966) or a self-referential research field (Aljets, 2015), due to its high degree of permeability vis-à-vis non-national and non-scientific actors and its various self-proclaimed missions for the wider society – that extend beyond seeking (academic) knowledge.
The period of observation starts in 1995, with Germany’s participation in the Third International Mathematics and Science Study (TIMSS), the results of which precipitated an alarming event for German educational policy-makers, even if the media made much less of these than about later ILSAs (see below). We end our observation with the latest round of the OECD-PISA study in 2015.
Methods, sampling and data
In a preliminary stage, we gathered quantitative data on research funding, the structural scope of programs and related projects and changes in scientific personnel to complement our qualitative interview and documentary material.
Especially to understand the most recent period, semi-structured expert interviews constitute the main data source (see Bogner et al., 2009). We conducted ten interviews (approximately 90 minutes each) with experts from mainly two fields, science and policy-making, based on a purpose-driven sampling. 1 In this field, sampling was facilitated by the prominence of key figures involved in conducting ILSAs and field development in Germany more generally. For policy-makers, we interviewed actors involved in the major funding initiatives (state administration) and the relevant boards. We also spoke with actors occupying the middle ground between educational research and policy-making in inter/national contexts, including OECD and the International Association for the Evaluation of Educational Achievement (IEA). Finally, a site visit to the OECD’s Paris headquarters provided insights from conversations with analysts and education experts involved in PISA and global educational governance more generally.
We chose interviewees whose role in EER was relevant throughout the observation period (possibly from early to late stages of the process) and whose functions were multiple (e.g. researcher, advisory board member, project leader and consultant) to ensure that our analysis encompasses multiple perspectives. To assure anonymity, organizational affiliations are omitted in the presentation of results and interviewees are coded by A for actor and a consecutive number.
Our second key source consists of policy documents from the main organizational actors involved (N=38). The text corpus includes major statements from the Science Council (Wissenschaftsrat), the German Research Foundation (Deutsche Forschungsgemeinschaft), federal and state (Länder) ministries of education, speeches by Ministers of Education, and even federal government coalition agreements. We included calls of major programs and projects, organizational self-portrayals of the key EER institutes in Germany and professional associations as well as lists of project descriptions and conference participants to identify the national and international networks in which the field of German EER is embedded.
The emerging organizational field of empirical educational research
Field structure
Our first findings concern the structure of the EER field emerging in Germany over the past two decades. This field is distinct from both classical university-based scholarship and science-based, research-intensive industries in that a large majority of the field members are not purely academic (i.e. university-based), yet often without explicit commercial interests. EER in Germany, we observe, is unique in numerous respects. The built-in promises of strong applicability for educational governance and policy-making moves it close to public actors. The rapid and considerable investments in EER infrastructure led to the founding of new organizations and the upgrading of extant ones, which now enjoy previously unheard-of annual budgets and operational portfolios. The recurrent ILSAs set a fast pace for organizations to design, collect, process, analyze and publish vast amounts of complex data. This spawned large operational overheads. Most importantly, due to the formidable challenges posed by such data-intensive research, the field stretches out – lattice-like – into organizations beyond the research university and beyond national boundaries, strengthening a vast international and extramural research network.
Not unlike organizational fields studied in an industrial context, German EER is organized around a fairly clear-cut division of labor, albeit with a major distinction between the actual research workers at the core of the field (its population) and the organizations surrounding the core (the wider field). Figure 2 shows organizational members at both levels 2 and these organizational members are not necessarily distinguished by type, but by what they do, that is, their roles in the overall field.
The structure of the empirical educational research field and population in Germany (source: authors’ own depiction).
At the field level, we find a global and highly diverse set of organizational actors. Most of these are not exclusively active in education or research, but science more generally. Field members encompass governmental (Federal Ministry of Education, BMBF) and non-governmental (German Empirical Educational Research Association, GEBF), national (State Ministries of Education organized in the Standing Conference of Ministers of Culture or Kultusministerkonferenz, KMK) and international organizations (IEA and OECD). Most field members do not engage in data work as their major function, although they do fund and publish data reports. Instead, their functions are primarily regulative by setting legal frameworks that are often necessary when conducting large surveys (KMK and BMBF), financial by providing resources (German Research Association, DFG; BMBF), normative by setting agendas and standards (German Council of Science and Humanities, WR; DFG; GEBF) and cognitive by advancing methodological and policy designs (IEA and OECD). However, we acknowledge boundary-spanning among organizational actors and between levels. The IEA and OECD, in particular, conduct original (meta-level) research and host numerous analysts. Ministries, too, have increased their internal research budgets, with the national education ministry financing regular national level education reporting.
The population level is more homogeneous by type and mission than the field level consisting of “aggregates of organizations that are alike in some respect” (Scott, 1998: 125). Here, we find research organizations proper, all dealing directly with data. Some of them have escaped legitimation crises before ILSAs became a recurrent event in Germany (e.g. German Institute for International Educational Research, DIPF; Max Planck Institute for Human Development, MPIB). Others, newly created, assess quality (Institute for Educational Quality Improvement, IQB), internationalize German research (DIPF’s International Cooperation in Education, ICE), or bundle ILSA expertise. Examples include the Center for International Student Assessment (ZIB), collaboration between DIPF, the Institute for Science and Mathematics Education (IPN) and the Technical University of Munich’s School of Education or the Leibniz Institute for Educational Trajectories (LIfBi), which hosts the National Educational Panel Study (NEPS). All of these organizations have experienced dramatic expansion in terms of operational portfolios, research units, budgets and staff. For example, institutes within the Leibniz Association like IPN and DIPF doubled their staff in the last 7 years to more than 170 and 300 employees each. The IEA’s Data Processing Center (DPC) founded in 1995, originally employing five people, is now staffed with 130 employees.
It is important to note that the population level is not necessarily national or limited to public research institutes. Certainly, ties are tighter between national ones and those organized under the same umbrella (e.g. the Leibniz Association), yet the Australian Council for Educational Research (ACER), the Cochrane Center, the Educational Testing Service (ETS) and the IAE–DPC are international research or research-related institutes specialized in educational test design, assessment, analysis, and consulting that are active in the field in Germany. As one interviewee noted: “It doesn’t matter if I send an email to someone, skype or phone someone next door, next city or next continent” (A IX).
Another distinct feature of the population is that universities are conspicuous by their absence. Even those university research units known in the mid-2000s for their empirical research were merged, such as the newly-established extramural Leibniz Institute (LIfBi) initially part of the University of Bamberg. Only recently has the field started to feed back into the traditional university infrastructure. First, some organizations officially affiliate with universities (e.g. ZIB in Munich) sometimes due to prominent individual researchers continuing their EER career in universities. Second, strong legitimacy, unprecedented long-term funding, and a broad research agenda assured by big data have structurally changed higher education. The official Education Science Data Report mentioned “empirical educational research” for the first time in 2006. Portraying the rise of EER in Germany, Enno Aljets (2015) estimates that if job postings in EER more than doubled from 2003–2006 to 2007–2010 (from 34 to 73), the actual number of university professors officially working in EER would have almost tripled from 27 (3% of all education professors) in 2006 to 75 (8.5%) in 2012. Another structural change concerns teacher education, now increasingly organized within so called Schools of Education, usually with explicit EER profiles and offering doctoral studies, found in Bielefeld, Bochum, Duisburg-Essen, Frankfurt am Main, Munich, Tübingen, and Wuppertal, for example.
As mentioned above, inclusion in the field is primarily defined by cognitive and functional principles. Moreover, regardless of their positioning, members at each level recognize each other as partaking in a common project and do so to a greater extent over time as interaction becomes more ritualized and tasks more complex. Examples of such growing awareness and trust among members will be presented below. Importantly, we make no assumptions about field hierarchy here. As much as population-level organizations depend on resources and regulatory support (e.g. data access) from field organizations, the latter depend on the expertise held by population actors; a symbiosis between levels as well as within the same level, as organizations interlock. The federal BMBF, for example, decided to boost EER in Germany through substantial framework programs. These programs, however, have not always found sufficient applicants in an as-yet insufficiently institutionalized field. Indeed, without support from state ministries of education to support developments more generally, these top-down programs risk missing the mark. A further example is that PISA required expertise from IEA and ACER. Leibniz institutes in charge of PISA contract with DPC to carry out data collection. The DPC, in turn, learns from sampling strategies developed by Weststat and Statistics Canada: “Certain research institutes have specialized in a particular field, they do what they can do best. In other cases, as with large-scale sampling, there are only a few institutes worldwide that can do that” (A IX).
Another case is LIfBi, which, in designing NEPS, drew on experience from ETS (recruiting staff), with the latter recognized as an essential international project partner – even by federal ministers (Schavan, 2009).
Field construction
We reconstruct the emergence of EER in Germany focusing on crucial building blocks while acknowledging the chronology of major and minor events (see Table 1). However, we question whether strict linearity, periodization, and causal sequencing – as often implied in accounts of institutionalization or process-tracing – do justice to the complex rhythm of this empirical case.
Key events in the construction of the empirical education research (EER) organizational field in Germany (source: authors’ own depiction).
Note: *, for the defininitions of abbreviations used in Table 1, see text.
Prior structures and the field environment
The first wave of international assessments initiated by IEA in 1960 (feasibility pilots) and 1964 (mathematics) was not well-received among German education scholars. Anecdotes have it that one survey item used in the first study was decried in the education community due to its absurdity and the whole assessment enterprise of the IEA was denigrated, with IEA becoming an acronym for “it est absurdum” (A IX). Consequently, Germany did not participate in the second round of the International Mathematics and Science Studies in the early 1970s and did not show much interest in participating in further studies until the 1990s.
Many research institutes that continue to be of major importance up to today were founded at that time of the early assessment initiatives, not long after World War II, such as the aforementioned DIPF (1951), MPIB (1961), and IPN (1966). They were “fact-based research bodies”, joining wider calls for a “realistic turn in pedagogical research” (Roth, 1963: 109), but were soon led by more historical, comparative and sociology-oriented directors reflecting the wider research tradition in education (A VII). Apart from a few mainly empirically-oriented researchers, usually organized in the empirical section of the DGfE (founded in 1965), research on education in Germany was dominated by deeply-rooted scholarship of educational theory, philosophy, and history, with different (sub-)disciplines such as pedagogy, didactics and educational science. These subfields added up to a highly idiosyncratic amalgam, with a markedly different approach to those established in other countries (Biesta, 2015).
Another of the necessary building blocks can be found in the environment of the emergent field. Literature on organizational fields is often confined to national systems, a situation that is less than satisfactory not only when explaining the current scope of the field, but particularly when looking at the initial stage. The wider environment of German EER is, of course, international and this increasingly global context provided important “building materials” needed in the field’s construction. The materials include international organizations’ and other countries’ experiences in both EER and the wider arena of evidence-based policy-making. By the end of the 1990s, such bodies as the National Education Research Forum, the Centre for the Economics of Education, the Centre of the Wider Benefits of Education or the Evidence for Policy and Practice Information and Coordinating Centre had been established in the UK. In the US, the Institute of Education Sciences, the What Works Clearinghouse, or the Best Evidence Encyclopedia followed in the early 2000s. Moreover, the non-governmental Campbell Collaboration started – as did its sibling in medicine, the Cochrane Foundation – to produce and publish systematic reviews in education. Several international organizations (IOs) that have become crucial actors in education over the past 20 years have also begun to call for “more evidence” in education (OECD, 2007).
Such global developments did not remain unnoticed in Germany (A IV). The German Cochrane Center, for instance, dates back to the early 2000s and it is now regularly invited to conferences on evidence-based policy-making in education.
3
Repeatedly, policy-makers and advisory bodies have stressed the importance of “catching up” and “keeping up” with “international knowledge standards” (WR, 2001: 72), situating Germany in the context of the “European educational space” (Schavan, 2009: 2). Zymek (1975) and Gonon (1998) long ago pointed to the power of “international arguments” in domestic educational reform debates; this applies to the making of EER, too: “Those countries that have participated in international assessments with success explain their performance referring to the implementation of “learning organizations” in the educational systems.” (BMBF, 2008: 6)
Innovations, shocks, and ideas
Organizational fields heavily depend on innovations. These innovations provide opportunities for new entrants to challenge an established order, changing the boundaries of a field. Several fundamental technological novelties transformed educational research and research organization in general, from the wide-spread use of information technology (IT) in international project management, high-performing computing software, video analysis or technology-based assessment. Yet the major innovation in educational research in the 1990s is the introduction of international, comparative, large-scale output-oriented assessments in education first through TIMSS, then PISA and others. The results of these comparative assessments produced a societal shock, with the effects on education systems in Germany variegated and hotly-debated, but difficult to causally identify. Yet it is beyond any doubt that the building of the EER field in Germany has been and continues to be largely composed of institutional and organizational responses to these globe-spanning ILSAs and their comparative determination of value. While all of our interviewees agreed on the relevance of ILSAs as the embryo of the nascent field, they stress different dimensions of this innovation.
PISA, for example, shifted the focus from curriculum-based learning (and soft skills) or “inert knowledge” (A III) to cognitive skills in the guise of scientific literacy, which later provided much inspiration for important curricular reforms (national education standards) in Germany, especially in mathematics and natural sciences (A VIII). It used an age-based sampling, included reading skills and the (controversial) application of the Rasch model, a psychometric model for analyzing categorical data (e.g. answers to questions in reading assessments or questionnaire responses) (A V). The PISA results were particularly explosive in the German context of the widespread exclusion of students attending special schools and considering the particularly poor performance of pupils with lower socioeconomic and/or migration background. The OECD’s decision to later present rankings of countries (and intra-national rankings of German Länder) further fanned the controversial reception of results. Overall, the rise of EER has opened up tremendous opportunities for experts in quantitative methods. Probabilistic models (instead of distributional models), multi-level analysis, longitudinal designs, context- and application-embedded testing all have become markers of innovation in EER. Indeed, continuous testing and monitoring at all educational levels now keeps those in the field of EER very busy conducting international assessments such as TIMSS, PIRLS, PISA, PIAAC or TALIS, and also through national extensions of international studies or ambitious national projects like the longitudinal NEPS.
It is important to note that ILSAs, such as TIMSS and PISA, marked a methodological turn, but they did much more, as they were disruptive events that sparked considerable public outcry—more than in other countries (Gläser et al., 2014). In Germany, the disruption caused by these ILSAs opened up a window of opportunity for the building blocks of EER to be placed one atop the other. TIMSS and PISA represented a “shock” to national policy-makers as they revealed mediocre results achieved by German students, something nobody would have expected. Yet if TIMSS triggered reactions among policy-makers and remained largely an “insider’s” debate (see KMK, 1997), the first PISA round in 2000 became an extraordinarily visible writing on the wall. Politicians, parents, teachers, administrators, and researchers were suddenly involved in a rather fierce debate on German education, often fueled by sharp comments from the mass media. 4
The impression arose among field members that educational research in Germany had for too long been a “sleeping beauty” (Buchhaas-Birkholz, 2009: 27), unaware of the actual state of quality. With important exceptions, German educational research was perceived as ill-equipped to deal with large-scale assessments and with the challenges of education reform in general. The humanities-laden legacy of German education scholarship weighed heavily on the research structures, mostly inside universities, but also outside them. Thus, as discussed below, much policy action post-PISA 2000 was directed not so much to change existing research practices, but to build an entirely new research infrastructure and culture.
Yet, ILSAs are only half the story. Innovative testing and measurements of school performance coincided with the idea of evidence-based policy-making diffusing from health policy to social and educational policy domains. Policy-makers’ interest in educational data is manifest in virtually all statements. The BMBF sees national and international assessments, the search for causal mechanisms in teaching and learning, educational statistics, and a “system of internal and external evaluations” as directly linked to evidence-based practices allowing for “long-term preventive policymaking” (BMBF, 2008: 6; also WR, 2001: 69). Under the German EU Presidency, the BMBF initiated the first Europe-wide conference on “Evidence-based Education Policy” (2007) attended by more than 300 stakeholders from EU and Council of Europe member countries. The KMK (2006: 5), too, is convinced that “outcome-orientation, accountability, and system monitoring mark a paradigm change” in German education policy. Scientific knowledge becomes the primary grounds upon which to make political decisions: “As with other policy domains: political action in education can only be genuinely responsible if we face and take into account scientific knowledge and findings” (Schavan, 2009: 3).
Political will and resources
Among the most remarkable features of the policy discourse in Germany around the millennium was that the relevant field members became almost unanimously convinced of the need to develop EER. State ministers – the primary locus of education policy-making in Germany – first reacted in 1997 agreeing to further participate in ILSAs. The Science Council (WR) and the German Science Foundation (DFG) joined the debate in 2001 and 2002 with clear statements on the lack of resources, researchers, quality and international visibility in education research (DFG, 2002; KMK, 1997; WR, 2001). The federal government, which is usually barred from involvement in educational (research) policy-making, entered onto the stage in 2007/2008 with its highly significant Framework Program for the Promotion of Empirical Education Research.
Since then, EER has seen an unprecedented volume of investment. It is difficult to identify exact figures because financial support is often spread across different funding sources and funding lines. It includes international and national studies, graduate schools, schools of education, and university chairs. As prescient examples, we examine the federal BMBF-sponsored Framework Program. The program represents an exceptional intervention in educational research in terms of financial scope, structural width and substantive depth. Between 2007 and 2019, the BMBF Framework Program will have funded roughly 342 research projects in eleven defined priority areas, a data center, NEPS and the training of young researchers in EER (doctoral stipends). The total sum – more than €163 million – is remarkable considering that educational departments at German higher education institutions received between 45 million (2007) and 80 million (2013) annually. 5 The Program also shows the proactive role of the DFG in EER, in assessing and reviewing the NEPS project proposal, although it does not fund the panel study (a very uncommon practice). The BMBF knew that with DFG support the program would gain significant legitimacy (A VI).
Other important initiatives include the funding line in higher education (2008–2018: €90.2 million; 154 projects) 6 state-by-state comparisons of school achievement and international assessments (e.g. PISA with €15 million).
Besides financial support, BMBF and KMK are also key regulators. National educational standards now used as benchmarks in mandatory intra-national comparative studies were introduced in 2003. States also agreed to make participation in certain ILSAs mandatory (TIMSS, PISA and PIRLS/IGLU).
Further measures include facilitated access to data, an often thorny issue for empirical researchers. The 2006 state-level Comprehensive Strategy for Educational Monitoring ensures that access to schools and related data is granted. In 2012, the Educational Research Data Center opened at DIPF; soon after, the Educational Research Data Consortium brought BMBF, DIPF, IQB and others into collaboration on accessibility and usability of scientific data.
Eventually, BMBF and KMK agreed on a new system of educational monitoring (KMK, 2006) with the first National Education Report published in 2006 and appearing biannually since. In parallel, state educational reports (produced by more than ten Länder by 2015) and numerous local or district educational reports increase available knowledge on developments in education (see Busemeyer and Vossiek, 2015 for a review).
Finally, although there are signs of growing consolidation at the organizational level and more widespread institutionalization of the field as a whole at the level of the research system, the field’s strong dependence on political support, massive investments and recurrent ILSAs suggests three built-in threats. First, the on-going dominance of expensive, large-scale and program-based funding bears the risk of fragile sustainability. If governmental action has always been important in creating and shaping research fields due to the necessary investments (Cozzens and Woodhouse, 1995), with the federal ministry being the primary institutional entrepreneur, in Germany this is a relatively new phenomenon and demonstrates the paramount and growing importance attached to education among those in government. Yet, actors also complain about the volatility of the research agenda, with months of relentless work followed by times of labor slack (A IX). A second and related aspect is that the rapid influx of investments prioritizes quantity over quality. Several observers (A II, VII and VIII) noted that the funding logic of the BMBF differs from the DFG in that it puts much more emphasis on economic and social impact than scientific quality alone, as in the extensive DFG peer reviews. Whereas one mode of governance emphasizes structural aspects, the other utilizes a more epistemic approach, with implications for securing quality in research. Only recently has the BMBF widely applied peer-review in funding decisions, which also indicates that the times of “easy and cheap money” are over (A II, VIII). Further, some observers forecast an inevitable shake-out in store for the organizational field. They fear that many empirical researchers trained and recruited in the wake of ILSAs and the Framework Program will face grim job prospects once the flood ebbs, especially in an academic system in which jobs at the level of full professors as well as more junior tenured positions are scarce (A V, VI).
A third aspect concerns the relationship between research and policy-making. Polemic reactions from the community follow once researchers move closer to policy-making debates (A VII). Some interviewees call it an “unfortunate incident” (A V, VII) that the notion of evidence-based policy-making had made its way into the academic discourse already by the early and mid-2000s. These academics fear that expectations of highly usable research findings ready for implementation were unduly awakened among policy-makers. With such hopes now dampened, the more realistic prospects could well lower spending enthusiasm in the future and even call the evidence-based policy-making experiment as a whole into question.
Belief system and identity
Belief systems bind together members of a given organizational field and help to integrate new candidates for membership in phases of formation. Conversely, they draw boundaries of distinction and can vary in exclusiveness (Scott, 1994). In the case of EER, the single and dominant belief system consists of a positivist, mostly quantitative “exact social science” (WR, 2001: 33). Consequently, humanities-based education scholarship is not viewed as a part of this belief system – or even as a hindrance (WR, 2001: 33). The BMBF leaves no doubt that “empirical education research is markedly different from the conventional work done in the more humanities-informed (school) pedagogy” (BMBF, 2008: 8).
In a peculiar analogy to the so-called “methodological dispute” (Methodenstreit) in the social sciences in the 1960s between positivists and critical theorists, these two paradigms seem to clash again in education, bringing the debate into policy-making agencies and the mass media. 7
Further strategies of boundary-drawing and identity-formation are clear in two field-wide developments. One is the “rebranding” of teacher education as “Schools of Education” that, alongside traditional teacher training, assume the additional mandate of training in research (methods). Another, and among the most consequential for education scientists’ and professionals’ identities, is the founding of a new professional academic association devoted to EER. In fact, two different associations now serve the education community, with the long-standing German Educational Research Association (Deutsche Gesellschaft für Erziehungswissenschaften, DGfE) and its empirical section now competing with the recently-founded Association for Empirical Educational Research (Gesellschaft für Empirische Bildungsforschung, GEBF, 2012). Prominent educational researchers from several disciplines united to found the GEBF in 2012, signaling that the organizational field has hit puberty – symbolically dissociating itself from previous generations. While some interviewed EER researchers are still members of both organizations, these increasingly wonder if the traditional camp “wants them to be around at DGfE conferences at all” (A V). Both sides draw boundaries and claim to pursue the “correct” kind of educational research. Traditional DGfE members reproach the new empirical generation, calling it “methodological economism” (Radtke, 2015), while the latter criticize their pedagogical counterparts’ “other-wordly aesceticism” (A III) and propose applied research with a “service character” (A II, III, IV, VII and VIII).
As telling as such a new association is in terms of symbolic boundary-drawing and identity formation or indeed cleavage construction, a more important analytical thrust lies in focusing on its organizational character as a forum, coalition or research prism – more than as an organization of disciplinary representation. Interviewees actively involved in the organization’s establishment compare it to collaborative scientific communities in climate research, maritime research, public health or life sciences (A VII and VIII) and stress the GEBF’s multidisciplinary, or in fact interdisciplinary, nature. Serving as a center of gravity in a research field with a permeable orbit, the GEBF deliberately attracts researchers from diverse fields such as education, psychology, sociology, economics, statistics, and computer science as long as they share and commit to a similar empirical trajectory.
Time
Interaction, awareness, ritualization and trust
Based on our research on the development of the organizational field of EER, we argue that this case neither falls into the general stages of institutional change as outlined by Greenwood et al. (2002) nor the more case-related phases proposed by Aljets (2015). Instead, we make two claims in relation to temporal dynamics: interactions among field members on both levels have increased; and collaborations in research work on the population-level have grown ritualized. Consolidated networking occurs horizontally at each level and vertically between field and population members. What started as ad-hoc coalitions born of necessity have evolved gradually into more routinized consortia aimed at creating economies of scale and stronger visibility for individual researchers and the field as a whole.
The solid backbone around which diverse organizations vertically cluster is made up of ILSAs. Fields gain density over time as their members share ever-larger information loads (DiMaggio and Powell, 1983). Between 1995 and 2015, Germany participated in fourteen ILSAs. A typical PISA survey is a mammoth logistical job dealing with 40,000 students in 1,500 schools in 16 independent and differently-administered states (A VIII). Such magnitude means, in our interviewees’ words, that “while one PISA round is not even finished, the next one is in full action, and a third one in preparation” (A VIII and IX).
Coordination and management of these assessments usually brings together a similar set of national coordinating actors, namely BMBF and KMK, and research organizations, such as the Institute for School Development Research (IFS), IPN, DIPF, TUM School of Education and IQB as well as diverse international actors. Here, we identify ACER, OECD, IEA/DPC and ETS (with the latter two collaborating in their joint research institute, IERI). As one observer ironically notes: “We know our usual suspects” (A IX). We also find here an example of horizontal cooperation in the Center for International Large-Scale Assessment (ZIB) aligning DIPF, IPN, TUM School of Education and IQB around PISA to “be more efficient and to bundle expertise” (ZIB, 2015). An even larger group is the new research consortium Leibniz Education Research Network (LERN) that unites educational researchers from 15 Leibniz institutes and the IQB with the goal to “optimize social development and the harnessing of educational potential” (DIPF, 2015).
Repeated interaction and routinized divisions of labor provide fertile ground for building awareness and trust. As the early TIMSS and PISA debates had shown, educational research can become a delicate political issue. Policy-makers in this initial phase were interested in collaborating with researchers that were competent and reliable and whom they could trust. For researchers, political and financial support to conduct large-scale research was at stake. For them, the perspective of long-term continued funding was crucial. For their part, politicians felt that education policy-making and educational researchers would increasingly take on the roles of “natural partners in the development of the education system” (Schavan, 2009: 3).
Discussion
The emerging empirical research infrastructure in Germany should not be mistaken for the institutionalization of an academic discipline as in Joseph Ben-David and Randall Collins’ (1966) account on the evolution of psychology, because its origins are explicitly extra-mural and multidisciplinary. Instead, we have identified a highly diverse organizational field that cuts across categories of levels (international/ national/ subnational), funding (public/ private), mission (scientific/ commercial), function (knowledge-producing/ knowledge-using), and, perhaps most importantly, disciplines.
Universities, in particular, with their age-old division of labor and reification of disciplinary boundaries, seem less well-equipped to handle the ILSA innovations and to span disciplines, which are burdensome in a field that is more defined by problem priority areas and policy-generated purposes than academic traditions. As one observer noted, it not only takes visionary heads leading their organizations to new research frontiers, but also the room for maneuver to do so, something less often found in abundance within academic departments (A VI).
Nevertheless, the boundary-spanning bridges between extramural and university research, the mobility of leading professors and senior scientists, an ever greater number of professorial chairs in EER, the empirical socialization of young researchers in newly created Schools of Education and Graduate Schools together with the doctoral stipends offered through BMBF and DFG funds are all likely to stretch the field into universities. Especially the educating, training and recruiting efforts represent an important part of institutional work necessary to solidify an emerging field (Lounsbury, 2001).
The founding of the GEBF also suggests significant consolidation as professional associations have been shown to play a major role in the constitution and evolution of organizational fields (Greenwood et al., 2002). Such associations are carriers of normative codes, exert standardizing pressures, theorize practices, and legitimate change. Although still a nascent organization, the GEBF has already, through its conferences and its journal, achieved a measure of institutionalization.
Moreover, EER is also more than a pure research field institutionalized as a functional response to ILSAs, as depicted in Aljets’ (2015) important study. Although PISA and other ILSAs may be considered triggers and catalysts putting into motion important political support and unleashing unprecedented federal funding, such a limited focus obscures the wider debate on evidence-based policy-making in education that explains public actors’ behavior.
More importantly, this interpretation misses the larger frame of the rationalization of educational research and thinking in Germany. With almost no record in educational monitoring, reporting, large-scale data generation and analysis before the end of the 1990s, the country has – in less than 15 years – “overcompensated” for this lack of educational monitoring. Indeed, key figures in the field have called for less spending – and less data (A V and VIII), as they acknowledge the lack of capacity to utilize all these resources.
The transformations shown here reflect wider tendencies of the rationalization of the world (Bromley 2010, 2016; Meyer et al., 1997) and the ways of thinking about education in an “age of measurement” (Biesta, 2009). We are not just witnessing the diffusion of a certain research practice, but the emergence of a new authoritative model of how to govern and carry out (particular types of) educational research. This includes whole new infrastructures as well as relationships among organizations constituting an expanding organizational field of EER, legitimated by developments in the international environment, by national decision-makers, and by a burgeoning professional association. In this sense, the German infrastructure can be understood as docking onto an increasingly internationalized education and educational research sector – at long last (Chabbott, 2003; Dale, 2005; Parreira do Amaral, 2011). However, in order to do so, Germany’s traditional education science seemingly had to be cleansed of its “provincialism” (Actor III), distinctive heritage, national exceptionalism, and parochial practices that constituted the “pedagogical Sonderweg” (Lenzen, 1998: 3; also Tröhler, 2015: 4).
Yet many of the study’s interviewed experts view this sprawling international and interorganizational nexus with mixed feelings. Some admire the complex, but smooth operations of regular large-scale assessments (A VII). Others point to risky monopolies created in data collection and analysis (A V). Some feel that decision-making in these large programs, governed by an emerging “holy alliance between science, politics, IOs and industry” (A VIII) becomes centralized, with scientists in particular locales having little say. Overall, this boom in measuring “competence”, for example, also invites purely commercial actors such as large IT firms to enter the scene with their research portfolios, edging out fundamental scientific priorities and paradigms (A VIII).
Actively-involved EER researchers confirm that this new field revolves around an “object” (A V) or an “issue” (A VIII) – as opposed to a complete discipline. Importantly, the questions those researchers seek to answer emanate less from the scientific system itself, rather than reflect practices embedded in the field, education system, and society. Here, science is understood as morally obligated to give something back to society, to repay its debts in research understood to be relevant (A VII), rather than autonomous science seeking new knowledge for its own sake.
Conclusion
In this paper, we tracked the construction of the organizational field of EER in Germany over the period 1995–2015 by disentangling the field’s building blocks and institutionalization processes. We depicted “empirical educational research” flourishing in the wake of methodological innovations and political implications catalyzed by international large-scale assessments. We noted three key characteristics of this emergent field: its exclusive focus on data-intensive quantitative empirical analysis; its highly international scope (importing foreign ideas and exporting German research); and its proximity to applied educational research, making it the preferred candidate for activist policy-makers seeking to base their decisions on evidence and expand their influence in the governance of education systems.
We identified an organizational field with organizations directly conducting research (organizational population) and diverse organizational actors surrounding that population that provide the financial, regulative, normative, and cognitive frames. This field is based on a common belief and meaning system as well as a division of labor, with stark boundary-drawing to distinguish it from traditional education or education studies more generally. After a boom time in the mid-2000s, participants in the field now call for more careful spending, an increased focus on quality and less dependence on ILSAs as the major infrastructure producing the objects of research. Institutionalization progresses, with consolidation visible in the tightening of old networks and the forging of new ones in each successive researcher generation. These networks are not necessarily confined to national boundaries. In fact, EER in Germany can best be understood as a national node in an expanding international research network that prioritizes certain types of knowledge and proffers data and answers to facilitate education governance based on its construction of problems and standards. Exploring the interaction effects of expanding inter/national research infrastructures – and the growth of a truly global organizational field of EER – would provide considerable analytical thrust to future studies of institutional change in educational knowledge production and governance.
Footnotes
Acknowledgements
The authors thank the interviewees for sharing their knowledge of the field and Marcelo Marques, Jennifer Dusdal, Barbara Hoenig, and the anonymous reviewers for helpful comments.
Declaration of Conflicting Interest
The authors declare that there is no conflict of interest.
Funding
This research was funded by the University of Luxembourg’s tandem program for interdisciplinary research (Project EDRESGOV: The New Governance of Educational Research: Comparing Trajectories, Turns and Transformations in the United Kingdom, Germany, Norway and Belgium).