On the re-institutionalization and diversification of Public Administration education in Central and Eastern Europe: A case study of post-communist Romania

Abstract

The article provides a complementary view to those accounts of the growth of public administration education in post-communist Central and Eastern Europe that (a) explain its development primarily as a reflection of changes in administrative cultures while (b) emphasizing the ongoing process of curricular diversification. Rather, the article shifts the focus on the internal dynamics of higher education. It shows that the development of Romanian public administration education can be attributed, to a considerable extent, to unique opportunities arising in a period of post-communist re-institutionalization, of which enterprising academics ably took advantage. It further argues that, curricular variety notwithstanding, public administration departments in this country may be growing more alike in other respects. In so doing, the article contributes to an expansion of the traditional narrative of the growth of public administration education in post-communist Europe.

Keywords

Public administration education diversification curricula isomorphism department rankings g-index

Introduction

A seasoned insider has recently observed that administrative science appears to be afflicted by a “narcissistic…self-obsession,” noting further that “[i]t is hard to think of many areas of scholarship that have spent so much time ruminating on what they are, with such inconclusive results” (Fenwick, 2018: 10). Administrative science has arguably developed, in part as a result of this incessant self-reflection, a dominant disciplinary narrative.¹ As in all academic disciplines in the social sciences, one of its main roles has been to position administrative science in relation to neighboring fields and to tackle intra-disciplinary tensions.

In the version we purport to reconstruct here, this disciplinary narrative, or at least the part of it directly relevant to public administration (PA) education, identifies administrative science as exceptional in a number of respects. The first is related to PA education’s service to a national system of institutions. The latter has a specific institutional and organizational structure, with a particular history and intellectual tradition. This seems to lend PA an inherently parochial dimension relative to other fields of study in higher education (HE) (e.g., Kickert, 2008). It also gives rise to arguments that, because one of the roles of PA education is to support a national bureaucracy, a country’s PA programs would or—more prescriptively sometimes—should converge towards one integrated model (Duggett, 2003; Hajnal, 2016).

Conversely, globalization and Europeanization—the latter’s impact on the relevant national administrations being more direct—have spawned the opposite concern: that homespun PA education too often ignores other systems or international developments (Farazmand, 2002; Hajnal, 2003; Geva-May et al., 2008; Brans and Coenen, 2016; Manoharan et al., 2017). One result has been calls for the convergence towards a mainstream variety of models of educating and training public servants. In recent times, this model has been frequently, though not necessarily, identified as close to the Anglo-American approaches (Staroňová and Gajduschek, 2016).

Thirdly, the disciplinary narrative highlights a tension at the core of PA schools’ mission to train civil servants for national or local bureaucracies. The latter are in a state of flux generated by public sector reforms, which institutionalize changing conceptions of the public sector and frequently come with their specific (public) “management fads” (Abrahamson, 1996; Hendriks and Tops, 2003; Goldfinch, 2009; Birnbaum, 2000). PA schools and programs are—at least in theory—expected not only to respond to, but also to effectively sustain and support such external changes in vision, norms, and practices (Duggett, 2003; Osborne and McLaughlin, 2008; Stout, 2018). This has occasionally led to intense introspection concerning the tensions inherent in PA departments’ training and service roles, on the one hand, and their critical and research functions, on the other (e.g., van der Meer and Marks, 2016; Fenwick, 2018).

Lastly, the disciplinary narrative highlights administrative science’s multi-disciplinary nature. The latter draws its main theories and methods from other established fields of science—primarily political studies, law, sociology, and economics—and, in fact, is frequently taught as part of the latter (Randma-Liiv and Connaughton, 2005). Consequently, PA education blends in its curricula disciplinary perspectives that often espouse different philosophies and contrasting emphases. This signals the potential for locally tethering PA education to one of the four fields of study to the detriment of the others. It also suggests an inescapable diversity across the discipline. Some analyses thus stress the absence of any “strong convergence among [masters of public policy or PA] programs to either a ‘western’ or United States (US) model, or more neutrally…to any given ‘type’” (Pal and Clark, 2016: 294).

The disciplinary narrative briefly outlined above entails, as all of its type, radical simplifications. Key questions in PA education—concerning teaching methods, learning outcomes, the place of ethics and values, to mention just a few—have been left out of the sketch. Doubtlessly, many important nuances get lost in the broad brush. Nevertheless, the narrative is of interest to us here because, when such accounts gather foothold in a profession, they shape the professionals’ broad image of their field. In the case at hand, two consequences of the narrative are relevant for our purposes, which are ultimately to explain the institutionalization of PA education in post-communist Central and Eastern Europe (CEE).

The first is a fundamental preoccupation with variety and convergence across the field—of paradigms, perspectives, professional roles, practices, and of training and academic programs. This concern, however, is often defined in rather narrow disciplinary terms, as we shall see. Secondly, the narrative privileges some explanations, at the expense of others, of why and how the field evolves. Specifically, it lays a strong emphasis on the connection between national administrative structures/culture and PA education. In our view, this has happened with scholarship on the growth of PA higher education in post-communist CEE generally, and in Romania specifically.

In this article, we set out to answer two questions related to the points above. We see the questions as connected because they both illustrate partial blind spots in PA education literature that we associate to the disciplinary narrative, as detailed in the following section. Our first question concerns the explanation for the protracted institutionalization of PA higher education in post-communist Romania. We approach this issue via an account of the growth of PA education in this country, focusing on the period after 1990. Our bird’s-eye account engages critically the traditional narrative, which essentially claims that the path of this growth followed post-communist developments in the structure of PA. In opposition to this perspective, which attributes developments in PA education to factors outside academia, we shift the emphasis on to drivers and actors within the higher education system. These include the struggle over academic diversification and the countervailing pressures towards uniformization. We argue that the lengthy institutionalization of PA higher education can be additionally explained against the background of the oft-invoked reform of Romania’s post-communist administrative state, through the agency of entrepreneurial academics hailing from non-legal disciplines who capitalized on system-wide opportunities.

In answering the first question, we take for granted the general description of the current state of PA higher education in Romania (a diversity of programs weakening the once-dominant legalism), while disputing the completeness of the original account by adding a new dimension (the internal dynamics in higher education). The article’s second question asks in what sense PA education in this country has, in fact, become more diverse. We suggest that, over recent years, the field may have grown less so—on some measures.

Within PA education literature, key concepts such as “convergence” and “diversity” are typically assessed in terms of the composition of curricula in programs offered across a nation (or several). This “scholastic” approach to diversity is related methodologically to the disciplinary narrative (as we argue below), but one artifact of this perspective is that it sometimes obscures patterns of homogeneity. This is, indeed, what we find in the second part of the article: although looking at curricula may suggest an increasing variety of programs, in other respects Romanian PA departments seem to be growing more alike. To establish this, we analyze the results of a performance ranking of PA departments that we conducted in late 2017. To gauge the diversity of PA education over the better part of the current decade, we compare the results with a previous (“official”) national ranking carried out around 7 years before. We conclude by speculating on the causes of our findings and on where this might lead the discipline in the coming years.²

Thus, the article makes two main contributions. First, it provides a complementary account to explanations attributing the development of PA higher education mainly to changes in national administrative structures and, ultimately, administrative culture. Second, it offers a broader engagement with PA education literature, suggesting that its prevalent interest in inner-oriented disciplinary questions, likely a reflection of the soul-searching pervasive in administrative science, comes with limitations that may be avoided by taking a complementary look at broader developments in higher education. In both respects, the Romanian case suggests ways of rethinking the current approach to the growth of PA higher education in CEE countries more generally—and perhaps beyond this region as well.

Explaining diversity and change in PA education in CEE

In this section we return to the disciplinary narrative to tease out some of its implications and particularities and to build our case that a complementary perspective on the development of PA higher education is in order, at least in CEE.

We have already mentioned the intense preoccupation in PA literature with diversity, change and convergence across the field, in good part a consequence of the self-introspective concern with questions of disciplinary parochialism and universality. The first point worth highlighting in this connection is that the disciplinary narrative has tended to underscore the existence of a plurality of models for PA education systems around the globe or across regions of interest. On the flipside, when national systems are compared to each other, they are usually treated as being relatively homogenous internally. This perspective appears quite intuitive given the variety of national administrative cultures, each of which is presumed to be—as the choice of the term “culture” itself attests to (Sewell, 2004)—somewhat uniform. Hence, for example, the arguments on the importance of studying the history of national PAs (Raadschelders et al., 2000).

Post-communist CEE countries, where PA education was completely remade after the demise of socialism, are often used to showcase the twin dimensions of global/regional variety and of national uniformity. On the one hand, some of these countries’ traditionally “legalistic” approach to PA education is set in contrast to the varieties in North-Western Europe or the US. On the other hand, the continuous remaking of CEE PAs since 1990 is said to have occasioned a search for, and sometimes a shift to new models of PA education, varying from one country to the next (Hajnal, 2003; Randma-Liiv and Connaughton, 2005; Hajnal, 2015; Staroňová and Gajduschek, 2016).

The second point is that the narrative on PA higher education usually relies on evidence gathered from qualitative and quantitative analyses of academic curricula (and infrequently of faculty’s disciplinary backgrounds) (e.g., Connaughton and Randma, 2002; Hajnal, 2003; Kickert, 2008; Hajnal, 2015; Brans and Coenen, 2016; He et al., 2016; Pal and Clark, 2016; Staroňová and Gajduschek, 2016). Especially in comparisons across countries, typologies or morphologies of schools or programs in PA usually boil down to an assessment of disciplinary identity, typically defined in terms of what is taught. Other potentially relevant factors are generally left out of comparative assessments, no doubt in part due to difficulties in finding high-quality, comparable data from multiple countries (for exceptions, see Sanabria-Pulido et al. (2016) or some of the literature on executive education). Such factors include, for example, the profiles of enrolling students, where the latter intern, the involvement of practitioners in teaching, the type of organizations or sectors where graduates go to work, the kind or field of outreach activities, and so on. Neither are PA departments or programs typically compared to others in neighboring disciplines (say, political science or business) to gauge the magnitude of internal differences against some external standard. As a result, the questions of diversity, convergence and change in PA education are most frequently assessed in terms of curricular composition alone, which also happens to be a dimension that is easily traced back to national administrative culture.

Such analyses of curricular design, at least in the form typically conducted, have a number of limitations. We discuss three below, and tackle two of them in the article. One limitation is general in nature; it will not be dealt with beyond this paragraph. Namely, the curriculum-based approach tends to de-emphasize content diversity within national systems by consigning PA education to ideal types such as “legalistic,” “public affairs,” or “corporate” (e.g., Hajnal, 2003). Although intuitive and doubtlessly useful for some comparative purposes, such labels and clusterings may occasionally obscure content variety among programs both in terms of the weight of the “main” approach (e.g., the share of law courses in law-oriented programs may differ substantially from one academic system to the next), and in terms of the overall disciplinary mix (e.g., two public policy-heavy programs might overall exhibit a relatively different disciplinary composition). Conversely, analysis in terms of ideal types based on scholastic content may also overstate the extent of variety among national models. A “corporatist” (management-oriented) and a “public affairs” national PA systems, for example, may actually share a lot in terms of what is taught and how, their different emphases notwithstanding. (Hajnal (2003) provides, in his annex, an illuminating discussion on using the country level as the unit of analysis in comparative studies of PA education systems.)

The second limitation of the focus on curricular structure is more important for our goals here: this approach often takes a nation’s PA higher education as a rather rigid expression of that nation’s administrative culture (Hajnal 2015: 97; Randma-Liiv and Connaughton, 2005). The ideal types of PA education are modelled after the “Napoleonic,” “Weberian,” “Anglo-Saxon” typology of administrative cultures (Pollitt and Bouckaert, 2011; Kickert, 2005). Not only do PA educational models (say, the legalist) unsurprisingly align with administrative ideal types (the Napoleonic), but swings in the former are usually interpreted as responses to alterations in the latter. In this approach, changes in PA education are ascribed primarily to forces outside higher education, such as the rise of a new administrative philosophy (say, the New Public Management) or the sweeping drivers of globalization or Europeanization. Occasionally, shifts between models of PA education are overtly characterized as—to quote one of the most important CEE researchers on the topic, summarizing the opinions of academic respondents—“all rooted outside the realm of PA higher education” (Hajnal, 2015: 108).

As scholars of higher education, we find this perspective somewhat baffling. A lot of what happens in PA higher education is certainly driven by outside forces, from broad societal trends in demographics and employment to changes in administrative frameworks and practices. However, in higher education—as in other fields—change happens or fails to happen, to a considerable degree, through the agency of organizational and individual actors responding to field-specific opportunities and incentives (Mahoney and Thelen, 2010; McAdam and Fligstein, 2012). The proximate drivers of change and stasis have to be sought within the field as well, not only outside of it. This seems a particularly relevant perspective in fields undergoing a relatively rapid de-structuration and re-institutionalization, such as PA higher education in post-communist CEE.

We tackle this issue in the following section, where our account of the growth of Romanian PA higher education emphasizes the behavior of key groups of academics acting in an environment experiencing re-structuration in the 1990s and after. This country is interesting as a case study because, although after the fall of socialism it started as a textbook “legalist” example, it gradually moved away from this model.

The third limitation of the curriculum-based viewpoint concerns its conception of diversity, convergence, and change. Undoubtedly, looking at PA education systems from the scholastic perspective comes naturally for an academic field supposedly tormented by a “narcissistic” “self-obsession” (Fenwick, 2018: 10): developments are assessed in terms of disciplinary perspectives, affiliations, and positionings. Although useful for some purposes, it remains unclear what the curricular differences in the ideal types actually amount to in practice. What are the actual effects of teaching PA “legalistically” rather than “corporately,” for instance?

Defining diversity and convergence scholastically seems to assume that if programs offer significantly different (whatever “significantly” may mean for practical purposes) curricular menus, they also generate substantially different outcomes or impacts. Yet few, if any, relevant studies actually show—or even purport to demonstrate—this beyond purely theoretical considerations (e.g., Broucker, 2015). We do not have a clear picture, for example, of the extent to which the types of programs discussed above differ meaningfully in terms of the actual core competencies they instill (Haupt et al., 2017). Much less do we have a firm, non-anecdotal sense of whether, in a particular country, a legalistic program, as opposed to a corporate or a policy-oriented one, would better match their graduates to various roles in the bureaucracy. In this sense, comparing national systems of PA education in terms of curricular composition, or assessing one particular system’s development over time in the same terms, may be a useful exercise in tracing boundaries and delimitations within the discipline, but so far has shown a limited relevance beyond this objective.

We deal with this issue in the third section of the article, where we look at diversity in Romanian PA higher education not from the typical curricular perspective, but in terms of other measures relevant for higher education. We find that although the field may have grown more diverse curriculum-wise, as shown by various studies referenced herein, in other respects the picture appears to be different.

The protracted institutionalization of PA higher education in post-communist Romania

Romania experienced a dramatic change in state institutions over the last three decades, transitioning from a communist satellite state to a democratic member of NATO and the European Union (EU). The changes entailed radical reforms that both determined and were driven by the transformation of PA. This, in turn, affected the course of PA higher education.

Ever since the change of political regime in the final days of 1989, Romanian PA education has been undergoing a process of institutionalization that mirrors the experience of other CEE countries (Hajnal, 2003; Staroňová and Gajduschek, 2016). In a nutshell, this process entailed the establishment and abandonment of numerous and, for a while, varied PA academic programs, the creation of organizational units delivering such programs (i.e., departments or schools [“faculties”] of administrative science), and the growth of normative frameworks involving accreditation, staff evaluation, curricular development, or organizational performance. As far as academic program design is concerned, as in a few other CEE countries, PA education has been gradually but consistently moving away from a law-centered model towards a more balanced mixture. Despite increasingly stable arrangements, this process of institutionalization is arguably still not complete in Romania.

Since its inception a century ago, the organization of PA education in this country has been strongly influenced by the so-called “continental” approach, in particular its “Napoleonic” variety (Pollitt and Bouckaert, 2011). The professor who pioneered modern administrative and constitutional law in Romania, Paul Negulescu, established the Institute for Administrative Sciences (ISAR) in 1925. The latter, in turn, became a founding member of the Brussels-based International Institute for Administrative Law. This affiliation reflected modern Romania’s legal and administrative system, designed and subsequently re-organized from French principles and on the French model (Dincă, 2012). The influence of the legalistic outlook in PA education persisted through communist times.

Although ISAR was dismantled after the changes of political regime following the Second World War, administrative law remained a field of study in socialist-era law schools. The University of Bucharest’s Faculty of Law maintained a Department of Administrative Law. The other flagship, Babeş-Bolyai University in Cluj, offered a degree in Economic and Administrative Law in its Sibiu branch starting in 1971 (Sora, 2011). These legal specializations aside, it was the “executive” programs for party cadres that arguably represented the most important form of administrative education during communism. After a number of successive organizational transformations and mergers spanning over two decades, the Ştefan Gheorghiu Academy (SGA) came into its final format in 1970.³ One of the more interesting, albeit shorter-lived, organizations that provided a type of executive education was the Centre for the Training of Enterprise Leaders (CEPECA), founded and funded under a United Nations Development Programme (UNDP) project between 1967 and 1971. It too was eventually absorbed by the SGA.

After the demise of state socialism, academic PA grew steadily during the first post-communist decade and explosively after 2000—much like, in fact, the rest of Romanian higher education (Andreescu et al., 2012; Proteasa et al., 2017). Economic and Administrative Law was established as a Bachelor of Arts (BA) program in the early 1990s, alongside short-term degrees in Secretarial Work for Local Administration and in Professional Communication (Sora, 2011: 56). However, the founding moment for PA education as we now know it was arguably 1995: three of the major organizational players were established in some form or other that year (see below). Not coincidentally, the first post-communist Law on Education was finally passed, after much delay, also in 1995. Representatives of PA programs and the Ministry of Education met that same year to craft a unified PA curriculum where law courses were “overwhelmingly” dominant (Hinţea and Ringsmuth, 2000). Although it was met with some resistance, the framework curriculum was nonetheless welcomed, as it bestowed on the new PA programs a mark of official recognition. The law-centered design was revised in a subsequent meeting in 1999. Proposed changes included coordination with programs in the EU, involving a more balanced disciplinary diet of law, administrative science, economics, and political studies. Out of the country’s around 90 universities, 17 offered a variety of PA programs in 2000 (Hinţea and Ringsmuth, 2000).

Seeking convergence with “European” programs implied, at the time, an effort to make PA education more diverse. One channel was foreign support for Romania’s fledgling democracy. In the late 1990s and until Romania became a member of the EU in 2007, the country received assistance from organizations such as the World Bank, the International Monetary Fund and the Organisation for Economic Co-operation and Development, or through the EU’s pre-accession assistance program PHARE (Cernat, 2006). PA reform and the dissemination of Western-style good practices were among the points agreed with the international organizations (Matei, 2009). The good practices ranged from public management techniques to, for example, the publication of academic works in administrative science, especially textbooks and handbooks (some of them still in use). In this context, traditionally law-centered PA education opened up to public policy and public management approaches.

The two paragraphs above recount the standard explanation of the move away from a law-based curriculum. However, although changes in PA curricula were certainly justified in terms of Western administrative good practices and the adjustment to “European standards,” in practice the transformation had more complex roots. The justifications were an expression not only of the desire to change Romania’s PA, or of the training and skills opportunities afforded by Western initiatives, but also of incentives internal to higher education actors. In particular, new organizational challengers emerged on the scene in 1995, upsetting the previous law school-dominated regime. Relevantly, the three key emerging players all originated from non-legal disciplines and departments.

The National School of Political Studies and Public Administration (SNSPA) in Bucharest, arguably born out of the ashes of the communist-era SGA, was re-organized in 1995 as a specialist public university dedicated to the social sciences. Its Department of Public Administration offered a 4-year (then the standard) undergraduate degree, as well as short-term (3-year) “college” programs in important urban centers.⁴ Several of the key individual academics in charge with PA programs at this university hailed from the area of public sector management rather than law. Also in 1995 and in Bucharest, the Academy of Economic Sciences, a large higher education institution offering a variety of economics and business programs, signaled its entry into the PA education arena. It first established a PA program within the Faculty of Management. Finally, the same year saw the founding of a PA program at the Babeş-Bolyai University in Cluj, this time within the Faculty of Political Science.

Gradually, robust PA departments grew out of these three young programs. Some subsequently separated from the original faculties into the leading PA schools. They were joined by others later, such as the University of Bucharest, with a new faculty offering business and PA programs.

The important point to take home from this hurried review of the mid-1990s is that each of the three fledgling actors grew around organizational units not associated with legal disciplines. They challenged the law-centered curricula precisely because PA programs were, at the time, the almost-exclusive province of law schools. They benefited from financial assistance and training offered by the various Western initiatives mentioned previously, as well as the US Information Agency and the EU’s Tempus (Hinţea and Şandor, 2002). Sure enough, they legitimized their challenge to law-centered PA education in terms of trends and practices in Western administrative cultures. However, the discursive strategy should not be mistaken for the notion that these programs simply arose out of or reflected actual changes in Romanian administrative culture. In fact, the disconnect between a degree in PA and a job in the public sector remains one of the system’s major problems and is openly acknowledged even at the late date of this writing (Hinţea, 2013: 295; Hinţea et al., 2006). Rather, the entrepreneurial academics who eventually managed to turn their programs and schools into the country’s largest seized an opportunity to change the course of the ongoing re-institutionalization of PA academic education.

Taking a broader, system-level view, the pull towards diversity in higher education was hardly limited to the particular area of PA, or stronger here relative to other fields of study. The growth described above was part and parcel of a massive expansion in both public and private higher education, with educational and curricular experiments being undertaken in and around many disciplines in the 1990s. This phase of “anarchic” differentiation (Miroiu, 1998) in higher education gave birth, in time, to a push towards homogeneity. Throughout this re-institutionalization, the PA education arena was subject to a variety of normative and mimetic isomorphic forces (Andreescu et al., 2012; Proteasa et al., 2017). Accreditation in the 1990s initially consolidated the law-heavy curriculum in PA. Later on, the country’s main accreditation (re-christened “quality assurance”) body took on board the moderate re-orientation away from legalism, while imposing on the PA field an updated but still uniform framework and standards. The Bologna Process eliminated definitively the “collegiate” (i.e., short) degrees in PA (and elsewhere). Isomorphic pressures were also exercised by a hiring and promotion system that designed and enforced national minimal standards. The introduction of university classifications and program rankings in 2011 eventually added to the pressures for uniformization. (We return to this subject in the next section.)

Despite these forces pushing in the direction of a more homogeneous PA higher education, in Romania the field remains still relatively diverse by the measure of curricular structures (Hinţea, 2013; Hajnal, 2015). However, this is far from a clear reflection of Romania’s changing administrative culture. Instead, these developments were possible because the field of PA education was progressively staked out by academics who made a claim to the discipline from within the non-legal academic areas of political studies, public management or business administration. The challengers succeeded not only because new approaches and techniques were being introduced in the country’s PA (ironically, they remain somewhat marginal to this day), even though they certainly legitimized curricular and organizational changes as part of the ample process of “Europeanization.” Rather, it was the diversification within academia, for a while bordering on the chaotic, and the subsequent struggle over standardization that enabled PA to establish and then maintain its diversity of programs.

Diversity and the ranking of Romanian PA departments

As noted previously, looking at the curricula of Romanian PA programs suggests that, although legal subjects still enjoy occasional prominence, there has been an opening towards public policy and management (Hinţea, 2013; Hajnal, 2015). This has been considered a marker of diversity within the system—as long as diversity is conceptualized primarily in disciplinary terms (curriculum or affiliation to schools defined by discipline). But one can think of diversity differently, for example in terms of the spread of programs’ academic performance defined in a broad sense. We find this question relevant because, as argued above, the effects of programs’ curricular composition on the outcomes of PA education remain unclear—and are rarely explored empirically to begin with. In CEE, but also beyond this region, we know little about what the neat distinctions between legalistic, corporate, and public affairs models entail beyond positioning programs within a somewhat contrived disciplinary space.

Ideally, one would complement the scholastic approach to diversity with measures of outcomes or impact relevant for PA education and its goals. We do not have such measures available, for the same reason they are hardly used at all—the difficulty of collecting good data and the paucity of secondary data (Powell and Saint-Germain, 2016). As a result, we try a different and admittedly inferior tack: we contrast the finding of curricular diversity with the results of a comparison of performance rankings of Romanian PA departments conducted in 2011 and the end of 2017. The comparison enables us to argue that, scholastic variety notwithstanding, in another respect the departments have become increasingly similar in the intervening period.

The next subsection introduces the context of our exercise, whereas the one following it touches on some methodological issues. (The bulk of the methodological discussion and of limitations is reserved for the annex available online.) We then present the results and draw the implications.

Ranking Romanian universities and programs

The Romanian government conducted its first national classification of universities and rankings of study fields in 2011, under a new Law on National Education adopted the same year (Păunescu et al., 2011).⁵ The declared rationale for the classification and rankings was twofold. The first objective was, in line with the government’s new language on public sector reform, to improve accountability and quality. Secondly, the classification and the rankings were overtly justified in terms of gauging the diversity of universities (to be categorized into “research oriented,” “research and teaching” and “teaching oriented”) and, respectively, of academic programs (grouped in five classes, A through E). The policy-makers’ promise was to subsequently allocate funds to the best universities in each class and to the highest-ranking programs in each field. In practice, the performance-focused rhetoric drowned out the diversity-based arguments and generated controversy and a wave of contestation. Though mandated under the law as a periodic exercise to be conducted every few years, the classification and rankings stalled immediately after the first round of results, as the process was hobbled in the courts.

One effect was a debate among academics and other experts concerning the very point of such endeavors, in Romania and generally. More sophisticated efforts at the European level (e.g., van Vught, 2009) also oriented attention towards these topics. Furthermore, criticisms directed at the classification and ranking methodologies prompted native researchers to advance alternative approaches. Thus, Vîiu et al. (2012) and Păunescu and Hâncean (2013) carried out rankings of political science and international relations and, respectively, sociology departments using an independently developed methodology (the same in both cases). Their more streamlined and practicable approach largely confirmed the outcomes of the complex national exercise, which used a much larger number of indicators in a convoluted design (Andreescu et al., 2015).

In this paper, we adopt and tweak the simplified methodology to rank PA departments. We compare the results to those of the national rankings of 2011 for the field of PA to reflect on changes in the landscape over the intervening period of around 7 years. In particular, we explore whether the departments have spread further apart or clustered closer together in terms of their ranking class relative to 2011.⁶

Method and results

This subsection briefly outlines the structure of the performance index. In the online methodological annex there are additional details concerning the meaning of indicators, data collection procedures, a few tweaks to the original approach, and a brief discussion of methodological limitations.

To rank the PA departments, we slightly adapted the method developed by Vîiu et al. (2012), which is convenient in context for a number of reasons. First, it was already employed on an additional occasion (Păunescu and Hâncean, 2013) and proved feasible and reliable. The data can be, in theory at least (our experience proved more complicated), collected with relative ease, as it is part of mandatory reporting by Romanian universities to a variety of public agencies. Second, the method was used successfully in fields neighboring or partly overlapping with administration science (political science and sociology). Third, the ranking method relies on performance indicators whose meaning and relevance are clear in the domestic context.

The index represents a weighted sum of two dimensions—Research and Education (Table 1). Each dimension consists of several indicators. To compute the index, the weighted individual indicators for each dimension are added then the resulting values (the weighted dimension scores) are summed up. We have kept the weights used in the articles mentioned above, which justified and validated the methodology vis-à-vis the national exercise.

Table 1.

Dimensions, indicators, and associated weights in the performance ranking.

Dimensions	Indicators	Individual indicator weights	Dimension weights
Research	Successive departmental g-score	0.77	0.50
	Ratio of no. of research projects to full-time department staff	0.10
	Ratio of PhD students to total students	0.13
Education	Department size index	0.40	0.50
	Teaching staff to student ratio	0.20
	Ratio of graduate to undergraduate students	0.10
	Ratio of ERASMUS students to total students	0.10
	Quality assurance “confidence score” for study programs (based on evaluation by ARACIS)	0.10
	Quality assurance “confidence score” for the higher education institution (based on evaluation by ARACIS)	0.10

Source: Adapted from Vîiu, Vlăsceanu and Miroiu (2012)

ERASMUS = european academic exchange program; ARACIS = national accreditation and quality assurance agency.

We included in our rankings departments offering PA programs (BA and Master of Arts (MA)) from 23 universities (see the first subsection of the methodological annex for details about the sample) (Table 2). The figure implies that only about two-thirds (64%) of the higher education institutions currently offering PA programs are featured in our performance ranking. Nevertheless, every university—arguably with one exception—with a significant presence in PA education is on the list.

Table 2.

Final scores and classes in the 2017 PA department rankings compared to the 2011 rankings.

PA department (university name)	Score	Score after normalization	2017 class	2011 class	2011 vs 2017
Academy of Economic Studies (ASE)	0.704537	1	A	A	-
Timişoara Polytechnic University	0.5444	0.772705	A	C	+2
National School of Political and Administrative Studies (SNSPA)	0.529051	0.750919	A	A	-
Alexandru Ioan Cuza University in Iaşi (UAIC)	0.5064	0.71877	B	C	+1
University of Bucharest	0.490515	0.696223	B	B	-
Babeş-Bolyai University in Cluj	0.448751	0.666095	B	A	-1
University of Piteşti	0.446826	0.634212	B	D	+2
Ştefan cel Mare University in Suceava (USV)	0.431099	0.61189	B	B	-
Aurel Vlaicu University in Arad (UAV)	0.39541	0.561233	B	D	+2
Valahia University in Târgovişte	0.394994	0.560644	B	C	+1
1 Decembrie 1918 University in Alba Iulia	0.356098	0.505435	B	E	+3
University of Oradea	0.354473	0.503129	B	D	+2
Dunărea de Jos University in Galaţi (DJUG)	0.340616	0.48346	C	B	-1
Petru Maior University in Târgu Mureş	0.340145	0.482793	C	C	-
University of Petroşani	0.332398	0.471796	C	C	-
West University of Timişoara	0.331361	0.470324	C	C	-
Eftimie Murgu University in Reşiţa	0.329303	0.467403	C	C	-
Constantin Brâncuşi University in Târgu Jiu	0.322969	0.458413	C	C	-
Ovidius University in Constanţa	0.320826	0.455371	C	C	-
University of Craiova	0.304239	0.431829	C	C	-
Petroleum-Gas University in Ploieşti	0.287653	0.408287	C	C	-
Nicolae Bălcescu Land Forces Academy	0.26482	0.375878	C	A	-2
Andrei Şaguna University in Constanţa	0.234868	0.333365	C	E	+2

Source: Own calculations (data set available on authors’ ResearchGate profiles). Results are presented as final index scores (the index is sub-unitary because all individual indicator scores were normalized, then weighted and added). The final scores were then relativized to the highest score to identify the occupants of classes on the scale from A to E used in the 2011 national rankings. For purposes of comparability, we used the latter’s classes and thresholds (although they were not justified anywhere in the methodology and may seem arbitrary). The current class is shown next to the 2011 class. The final column indicates where the scores moved up (+), down (-), and by how many positions.

PA = public administration.

Discussion

There are a couple of notable points in this side-by-side comparison of PA departments in 2017 and 2011.

First, PA departments have become more similar in terms of general academic performance. Categories D and E are completely absent in 2017. All departments at the bottom end of the 2011 ranking (in D and E) are now in classes B or C. This suggests the two bottom classes have now become irrelevant and the hierarchy has flattened. Although this result is based on a subset of PA departments and probably biased towards the more transparent ones, the sample is varied enough, as it contains six formerly D and E cases, and index scores across the sample range between 0.23 and 0.70.

We interpret this result as an indication that, whatever the “curricular identity” of the individual PA departments, the latter have become more similar on this performance measure. In our view, the most likely explanation is the pressures for standardization mentioned previously in this article. Currently, there is a single national curricular framework for PA programs (albeit one that is friendly to variety, relatively speaking), one set of program evaluation standards with respect to human resource capacity (covering things such as the share of full-time faculty in the programs or teacher-student ratios), and a single set of national performance standards for individual faculty. Under these circumstances, programs have been gradually moving towards a preferred template, as the rankings comparison suggests. We expect this trend to continue under the current standards.

Second, research performance represents the real differentiator among PA departments. Notwithstanding the increasing similarity of PA departments, the “research score”—which in our context is primarily a function of the scientific productivity of individual academics (the g-index)—remains the single relatively potent differentiating factor. Most likely, this is so because the index is harder to inflate rapidly by comparison with curricular or human resource standards. In fact, if the research score, which enjoys a substantial weight (50%) in the total ranking score, had received a smaller weighting (25%), all departments would be classed as either A or B, rendering the distribution flatter still.

We predict that, because research performance represents the main differentiator among departments, as well as due to the strong individual incentives to publish (virtually the only condition to access the titles of associate and full professor), increasingly more emphasis will be placed on publication prowess. Anecdotally speaking, this happened between 2011 and 2017. There is better evidence of this in neighboring fields. A recent study comparing research performance (also based on the departmental g-index) in political science across the same period (2012–2017) shows that all departments and most faculty have stepped up publication and the distance among departments in terms of publication output has been reduced (Miroiu et al., 2018).

This raises a number of interesting issues, which cannot be explored in detail here but are nevertheless worth mentioning. One is the gaming of performance indicators, whether through shady practices (Zulean and Şercan, 2018), or simply as a matter of quasi-official department policy. While conducting these rankings, we noticed in a number of instances what appeared to be co-publication and mutual citation networks designed to inflate individual and departmental index scores. Universities and departments are also busily exploring ways to temporarily affiliate—for lack of a better term—outside researchers for purposes of increasing publication count. Few events command the attention of universities and departments (and of the academic rumor mill) more than the issuance of the newest rounds of international rankings.

All of this is, of course, hardly limited to schools of administrative science, but it may have specific implications here. One concerns the directions wherein individual academics and their departments choose to invest resources. If money and effort are substantially re-oriented towards academic research and particularly towards publication in academic journals, less will be directed to other goals. Effects on PA education are worth monitoring because of the field’s complex mission, which in theory involves an applied focus as well as service to communities and advice to policy makers. Incentivizing academics and departments to invest most of their efforts in academic publishing will upset the proper balance between PA faculty’s orientation towards research and, respectively, service goals. This phenomenon is well known throughout academia (Boyer, 1990; Arum and Roksa, 2011), but in PA education it is particularly relevant given the field’s particularities. Standards for faculty hiring and promotion in PA, which in this country are set (nationally) to be the same as those across the other social sciences, and therefore fail to give due consideration to the field’s peculiar mission (van der Meer and Marks, 2016), will likely render faculty in PA departments more alike those in neighboring fields. In other words, activities aimed at boosting research performance will progressively crowd out engagement and outreach. Eventually, research-focused PA departments will increasingly resemble not only each other, as our rankings comparison suggests, but also, say, research-focused political science departments.

To sum up this section, the performance rankings-based exercise outlined above suggests that, on this admittedly limited measure, PA departments are becoming less differentiated. This is plausibly a natural consequence of the isomorphic factors (DiMaggio and Powell, 1983; Andreescu et al., 2012), normative as well as mimetic, at work in Romanian higher education more generally, and in PA education particularly. Once again, we believe this finding is useful as a complement to the common account of curricular variety in PA education.

Conclusions

One way to frame our general argument in this article is as follows: although somewhat narrowly constructed disciplinary questions—the emphasis on specific “traditions” or paradigms, the relationship to neighboring fields of study, etc.—are doubtlessly essential to understanding the shape and growth of PA education, forces operating in the broader field of higher education should not be neglected. In making this point, we are certainly not arguing that PA education literature has consistently ignored the latter while focusing exclusively on the former. Rather, our claim is that in countries—such as those in CEE—that have experienced a swift de-structuration and re-institutionalization of higher education as well as of the state bureaucracy, the focus on disciplinary questions has resulted in a partial blind spot. The dominant account of change in PA education in this region therefore needs to be enhanced with complementary perspectives taking into account developments beyond the change in administrative culture or structures, in particular those in the broader field of higher education itself.

Thus, we have argued that the prevalent narrative, tying the growth of CEE PA higher education to the evolving dynamics of its administration, conceals the role of opportunities uniquely arising in the field of higher education (in the Romanian case, diversification followed by standardization). Without the massive expansion and experimentation in academia in the 1990s, it is unlikely that PA education would have so vigorously thrown off the yoke of the legalist approach. Absent the national standardization that followed, the field would probably not have been able to consolidate its current identity, which conserves variety while also clearly demarcating PA education from law or political science. In an alternative scenario where diversification had not been doubled by top-down efforts at uniformization, PA education might have been re-absorbed by the faculties of law, political studies or management from which it originally struggled to distinguish itself.

These points highlight another important matter: the prevalent narrative has also tended to marginalize the agency of academic challengers capitalizing on the opportunities described above. The focus on changes in post-communist administrative structures and culture obscured the extent to which a few small groups of entrepreneurial academics acting at critical junctures—such as the passing of the new law on education in 1995 or the efforts to design and then reshuffle the accreditation system—were able to successfully set up and then grow a system of PA education. This, in turn, suggests that in Romania—and the CEE more generally—the prevalent account also overstates the link between PA higher education and the PA sector. Though the former obviously did not develop independently of the latter, neither did it systematically follow in its footsteps. The coupling was considerably looser. PA education sometimes acted anticipatively (i.e., academics designing textbooks and guidelines and training programs for public servants and an administration still too traditional for these approaches), but otherwise remained relatively shielded from developments in the country’s PA.

As a result of this gap between PA education and the changes in the post-communist administrative state, the former was able to progressively institutionalize while remaining relatively diverse. But other developments in higher education, to which we call attention in the article’s second main section, cast the prevalent account of the diversity of PA education in a somewhat different light. The intensifying pressures on individual academics to publish and on administrative science departments to justify their quality almost exclusively in terms of research prowess are making the latter grow more alike in specific respects. If they become too focused on the same metrics and grow increasingly similar, the field as a whole may come to suffer from a uniform neglect of students and outreach or engagement activities. In the end, and perhaps somewhat counterintuitively, such isomorphic forces could ultimately matter more than the scholastic diversity of departments. Curricular diversity may, in fact, turn out to be a mainly academic concern—in both senses of this term: a mostly abstract pursuit devoid of practical impact on the public sector; and one undertaken by academics excessively preoccupied with positioning themselves with respect to neighboring disciplines.

Supplemental material

Supplemental Material, REV_PAE_Romania_ANNEX_R2 - On the re-institutionalization and diversification of Public Administration education in Central and Eastern Europe: A case study of post-communist Romania

Supplemental Material, REV_PAE_Romania_ANNEX_R2 for On the re-institutionalization and diversification of Public Administration education in Central and Eastern Europe: A case study of post-communist Romania by Liviu Andreescu, Marian Zulean and David Diaconu in Teaching Public Administration

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Liviu Andreescu

Notes

References

Abrahamson

(1996) Management fashion. Academy of Management Review 21: 254–285.

Andreescu

Gheorghiu

Proteasa

, et al. (2012) Institutional diversification and homogeneity in Romanian higher education: The larger picture. In: Curaj

, et al. (eds) European Higher Education at the Crossroads Between the Bologna Process and National Reforms. Dordrecht: Springer, pp. 863–885.

Andreescu

Gheorghiu

Irimia

, et al. (2015) Mergers and classifications in Romania: Opportunities and obstacles. In: Curaj

Georghiou

Harper

, et al. (eds) Mergers and Alliances in Higher Education: International Practice and Emerging Opportunities. Dordrecht: Springer, pp. 36–62.

Arum

Roksa

(2011) Academically adrift: Limited learning on college campuses. Chicago and London: The University of Chicago Press.

Birnbaum

(2000) Management fads in higher education: Where they come from, what they do, why they fail. San Francisco: Jossey-Bass.

Boyer

(1990) Scholarship reconsidered: Priorities of the professoriate. San Francisco: Jossey-Bass.

Brans

Coenen

(2016) The Europeanization of public administration teaching. Policy and Society 35(4): 333–349.

Broucker

(2015) Defining the impact of public administration programmes for public sector organizations. Teaching Public Administration 33(2): 193–207.

Cernat

(2006) Europeanization, varieties of capitalism, and economic performance in Central and Eastern Europe. Basingstoke: Palgrave Macmillan.

10.

Connaughton

Randma

(2002) Teaching ideas and principles of public administration: Is it possible to achieve a common European perspective? EPAN Fifth Annual Conference, Granada, Spain 14-15 2002, available at https://www.researchgate.net/profile/Bernadette_Connaughton/publication/265074823_Teaching_Ideas_and_Principles_of_Public_Administration_is_it_possible_to_achieve_a_common_European_perspective/links/55eda9d708ae199d47bd688f.pdf

11.

DiMaggio

Powell

(1983) The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48: 147–160.

12.

Dincă

(2012) Sistemul administrativ românesc: inspiraţie franceză şi adaptare autohtonă. Bucureşti: Ed. Economică.

13.

Duggett

(2003) The British Civil Service College in comparative perspective. In: Rosenbaum

Gajdošová

(eds) State modernization and decentralization: Implications for education and training in public administration. Slovakia: NISPAcee, pp. 100–108.

14.

Egghe

(2006) Theory and practice of the g-index. Scientometrics 69: 131–152.

15.

Ellwood

(2008) Challenges to public policy and public management education. Journal of Policy Analysis and Management 27(1): 172–187.

16.

Farazmand

(2002) Privatization and globalization: A critical analysis with implications for public management education and training. International Review of Administrative Sciences 68(3): 355–371.

17.

Fenwick

(2018) Teaching public administration: Key themes 1996–2016. Teaching Public Administration 36(1): 6–13.

18.

Fowles

Frederickson

Koppell

JGS

(2016) University rankings: Evidence and a conceptual framework. Public Administration Review 76(5): 790–803.

19.

Geva-May

Nasi

Turrini

, et al. (2008) MPP programs emerging around the world. Journal of Policy Analysis and Management 27: 187–204.

20.

Goldfinch

(2009) Dangerous enthusiasms and information systems development in the public sector. In: Goldfinch

Wallis

(eds) International Handbook of Public Management Reform. Cheltenham: Edward Elgar, pp. 121–136.

21.

Hajnal

(2003) Diversity and convergence: A quantitative analysis of European public administration education programs. Journal of Public Affairs Education 4: 245–258.

22.

Hajnal

(2015) Public administration education in Europe: Continuity or reorientation? Teaching Public Administration 33(2): 95–114.

23.

Hajnal

(2016) Illiberal or simply unorthodox? Public administration education in Hungary: A comparative perspective. Teaching Public Administration 34(2): 206–225.

24.

Harzing

(2007) Publish or perish. Available at: http://www.harzing.com/pop.htm

25.

Haupt

Kapucu

(2017) Core competencies in Master of Public Administration Programs: Perspectives from local government managers. Journal of Public Affairs Education 23(1): 611–624.

26.

Lai

(2016) Teaching policy analysis in China and the United States: Implications for curriculum design of public policy programs. Policy and Society 35(4): 385–396.

27.

Hendriks

Tops

(2003) Local public management reforms in the Netherlands: Fads, fashions, and winds of change. Public Administration 81(2): 301–323.

28.

Hinţea

Ringsmuth

(2000) Public Administration Education in Romania. Available at: http://unpan1.un.org/intradoc/groups/public/documents/NISPAcee/UNPAN006438.pdf, (accessed 8 April 2018).

29.

Hinţea

Şandor

(2002) Impactul cooperărilor internaţionale asupra dezvoltării programelor universitare de administraţie publică în România. Studiu de caz: Universitatea Babeş-Bolyai. Revista Transilvană de Ştiinţe Administrative 8: 35–43.

30.

Hinţea

Ringsmuth

Mora

(2006) The reform of the higher education public administration programs in the context of public administration reform in Romania. Transylvanian Review of Administrative Sciences 2(16): 40–46.

31.

Călin

Hinţea

(2013) Public administration schools in Romania: Strategic choices for the future. Revista de Cercetare şi Intervenţie Socială 42: 294–309.

32.

Kickert

WJM

(2005) Distinctiveness in the study of public management in Europe: A historical-institutional analysis of France, Germany and Italy. Public Management Review 7(4): 537–563.

33.

Kickert

WJM

(ed) (2008) The study of public management in Europe and the US: A comparative analysis of national distinctiveness. New York: Routledge.

34.

Lim

(2018) The building of weak expertise: The work of global university rankers. Higher Education 75: 415–430.

35.

Mahoney

Thelen

(2010) A theory of gradual institutional change. In: Mahoney

Thelen

(eds) Explaining institutional change: Ambiguity, agency, and power. Cambridge: Cambridge University Press, pp. 1–37.

36.

Manoharan

Mirbel

Carrizales

(2017) Global comparative public administration. Teaching Public Administration 36(1): 34–49.

37.

McAdam

Fligstein

(2012) A Theory of Fields. New York: Oxford University Press.

38.

Marginson

(2007) Global university rankings: implications in general and for Australia. Journal of Higher Education Policy and Management 29(2): 131–142.

39.

Matei

(2009) Romanian public management reform. Vol. I: Administration and Public Services. Bucharest: Editura Economica.

40.

Miroiu

(ed) 1998. Invatamantul romanesc azi. Iasi: Polirom.

41.

Miroiu

Diaconu

Dumitru

(2018) Cercetarea în ştiinţele sociale din România: Între vocaţia excelenţei şi provocările instituţionale. In Frangopol

Banabic

David

(eds) Educaţia şi cercetarea Românească. Starea prezentă şi perspectiva. Cluj-Napoca: Casa cărţii de ştiinţă, pp. 173–190.

42.

Osborne

McLaughlin

(2008) The study of public management in Great Britain. Public service delivery and its management. In Kickert

(ed) The Study of Public Management in Europe and the US: A comparative analysis of national distinctiveness. New York: Routledge, pp. 70–98.

43.

Pal

Clark

(2016) Teaching public policy: Global convergence or difference? Policy and Society 35(4): 283–297.

44.

Păunescu

Vlăsceanu

Miroiu

(2011) Calitatea învăţământului superior din România: o analiză instituţională a tendinţelor actuale. Iaşi: Polirom.

45.

Păunescu

Hâncean

(2013) Ranking the Romanian departments of sociology: Comparative results of different evaluation methodologies. Quality Assurance Review for Higher Education 5(1–2): 5–17.

46.

Pollitt

Bouckaert

(2011) Public Management Reform: A Comparative Analysis—New Public Management, Governance, and the Neo-Weberian State, 3rd ed. Oxford: Oxford University Press.

47.

Proteasa

Andreescu

Curaj

(2017) Differentiation and academic control over policy in Central and Eastern Europe: The case of Romania. European Educational Research Journal 16(5): 605–625.

48.

Powell

Saint-Germain

(2016) Student learning outcome assessment in NASPAA Programs: A review of validity and reliability. Journal of Public Affairs Education 22(4): 507–514.

49.

Pusser

Marginson

(2013) University rankings in critical perspective. The Journal of Higher Education 84(4): 544–568.

50.

Raadschelders

JCN

Wagenaar

Rutgers

, et al. (2000) Against a study of the history of public administration: A manifesto. Administrative Theory & Praxis 22(4): 772–791.

51.

Randma-Liiv

Connaughton

(2005) Public Administration as a field of study: divergence or convergence in the light of ‘Europeanization’? TRAMES 9(4): 348–360.

52.

Tol

RSJ

(2008) A rational, successive g-index applied to economics departments in Ireland. Journal of Informetrics 2(2): 149–155.

53.

Sanabria-Pulido

Rubaii

Purón

(2016) Public affairs graduate education in Latin America: Emulation or identity? Policy and Society 35(4): 315–331.

54.

Sewell

(2004) The concept(s) of culture. In: Spiegel

(ed) Practicing History: New Directions in Historical Writing after the Linguistic Turn. London: Routledge, pp. 35–61.

55.

Sora

(2011) Evoluţia calificărilor din învăţământul universitar românesc, 1968–2011. Bucharest: National Agency for Qualifications (ANC).

56.

Staroňová

Gajduschek

(2016) Public administration education in CEE countries: Institutionalization of a discipline. Policy and Society 35(4): 351–370.

57.

Stout

(2018) The impossibility of a universal public service perspective. Teaching Public Administration 36(3): 222–236.

58.

UEFISCDI (Executive Agency for Funding Higher Education Research and Innovation) (2013) Raport asupra fluxurilor informaţionale ce privesc sistemul de învăţământ superior (SIS) din România. Higher Education Policy Series 7. Bucharest: UEFISCDI.

59.

van der Meer

F-B

Marks

(2016) Research and teaching PA: Towards research as teaching. Teaching Public Administration 34(1): 109–116.

60.

van Vught

(2009) Mapping the higher education landscape: Towards a European classification of higher education. Dordrecht: Springer.

61.

Vîiu

Vlăsceanu

Miroiu

(2012) Ranking political science departments: The case of Romania. Quality Assurance Review for Higher Education 4(2): 79–97.

62.

Zulean

Şercan

(2018) Democratic control of Romanian intelligence after three decades: Quis custodiet ipsos custodes? Defense & Security Analysis 34(4): 365–384.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB