Abstract
In this piece, the authors examine educational policy by focusing on the ways in which actors “play” or selectively follow, negotiate, and appropriate cultural instructions and rules. They outline a framework that situates assemblage, a notion utilized in actor-network theory, within the critical cultural study of policy. Treating policy assemblage as a dynamic cultural form, they argue, provides a way of revealing the complexities of sociomaterial connections inherent to policy implementation. The authors pay particular attention to what happens when disparate actors join together to perform policy-directed tasks. It is within these heterogeneous and hybrid linkages that policy negotiations and controversies can become productive play. The authors briefly discuss the dynamic composition of productive policy play. Then, applying it to a controversy revealed in the ethnographic analysis of No Child Left Behind conducted by the first author, they demonstrate the framework’s usefulness in considering the sociocultural processes of policy in action.
Much of life’s complexity lies in its temporal, incomplete, and always-becoming processes. Although anthropologists may be known outside the discipline as mostly concerned with enduring cultural patterns and their impact, current research and theory in the field is increasingly aimed at capturing dynamic phenomena. As Marcus and Saka (2006) state, contemporary cultural analysis aims to focus on “social process and cultural meaning on the ephemeral, the emergent, the evanescent, the decentered and the heterogeneous, all the while not giving up on a long-established commitment to account for the structured and systemic in social life” (p. 101). Similarly, McDermott and Varenne (2006) argue for “reconstructing culture in educational research” to highlight all the people and processes involved in making exacting problems real for certain people at particular times.
One way in which such sensibilities have intervened in empirical research on education policy is through notions of assemblage and, more recently, of play. Assemblage, a term often associated with actor-network theory (ANT), focuses analytic attention on how disparate material and discursive practices come together to form dynamic associations. The concern with play can be traced to symbolic anthropology (Boon, 1999; Geertz, 1973; Ortner, 1999). More recently, Varenne (2007) explored the implications of play in ways that link to Lévi-Strauss’s (1966) interest in “bricolage,” to de Certeau’s (1984) interest in the activities of the subjugated, to Gorur and Koyama’s (n.d.) exploration of productive sociotechnical controversies, and to McDermott’s interest in collusion (McDermott & Tylbor, 1983).
Assembling and linking policy associations are most often the products of “multiple agencies and multiple sites of discourse generation” (Ball & Exley, 2010, p. 151). These assemblages are deeply imprinted with the disparate aims, resources, and histories of actors brought together under the contingent nature of policy, as well as influenced by those who are not exactly linked. Historically, associations or assemblages appear, even in the absence of a consensus (or habitus), or of anyone’s specific intention, or of a rational democratic selection process among competing goods. Assemblages are thus, as noted by Gorur (n.d.), “neither inevitable nor inescapable,” but always partial and multiply, if temporarily, cohering.
Assemblage thinking moves material (often textual) objects from positions of passive artifacts to ones of cultural mediators and leads us to consider how things that may not be quite human, in the phenomenal sense, affect current and further practice. An assemblage, once emerged in history, becomes temporarily an object or a “thing,” if not a “fact,” that requires future actors to respond to it “as if” it were an agent requiring particular responses. It is thus always possible to bring objects, including assemblage, “back to light” (Latour, 2005, p. 81), back to bear upon a current situation. As Latour (2005) famously put it, “Things have agency” (p. 72). Practically this means that, when confronted by an object, whether natural, mechanical, or regulatory, actors must take it into account in terms of its overt properties, as well as unspecified latent properties that may emerge through their response to the object. Kicking a stone can hurt. Hurling a stone can kill. Defying a policy can be dangerous to one’s job. Old policies can be brought to light to transform current situations. In other words, as Latour also mentions, objective properties are not determinant (in a causal fashion); they just cannot be ignored.
Our concern with productive policy play reflects the developing sense in education research that policy is usefully seen as object-like, but not as determinant assemblages of discourses, peoples, regulations, rewards, and punishments. 1 If policy is object-like, and yet not exclusively formative, then all future responses to the policy require deliberate human activity. Policy is productive. But the resulting arrangements are never simply linear “implementations,” especially when they involve large numbers of actors, with varying authorities and knowledges, interacting with the policy over time. Thus, there will always be some “play”—or room for negotiation, interpretation, and selective appropriation—as policy directives move from administrative centers to diverse local contexts of implementation. Where policy is activated, play—both as entertainment and as serious attempts at transformation that may be reconstitutive—will exist (McDermott & Tylbor, 1983). Assembling policy and disassembling policy represent forms of what Clifford Geertz once referred to as “deep play,” in which a major bet is made that, should it fail, threatens the very status of the player (or players).
Federal education policy may be just one of these bets, as it produces an immense assemblage of people playing, producing subpolicies, assessing them, criticizing them, imagining alternatives, and sometimes making large bets that could have radically dangerous consequences for themselves or for those who had no choice but to participate in what is no longer a game. This was true of President Bush and Senator Kennedy when they negotiated what became the No Child Left Behind Act (NCLB) of 2001. It is also true of those whom Lipsky (1980) deemed “street-level bureaucrats,” in charge of implementing policy locally as part of their daily work lives.
In this article, we discuss these concerns by looking briefly at one of many local NCLB-guided subpolicies produced in the country’s largest school district, New York City (NYC). We trace the assembling of school officials, politicians, large corporations, and others who found themselves in new relationships with each other under NCLB. We mention the many ensuing difficulties and what some people, in local settings, attempted to do against or around the mandates to which they were accountable. Before we do this, we briefly discuss the theoretical and methodological notion of assemblage. We conclude with comments about what policy makers, in any setting, can do with the recognition that the processes we illustrate are not evidence of incompetence or malice but of the very nature of social life.
Policy Assemblage as Cultural Production
A Nation at Risk begins,
Our Nation is at risk. Our once unchallenged preeminence in commerce, industry, science, and technological innovation is being overtaken by competitors throughout the world. . . . If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war. . . . Our society and its educational institutions seem to have lost sight of the basic purposes of schooling, and of the high expectations and disciplined effort needed to attain them. This report, the result of 18 months of study, seeks to generate reform of our educational system in fundamental ways. (National Commission on Excellence in Education, 1983)
A Nation at Risk is a particularly famous and powerful object. It is credited with convincing President Reagan and Congress not to abolish the newly created federal Department of Education. It is also one of the many objects that eventually led to NCLB, the policy that concerns us here. Even as NCLB fades, its history has not yet been written. What is certain, however, is that during the 18 years that separate the report from NCLB, a host of people was assembled. These people made something (or some things) that disrupted what many other people had been doing, that constrained them in new ways, and that opened new routes for participation and resistance. The voices of billionaires and statisticians became stronger, whereas the voices of educators’ unions became weaker. Teachers had to learn how to prepare students for new tests. Principals had to learn new ways to generate and manipulate numbers. State officials had to learn to write new kinds of reports that would be differently (often electronically) circulated for new kinds of audiences. Whether all this activity has reversed “mediocre educational performance” is an open question. The overall assessment of the assessments will itself produce new assemblages, new disruptions, and new constraints, as well as new possibilities.
For now, we know that “culture” was produced through NCLB. It is not a culture as a consensus learned in obscure ways early in life, but a culture of ongoing debates about conditions participants discover as they engage with “the contemporary, the actual, the near future and recent past” (Rabinow, 2003, p. 55). It is not a culture centered on personal traits supposedly guiding behavior in subconscious processes the make participants (mis)understand their conditions or reproduce their subjugation. It is, in Varenne’s (2010) terms, an anthropological positioning of culture forged around figuring out the organization of one’s conditions and what can be done with them. It is also the production of new conditions in history, itself the temporary result of the joint activity of the contingent assemblages that earlier history has produced.
Thus, we extend the notion of assemblage that is associated with ANT, a theory initially developed as a framework for science and technology studies, to give a stronger sense of the actual activities that make science in history (Callon, 1986; Callon & Latour, 1981; Latour, 1988, 2005; Law, 1986). 2 The theory has now proven its usefulness for tracing the workings of activities that involve the linking of many people in many different conditions, each restating (translating) what others have done into the language and practices of their own settings. In the process, they add to (and subtract from) what had been said and done before, thereby producing some new object with new consequential properties for different future actors. Thus, a hunch (that the DNA molecule has the shape of a double helix) becomes a “scientific fact” that schoolchildren must learn and about which they are tested. Failing to remember this fact can have serious consequences for their future. Hence, a worry (that “our nation is at risk”) becomes self-evidence that Congress should do something, which becomes NCLB, which, in turn, spawns subsidiary regulations over the workings of state bureaucracies that write further regulations, and so on an so forth.
The central contribution of ANT to the social sciences has been a heightened sense of the linked distribution of activities in networked ad hoc and emergent assemblages that produce different experiential objects in each of the linkages. 3 Translation has become a key concept to deal with the (trans)formation of the common sense in setting A to the common sense in the linked setting B. At each of the linkages, however temporary, “one entity has worked upon another to translate or change it to become part of a collective or network of coordinated things and actions” (Fenwick & Edwards, 2010, p. 9). Translation involves a local entity facing the problems produced as it attends to a statement made by another entity that cannot be ignored. It involves enrolling actors to attend to the statement or mandate, mobilizing actors and forging new connections between entities, and then negotiating the frictions in the emerging assemblage (Tsing, 2005), as well as dealing with the rising evidence of play throughout the activities of the newly linked entities and actors.
To illustrate all of this, we sketch a few aspects of local translation of NCLB into new practices that produce new constraints on NYC participants (particularly principals but also teachers), while allowing some of these participants to play in ways that sometimes have serious consequences. The data come from a multisited ethnography the first author has been conducting since 2005. The goal of this work is to examine the localization (or local translation) of NCLB in New York. The initial ethnographic study concluded in 2008 but was extended, with additional interviews, observations, and policy artifacts collected through 2010. 4
Applying Productive Policy Play to the Analysis of a NCLB Controversy
We focus on one of NCLB’s performance provisions. It is well known that NCLB mandates high-stakes testing assessments that impose radical consequences on the everyday lives of principals, teachers, students, and parents. It is less known that the reporting of these assessments necessitates the assembling of constellations of people and entities, many of whom may be linked through other networks.
NYC, with its mayor, schools chancellor, educational support businesses, and ties to the Gates Foundation, has been, to date, an altogether enthusiastic participant in the implementation and further development of NCLB-type test-driven accountability policies. Through its committees and consultants, NYC’s Department of Education (DOE) has produced several test-driven accountability strategies aimed at increasing analysis of student achievement data. Practically, this has meant administering, scoring, and reporting more than 50 million standardized tests. In 2008 alone, the district spent an estimated $130 million on testing (Medina, 2008). Moreover, to prepare for the tests and manage the testing data, the district paid hundreds of millions of dollars to mostly for-profit vendors, including test and curriculum publishers, online database developers, and tutoring and test preparation companies, even as it faced, in 2008, a $771 million reduction in state aid (U.S. Department of Education, 2009).
Among the strategies, one involved purchasing and implementing the $81 million Achievement Reporting and Innovation System (ARIS). ARIS, a web-based platform created by IBM and six subcontracting companies, was launched by Mayor Michael Bloomberg and then–Schools Chancellor Joel Klein to “help determine which schools were performing up to expectations and which are falling short” (Gotbaum, 2009, p. 4). 5 ARIS functions primarily to warehouse and network testing data. It merges earlier district systems into one system, interfaces with the state educational systems, and houses analytic tools to evaluate comparative test scores and student achievement across the district and the state.
Many district administrators and city officials hailed ARIS as essential, especially for teachers and principals whose schools were already under review for low student performance. One expounded,
ARIS allows teachers and principals to see all their data in one place. That’s unheard of. . . . Principals might not like it, anyway at first, but this is the future. They won’t be able to meet their obligations to the district and so on if the data in ARIS isn’t looked at. No Child [Left Behind] requires this kind of hyper-specialized data system. We didn’t make it up. (interview, February 19, 2009)
This administrator’s comments echoed statements given by Chancellor Klein and U.S. Secretary of Education Arnie Duncan. Unveiling ARIS, Klein told journalists that “for New York City teachers, the future is now,” and he predicted that NYC would be a national model of data-driven instruction (Zelon, 2008). At a press conference in Brooklyn, Duncan expressed his hope that states would use the economic stimulus money to adopt accountability-oriented reforms, such as ARIS (Cramer, 2009).
But not all administrators and city officials were similarly enthused. ARIS soon became quite controversial. A district administrator, who had previously worked for the New York State Education Department, opposed the purchase of ARIS and candidly explained what he saw as the underlying dilemma for the city. Over two interviews conducted in 2008, he explained,
The way I see it . . . it’s a gamble. The city’s putting everything we have into ARIS, but for what? Nice sets of data reports? . . . I see ARIS as a way, a big public way, of saying to the feds “Look, this city’s doing what you told us we needed to do and we’re doing it better . . . because we’re New York, and that should be enough.” (interview, February 21, 2008) I’m not saying that ARIS might not help us comply with NCLB, . . . but I’ve, all of us have been dealing with this ARIS rollout debacle and in the end, trust me, everybody’s going to just do it like they always did it and figure out ways to get around ARIS. (interview, July 5, 2008)
Not only did the district administrator reveal his skepticism about ARIS, but more important to our discussion here, he pointed to the ways in which “everybody”—school leaders, district administrators, and city employees—would play not only with NCLB compliance but also with the requirements of using ARIS to delve into and use testing data. They would, in their own ways, as part of their compliance to the state and federal education departments, negotiate the regulations and push the edges of the rules. According to the administrator interviewed, they always had and they always would; playing was part and parcel of daily work in education.
The controversy became more visible (and perhaps more productive) in fall 2008 when Public Advocate Betsy Gotbaum surveyed school principals, the primary intended users of ARIS, to determine if they deemed ARIS worth the cost. Several principals interviewed in this study expressed concern at the price of ARIS during a time of budget cuts. A Brooklyn middle school principal lamented,
Now we’ve got to learn how to use ARIS and devote time and money into training staff how to use it. . . . I just see this as a bureaucratic nightmare and it’s just started. The roll-out of ARIS is way past due so now we’re scrambling, and it’s so full of glitches that I have staff people using the sources that were already available. (interview, March 12, 2009)
Similar concerns were aired at a district-sponsored ARIS workshop. Several principals requested additional funding and professional development for teachers who would be responsible for getting ARIS up and online at their schools.
During the 2009–2010 school year, other school administrators insisted that administrators’ and teachers’ concentration on the new system reduced the amount of time they were spending face-to-face with students. This echoed a concern noted in Gotbaum’s (2009) report, in which “a significant minority of principals” found ARIS to interfere with their jobs and the responsibilities of their staff, as well as it doing little, if anything, to improve teaching and learning in schools. A Manhattan assistant principal in this study complained,
Sure, ARIS is fancy. Teachers can look up trends in test scores and even blog, but . . . yesterday, [I went] in a class and no teacher. No teacher teaching. She’s at her computer, waiting for some table to load to see which students knew how to write topic sentences or something. I mean really. Is this a good use of her time? If this is compliance, it isn’t worth it. . . . Impacting teaching with data isn’t supposed to mean less teaching time. (interview, November 18, 2008)
In 2009, after this assistant principal had become the school’s principal, she explained how she had “assigned ARIS” to a handful of teachers, across the subject areas. They, in turn, generated “ARIS highlight” reports for the rest of the teachers, as “having too many teachers devote too much time to ARIS is a waste. . . . I don’t know if this is kosher, but I’m doing it” (personal communication, May 4, 2009). In our terms, this principal was playing with the mandate, possibly at some risk for herself.
A few other principals were playing with higher stakes given that their efforts were more overt. One of them had created a much cheaper, and arguably more efficient, streamlined data analysis program that he shared for a relatively minimal price. After the release of ARIS was delayed, one Brooklyn principal in the study explained that he had already purchased this alternative: “My accountability timeline couldn’t wait for ARIS. I’m accountable. IBM isn’t, and apparently the DOE [that] concocted this specialized system isn’t either, despite their obsession with data” (interview, December 7, 2008). A few principals in the study used $5,000 to $10,000 of their discretionary funds to buy versions of the alternative system, which tracked student progress, grades, and compliance with NCLB. One principal explained, “It isn’t fancy, but it also doesn’t take time away from instruction” (personal communication, November 21, 2009). Another concurred, noting that even principals who had waited for ARIS found it too time consuming and difficult and were purchasing alternatives systems.
Assessing Assemblages for Future Policy Making and Policy Study
We are not here in the business of deciding whether the principals who refused ARIS were more correct than Klein in the implementation of NCLB or in making NYC a better school system for children. As we write, Klein is no longer chancellor, NCLB is fading, and more and more schools are not using ARIS. We are in the business of demonstrating how sketching the linkages within and across entities can guide not only the analysis of current policies but also, we hope, the production of new policies.
Even though the preceding sketch is brief, it brings attention to the connections between Washington regulators, NYC politicians and administrators, large corporations, and principals. We do not have the space to do more than mention other people and entities that are part of other associations that also become involved in educational policy assemblages in general and in NCLB networks specifically. Most preeminent in these associations are the university professors and researchers involved in determining which data to gather, how to analyze them, and how to interpret many of the reports that the software, like ARIS, provides. Particularly noteworthy would be all the economists and statisticians involved in “value-added education” and “performance management.” The following summarizes the work of a famous group at the University of Wisconsin-Madison:
In the context of education, performance management refers to improving the effectiveness of schools and districts using rigorous data, regularly assessing school and district performance, and holding managers accountable. Every level of a school district can use performance management principles of quality and systems management and the associated tools and processes. (Wisconsin Center for Education Research, 2011)
Embedded in that paragraph is an opening for software engineers, interface designers, marketing people, and so on, in large corporations like IBM, which led the development of ARIS. Their work is itself constrained, and enabled, by a slew of people and things. 6
An interesting question concerns the classic issue of boundaries and their emergence: Whom can we see being caught within an assemblage? For example, the exact place of teachers, in our case, is somewhat unclear, although principals can try to browbeat teachers into using resources that they are not much interested in or at least into displaying their use of them—even if they do not use these resources. 7 The place of parents, and of people who do not have children in school, is even more uncertain, although we suspect very few are aware of the data-gathering assemblage that concerned us here. 8 The place of analysts such as ourselves and others in academic, politics, or government is unclear, although much of the rhetoric that introduces and justifies particular implementations suggest that proponents, from Arnie Duncan and Bill Gates to local principals, are aware that what they are doing is controversial and may have a much shorter half-life than they hope.
As Venturini (2010) notes, “Controversy functions as a ‘hybrid forum,’ a space of conflict and negotiation among actors that would otherwise happily ignore each other” (p. 261). The relationship between such actors would lead us much further than needed here. But the general principle remains. Who is going to be involved in what, in what ways, for what purposes, and for what consequences is not a matter of deduction of imagination but a matter of painstaking discovery. Only if we become aware of the linkages—their properties, the affordances of the various linked entities, and the means for communication across the linkages—can we hope to actually achieve political goals through policies that are truly aware of the conditions of their implementation.
We have emphasized play in all its forms because, too often, difficulties in policy implementation are presented as merely technical problems amenable to fine-tuning of procedures or interfaces. Policy makers tend to position school reform through a commonsense analogy to what they imagine engineers do technically when they optimize a machine. Yet what are framed as technical issues are quickly translated in policy processes to “socio-technical controversies” (Callon, Lascoumes, & Barthe, 2001) in which multiple actors are playing not only with their roles but also with each other, their policy contexts, and policy itself.
Thus the work of assembling an educational reform is not akin to that of engineers reducing play in the fitting of various parts of a machine they are designing and thereby improving its efficiency. Engineers do not have to deal with parts that themselves play, for example, in pretending that they are doing something that they are not doing in such a way that they are not caught. And engineers, mostly, do not have to design machines that are themselves a major bet that something altogether untried at a certain scale will work so well so that millions of people will have to live with this bet. Arguably, G. W. Bush and Edward Kennedy made such a bet when they worked together to pass NCLB—even though they both may have been trying to play the other. Klein and IBM made different kinds of bets as they signed the contract for ARIS. And principals make other kinds of bets when they look for alternatives. All are better seen as “bricoleurs” in the anthropological sense (Lévi-Strauss, 1966): people who put something together for certain immediate purposes and only need to work for the time that it takes for the need to disappear.
Footnotes
Notes
Authors
JILL P. KOYAMA is an assistant professor in the Graduate School of Education, Educational Leadership and Policy, State University of New York at Buffalo, 466 Baldy Hall, Buffalo, NY 14260;
HERVÉ VARENNE is a professor of education at Teachers College, Columbia University, Box 115, 378 Dodge Hall, New York, NY 10027;