Dying Bees and the Social Production of Ignorance

Abstract

This article utilizes the ongoing debates over the role of certain agricultural insecticides in causing Colony Collapse Disorder (CCD)—the phenomenon of accelerated bee die-offs in the United States and elsewhere—as an opportunity to contribute to the emerging literature on the social production of ignorance. In our effort to understand the social contexts that shape knowledge/nonknowledge production in this case, we develop the concept of epistemic form. Epistemic form is the suite of concepts, methods, measures, and interpretations that shapes the ways in which actors produce knowledge and ignorance in their professional/intellectual fields of practice. In the CCD controversy, we examine how the (historically influenced) privileging of certain epistemic forms intersects with the social dynamics of academic, regulatory, and corporate organizations to lead to the institutionalization of three interrelated and overlapping types of ignorance. We consider the effects of these types of ignorance on US regulatory policy and on the lives of different stakeholders.

Keywords

academic disciplines and traditions politics power governance expertise epistemology

Clint Walker is a veteran migratory commercial beekeeper, whose beekeeping firm has managed honey bee hives (or “colonies”) for the business of crop pollination since 1940. As the winter of 2006-07 began to thaw, Walker saw thousands of his seemingly healthy honey bee hives collapse in a manner that he had never before seen. Adult honey bees suddenly just disappeared, leaving no traces of their bodies. He soon learned that hundreds of other beekeepers—commercial, sideliner, and hobbyists¹—were having similarly baffling experiences all across the United States. Bee researchers, who were notified by commercial beekeepers about the strange phenomenon, called it the “Colony Collapse Disorder” or CCD.

CCD is of enormous social, economic, and environmental concern because it has accelerated the decline of the primary pollinating species in North America. Most existing populations of honey bees in the United States are managed by commercial beekeepers, and thus, CCD threatens the political economy of US agriculture. Many farms growing fruits, nuts, fibers, and vegetables have come to rely heavily upon the pollination services provided by managed honey bees. As of 2000, honey bees were estimated to have brought close to $15 billion worth of increases in crop yield and quality to the US agricultural market (Morse and Calderone 2000). CCD additionally deepens an ongoing ecological crisis that is marked by rapidly declining pollinator populations worldwide (National Research Council [NRC] 2007). Interested actors in academia, industry, government, and civil society have been involved in well-publicized efforts to uncover the symptoms and causes of CCD (Barrionuevo 2007; Stokstad 2007).

A honey bee colony suffering from CCD is characterized by a sudden loss in its adult population, leaving behind the queen, young emerging adults and brood as the only remaining denizens. The absconding bees, nowhere to be found, presumably fly some distance away from the colony before dying. But the abandoned colonies are seemingly healthy, with rich stores of honey, pollen, and brood. Beekeepers and bee researchers alike are puzzled by the fact that these abandoned stores, which would normally be plundered by “robber” bees from nearby colonies and other insect “pests,” are left untouched. Detailed anatomical and molecular analyses of the remaining bees from CCD colonies by researchers reveal signs of unusually high levels of infection (Cox-Foster et al. 2007; vanEngelsdorp et al. 2007; Johnson et al. 2009; Bromenshenk et al. 2010). Clearly, CCD-colony bees are extremely sick. But the factors that cause these bees to become vulnerable to a cocktail of microbial and fungal infections remain contested and controversial.

We utilize ongoing debates over the causal role of certain agricultural insecticides in CCD as an opportunity to examine the kinds of knowledge production practices and, by corollary, ignorance (Gross 2010) that are given legitimacy in the controversy over CCD, and we consider their consequences for the actors who have stakes in the outcome. Through our discussion, we explore how the privileging of certain taken-for-granted approaches to knowledge production leads to a systematic production of ignorance, and we consider the effects of this ignorance on US regulatory policy and the lives of different stakeholders.

University bee toxicologists, agrochemical companies, farmers, and commercial beekeepers have different stakes in understanding CCD and regulating the risk factors associated with it. These actors have contrasting approaches and make different claims about the causal role of manufactured agrochemicals in CCD. Their differing knowledge claims have different implications not only for research agenda setting but also for regulatory policies. Expert toxicologists’ claims, which tend to assert that there is “no conclusive evidence” linking certain manufactured agrochemicals to CCD, direct attention toward nonanthropogenic causes such as parasites and pathogens, and this orientation dovetails with the outlook of powerful agrochemical interests. Alternative knowledge claims promoted by less powerful stakeholders, such as commercial beekeepers, tend to implicate the manufactured agrochemicals causally, and suggest kinds of research generally not now being done in order to understand if and how these synthetic chemicals are implicated in CCD. This approach to research in turn implies a precautionary regulatory orientation, which is compatible with the livelihood stakes and commercial interests of the proponents of these alternative knowledge claims, and contrasts with existing policy.

Typically, US regulators and agrochemical companies privilege toxicologists’ approaches over the alternative orientations of commercial beekeeper groups. In this article, we argue that dominant approaches to honey bee toxicology institutionalize particular kinds of ignorance about the involvement of agricultural insecticides in CCD. This ignorance, in turn, justifies a lack of regulatory action on the part of regulators at the US Environmental Protection Agency (EPA) and ultimately serves the interests of elite agrochemical companies. Much like Frickel and Edwards’ (forthcoming) work, our research illustrates how “ignorance emerges from within the rules, procedures and protocols . . .” that constitute different professional and intellectual fields of practice.

Our article is organized into four parts. We begin with a brief review of the current state of research on the social production of ignorance. Here, we lay out a set of ideas that we believe build on the nascent project of understanding “the problem of undone science” (Hess 2007). In the next section, we provide a detailed description of the CCD controversy in relation to the US agricultural industry and the contested role of certain agricultural insecticides in it. The following section analyzes particular episodes and positions in the CCD controversy in order to show how the dominant approach supported by academic scientists, regulators, and agrochemical companies generates particular forms of ignorance. The concluding section discusses the implications of our study.

Our analysis relies upon information that we gathered from (1) semistructured interviews with beekeepers, academic scientists, agro-industry representatives, and federal regulatory officials; (2) participant observation at the 2009 annual meeting of the Entomological Society of America in Indianapolis, IN, the 2010 North American beekeeping conference in Orlando, FL, and at the “The Bee Lab”; (3) published documents that include research articles in peer-reviewed journals, honey bee and agro-industry trade journals, federal “guideline” documents on risk assessments of pesticides, and publicly available correspondence from commercial beekeepers and federal officials at the Office of Pesticide Programs within the EPA.

The Social Production of Ignorance

Actors’ knowledge of any phenomenon is affected by how and where they look (Haraway 1991). By looking in certain ways, one leaves unexamined other ways of understanding. As a result, the production of knowledge is always matched by the corresponding production of ignorance or “nonknowledge” (Harding 2000; Gross 2010). In technoscientific controversies, where actors across multiple groups struggle over how to understand and respond to a phenomenon of concern, asymmetries in access to resources and power shape what does and does not get ignored (Harding 2000). In this sense, the social production of ignorance belongs to a broader politics of knowledge. In recent years, an increasing number of analysts have recognized this and have begun to more explicitly probe the systematic production of ignorance through the lens of “undone science” (Hess 2007; Frickel et al. 2010), “knowledge gaps” (Frickel and Vincent 2007), “strategic ignorance” (McGoey 2012), and “scientific cultures of nonknowledge” (Böschen et al. 2010).

Undone science refers to the kinds of research that get systematically ignored, left unfunded, or incomplete, but is recognized by other actors as being worthy of serious consideration. In the process of their analysis, Frickel and his coauthors (2010) illustrate through several case studies how “regulatory paradigms” and social movements shape differing sets of done and undone science. Frickel and Vincent (2007) provide further hints into the influence of disciplinary fields on “regulatory knowledge gaps” in their analysis of expert understandings of contamination in New Orleans in the aftermath of Hurricane Katrina. They argue that scientific disciplinary practices create knowledge gaps—areas where there is an absence of understanding as the result of the imposition of “a particular framework of theoretical assumptions, standards of evidence, and styles of interpretation” (2007, 184), which reduce the “ecological and socio-historical contexts” in which regulatory knowledge is made. Focusing on government regulation, McGoey (2007, 2012) explores the benefits to regulators and regulated industries of maintaining ignorance strategically. Finally, Böschen et al. (2010) argue that scientific disciplinary fields are characterized not only by differing approaches to knowledge making—what Knorr-Cetina (1999) calls “epistemic cultures.” They are also simultaneously “scientific cultures of nonknowledge,” whose differing orientations to control and complexity lead actors to treat what is not known in different ways.² Social actors utilize these differing paradigms of nonknowledge in strategic and flexible ways toward advancing their own interests (Böschen et al. 2010). In beginning to conceptualize “the problem of undone science,” knowledge gaps, and nonknowledge, the work of scholars like Hess, Frickel, McGoey, and Böschen and their collaborators make valuable contributions to our understanding of the politics of knowledge production. We build on their insights.

Based on the existing scholarship, it remains unclear how the social dynamics of disciplinary institutions affect the production of ignorance, and how these intersect with particular practices of knowledge production in specific regulatory and corporate organizations. We must ask how disciplinary and nondisciplinary contexts of knowledge/nonknowledge and dominant norms of different stakeholders affect the social production of ignorance. We address these issues by taking seriously Frickel and Vincent’s (2007) call to analyze “the obstacles to usable knowledge [that] are deeply embedded in the machineries of knowledge production itself” (2007, 187).

We begin with an observation with which few in science and technology studies would disagree: methods of data collection, approaches to testing, structures of experimental design, and standards of evidence—what we call epistemic forms—emerge historically. The rise to prominence of some epistemic forms and the marginalization of others does not always reflect the inherent superiority of those accepted among researchers and others, but instead reflects historical struggles and debates that often lead the virtues of some approaches to be taken for granted, and the others to appear problematic by comparison. The result of such processes of institutionalization can prompt some knowledge that might have been produced utilizing marginalized epistemic forms never to be produced or for the knowledge produced using such sidelined forms to be ignored, not considered knowledge at all. In either case, the upshot is ignorance.

Our story of the production of ignorance centers on the dominance of the approach taken to the study of links between insecticides and CCD by honey bee toxicologists. Here, there are several important sociohistorical points to note. First, the roots of the work of these scientists are found in the research done by early entomologists and honey bee scientists, who were associated directly or indirectly with the US Department of Agriculture beginning in the late nineteenth century (e.g., Lowe and Parrott 1902; USDA 1907); their work is inextricably tied to a highly chemically dependent agriculture (Palladino 1996). Among the studies done by these scientists around the middle of the twentieth century was research in which “treatment” groups of bees were exposed to predetermined amounts of specific chemicals, and aspects of their mortality were compared to the nontreatment “control” group (reviewed in Anderson and Atkins 1968). In highly controlled laboratory and field experiments, scientists measured each chemical’s lethal effects statistically, using representations like “dosage-mortality curves,” “time-mortality curves,” and “time-concentration” curves (Anderson and Atkins 1968). This epistemic form, which came to be dominant in environmental toxicology research on bees, was structured to measure individual factors and their causal roles. It was designed to ascertain rapidly appearing lethal effects on specific individual insect targets (Suryanarayanan and Kleinman 2011) and amounted to what Böschen and his colleagues (2010, 790) call a “control-oriented scientific culture.” To understand the balance of knowledge and ignorance in the CCD controversy, it is crucial to realize that in emphasizing the rapid appearance of individual, and lethal causal factors, the dominant toxicological epistemic form ignored—meaning that it failed to study, indeed could not study or would not consider seriously—possible evidence of the effects of low or “sublethal” levels of insecticides.³ It could not seriously or systematically entertain plausible interactions with ambient factors such as other pesticides and pathogens. Thus, this epistemic form was structured in such a way that it could not consider a complex set of interacting factors that might plausibly lead to slow, progressive effects over multiple generations in a beehive’s life cycle. As we discuss below, this epistemic form is considered the legitimate means for understanding the potentially harmful real-world effects of synthetic chemicals on bees. It is the form underlying the EPA’s regulation and is used by agrochemical companies to justify their existing practices. Again, as we discuss below, commercial beekeepers offer an alternative epistemic form, which, with different methods of data collection, observation, measures, and analysis, suggests explanations that challenge those dominant in the CCD controversy. But their form and associated findings are largely ignored. There is a final aspect of the epistemic form of toxicologists doing bee research that contributes to the production of ignorance—the preference for conclusive evidence as defined by the statistical demand of 95 percent confidence that results are not due to chance. This amounts to a preference for false negatives over false positives and means that researchers may conclude that there are “no differences” between treated and untreated honey bee hives, when there could be.

In all, we see that the characteristics of honey bee toxicologists’ dominant epistemic form have the potential to lead to three interrelated and overlapping varieties of ignorance. First, there is research that does not get done because the accepted epistemic forms make doing such research highly unlikely. This work is not undone for lack of funding or for a strategic desire to avoid finding something out. Instead, there are certain questions that established methods cannot address. The research is undone because it is undoable and knowledge gaps result. Second, when an epistemic form prevents certain questions from being addressed, both the knowledge and the nonknowledge⁴ emerging from that work can be misleading, and results that are distorted in this sense lead to a second variety of ignorance, what others have called “false knowledge” (Smithson 1985). Finally, when institutionalized norms lead researchers to fear inconclusive findings, a third variety of ignorance results. Indeed, by not taking seriously inconclusive results, researchers and those who share their epistemic commitments prefer ignorance to knowledge.

CCD and Agricultural Pesticides

CCD was first reported by migratory, commercial beekeepers operating in the business of crop pollination (Stokstad 2007; vanEngelsdorp et al. 2007). Growers have become increasingly dependent on managed honey bees in order to pollinate their fruit, vegetable, and fiber crops. This has largely occurred in a post–World War II context, where populations of endemic “native” pollinator species have been disappearing, and larger monoculture farms have come to dominate the US agricultural landscape (Spivak 2010). Commercial beekeepers travel from one farm to another, renting out their honey bee hives during specific seasonal periods in the locales where growers’ crops are set to bloom. The beekeeping practice of placing hives at or near crop acreages for pollination exposes honey bees to grower practices, such as the use of insecticides to kill perceived insect pests (Spivak 2010).

Historically, beekeepers have experienced massive bee kills, typified by stacks of dead bees in front of hives, from exposure to “traditional” insecticides, which poisoned foraging bees that came in contact with crops treated during a temporary spraying period (vanEngelsdorp and Meixner 2010). An increasing range and diversity of classes of insecticides, such as the “neonicotinoids” and “ketoenols” that are replacing the older varieties, are characterized by a newer systemic mode of action. ⁵ Systemic insecticides such as imidacloprid, clothianidin, and spirotetramat, move through plant tissue to become localized in leaves, pollen, and nectar. They gain their toxic action by persisting in plant tissues for significantly longer periods of time, at apparently low, but highly effective doses compared to conventional insecticides.

Manufacturers and regulators tout these chemicals as posing “reduced risk” to humans (EPA 1999; Schmuck and Keppler 2003). They are also claimed to be “greener,” since their modes of application enable their prolonged presence within plants and theoretically preclude the need for frequent spraying.⁶ These systemic chemicals are a key part of the agrochemical industry’s vision of combinatorial “stacks” of pest management, used in complement with genetically engineered traits (Bayer CropScience 2010). However, their newer systemic mode of action of these chemicals creates the possibility that foraging bees may take up nectar and/or pollen containing low doses of the toxin and not immediately die from acute poisoning, but return to their hives with sublethal exposure, which when fed chronically to developing brood result eventually in CCD. Whether this is the case is the cause of an ongoing controversy, in which beekeepers, farmers, researchers, state and federal regulatory agencies, and manufacturers of the neonicotinoids are embroiled.

Nondisciplinary Form of Commercial Beekeeping Knowledge and Ignorance

Despite scientists’ experimental findings, several commercial beekeepers with years of experience in the business of crop pollination are convinced that the newer generation of systemic insecticides is primarily responsible for CCD. In a public letter to the EPA,⁷ veteran commercial beekeeper David Hackenberg, who first notified bee researchers about CCD, describes his experience:

In 2004, when our bees were first exposed to imidacloprid, we saw things happen in our bees that we have never seen before. Good colonies of bees run through pollinations and honey crops over the summer that we now know were exposed to Assail^® [imidacloprid] in Apple pollination and Admire^® [imidacloprid] in pumpkin pollination, by fall when no new food was coming into the hives, began to collapse at rapid pace, leaving nothing but a queen and a few bees in the [hive] boxes. The farmers that I work with are sensitive to using anything that would hurt my bees because they recognize how important good pollination is to the success of their crops. They were told by their chemical suppliers that these ‘new’ pesticides were ‘safer’ for honey bees and they could even apply them during bloom without damage to bees. We did not see any dead bees in front of our hives while they were in these pollinations. We don’t bring all our hives to these pollinations. In the fall, it was clear that the bees that had been on honey locations were OK with normal mortality of 10 to 15% loss, while the pollination hives had 75 to 80% loss. We saw this same problem with pollination hives in 2005 and 2006. It was in the fall of 2006 that we began to associate these losses with summer pollination exposure. I would like to see more research done on imidacloprid to determine chronic effects on honey bees. I believe that we should limit the use of imidacloprid until these questions are answered.

Hackenberg’s observations, which echo those of several other commercial beekeepers, point to an epistemic form with quite different characteristics than that associated with toxicologists doing bee research; a comparison between the two and the findings they generate suggest the complicated balance between knowledge and ignorance in the CCD case. We characterize commercial beekeepers’ epistemic form as real time and in situ. There is a parallel here with what Böschen and his colleagues (2010, 790) refer to as a “complexity-oriented culture” in which those engaged in knowledge production are open to unanticipated events and to uncontrollable and context-sensitive settings. This contrasts with toxicologists, whose data are generated through more controlled conditions, and premised on an effort to generate causal results about relatively rapidly appearing effects; beekeepers’ knowledge of CCD is based on the actual field conditions that commercially managed honey bees encounter. Moreover, their informal analysis packages crucial information about multiple, complex aspects of colony health into knowledge that is meaningful and useful to beekeepers. For example, “brood pattern”—the overall pattern in which brood develops on a hive’s comb—is used by beekeepers not only as an informal measure of brood health but also to gauge the queen’s reproductive health and the local availability of nutritional sources. From the perspective of beekeepers, it offers a comprehensive picture of the entire hive, and beekeepers can track changes in brood pattern over time, as they monitor their hives. By contrast, honey bee toxicologists tend to use formal and narrower statistical and quantitative measures of individual brood cells to gauge brood health over a limited period (e.g., Cutler and Scott-Dupree 2007). Incorporating a large number of variables that are not easily isolated, beekeepers see a broader picture of honey bee health, but they may not isolate specific independent causes of illness. This informal, nonreductive epistemic form of commercial beekeeper knowledge is highly attuned to the dynamic, local, and variable environmental conditions that impinge upon honey bee lives, and by corollary, beekeeper livelihoods. But from the perspective of traditional biological disciplines, its imprecision leads to knowledge that is at best loosely correlative, and rarely conclusive.

Beekeepers' research is closely tied to their livelihood stakes in understanding CCD. Based on their own assessments of honey bee health, several commercial beekeepers, as part of the recently formed National Honey Bee Advisory Board (NHBAB), are pushing the EPA to adopt a precautionary approach to the usage of the newer systemic insecticides. At the policy level, this would entail a suspension or significant limitation in the usage of entire new classes of insecticidal chemicals that share a common mode of action, in the event of conflicting evidence regarding their alleged adverse effects on beneficial insect pollinators. At a practical level, it could engender knowledge production practices that are more representative of the (environmental) settings in which commercial beekeepers ply their trade. In this context, commercial beekeepers have emphasized the need for more research on the longer-term, cumulative, and “sublethal” effects of the newer systemic insecticides on honey bees in both laboratory and realistic field settings. Here, a precautionary approach implies knowledge evaluation that would err on the side of false positives and undertake anticipatory protective action in the face of suggestive evidence. This would put the onus on the pesticide manufacturers to get such research done (through collaborations with academia or “in-house”), and from the perspective of precautionary-oriented beekeepers, such research should be a prerequisite to the pesticide’s registration. Additionally, this sort of precautionary approach could push the US agro-industry toward the development of viable nonchemical, ecological alternatives to the newer systemic chemicals (e.g., Altieri and Nicholls 2005). The approach advocated by beekeepers would produce knowledge in areas not systematically considered by researchers in academia, industry, and government. Scientists, however, largely reject or at best equivocate on the commercial beekeeper knowledge linking the neonicotinoids to CCD.

Disciplinary Form of Toxicological Knowledge and Ignorance on Honey Bees

Many researchers in academia, agrochemical industry, and the EPA invoke the dominant toxicological epistemic form we described earlier, and on the basis of this form of knowledge production argue that there is no conclusive “scientific evidence” for the causal role of the neonicotinoids in CCD. Again, their demand for conclusive results means ignoring plausible but inconclusive results. Along with the EPA, many academic bee researchers and many agrochemical companies demand experimental evidence of a definitive, causal link between the neonicotinoids and CCD. Toxicological forms of knowledge and ignorance on honey bee colonies are a result of design standards and methodological choices that do not necessarily reflect the on-the-ground realities of commercial pollination. Field experiments on whole hives are designed using standards that are established in laboratory contexts on individual honey bees, not whole hives. Laboratory-based standards such as the Lethal Dose, 50% (LD₅₀) and the No Observable Adverse Effect Level (NOAEL)⁹ assume that the tested toxin is the only one that an adult honey bee encounters in its environment. But the environments in which beekeepers work expose honey bees throughout their life cycle to a multitude of local and potentially interacting chemicals, pathogens, and parasites. Indeed, a recent survey of North American apiaries, by some of the last few public insect toxicologists in the United States, found 121 different pesticides and their metabolites associated with commercial beehives (Mullin et al. 2010). Contemporary experimental epistemic forms might be unable to test plausible scenarios where the neonicotinoids by themselves may not cause CCD, but contribute to it at low doses, in intricate interactions with other ambient factors. As a result, toxiciologists’ field experiments tend to overlook the ecological complexity of locations in which honey bee colonies operate. Research to understand this complexity is undone. It is not tested for.

An instructive example is a field study undertaken by academic honey bee scientists on the effects of the neonicotinoid, clothianidin (Cutler and Scott-Dupree 2007). The study design ignored the proximity of pesticide-treated experimental crops of Canola to supposedly unexposed “control” hives. But honey bees can forage over several miles in search of pollen and nectar, which they bring back to feed the rest of the colony (Spivak 2010). As a result, so-called untreated hives, while not receiving pesticides from the experimenters, could have had bees that foraged on the relatively nearby pesticide-treated crops. Similarly, so-called treated hives could also have had bees that foraged on the nearby untreated plot of Canola. In other words, an observed lack of “long-term impact on honey bees” (Cutler and Scott-Dupree 2007: 765) from clothianidin may be because all study hives inadvertently had access to both pesticide-treated and untreated experimental plots.¹⁰ Here, the reality and complexity of the experience of bees in the field is ignored, and the findings may be misleading.

The dominant toxicological epistemic form also tends to reduce the social history of the study location to a single point in time. Seeking to limit exposed bees to toxins, in order to produce conclusive results about the effects of these toxins, leads researchers to undertake their investigations in virgin field sites that do not contain insecticide residues (e.g., Cutler and Scott-Dupree 2007). However, in creating such artificial conditions, researchers ignore the possible effects on colony health from the accumulation over time of the newer systemic insecticides in crop fields (e.g., Bonmatin et al. 2005). Moreover, also with the goal of producing conclusive results, toxicological field research taking the dominant epistemic form and resulting regulatory policies have tended to rely largely on a single method of insecticide application—the seed coating of the neonicotinoid toxin, which involves a relatively small amount of the “active ingredient.” However, according to some beekeepers and toxicologists, adequate research “simply does not exist” on other methods of application seen in real-world conditions that use much higher amounts of active ingredient.¹¹

Apart from such design issues, toxicological ignorance is further shaped by measurement choices. Scientific studies typically compare the levels of “mean” and “variance” in measured parameters between the “untreated” and “treated” sets at “a 95 percent confidence interval” in order to assess the role of the pesticide/pesticides in question. Researchers judge the health of a colony by quantifying various parameters, such as the number of brood in specific developmental stages, the amount of stored nectar and pollen, and the number of frames “covered” with bees (e.g., Cutler and Scott-Dupree 2007). But just because a colony has an egg-laying queen, with large numbers of brood, honey, and pollen does not mean that it is healthy.¹² Such formal measures tend to overlook more qualitative and arguably equally important information pointed to by beekeepers, such as variations in “brood pattern” that can give alternative insights into brood development.

Formal measurements are further limited by the shared awareness among several beekeepers and bee researchers that a honey bee colony is a “superorganism.” In other words, colonies can respond in different compensatory ways to the same environmental perturbation, in this case from the neonicotinoids.¹³ This differential compensatory ability could further lead to different responses being measured by different studies. Researchers attempt to overcome variability between colonies by starting with a large number of colonies (sample size). However, as we noted earlier, the statistical requirement of 95 percent confidence that researchers impose upon experimental studies of honey bee colonies means that experimental studies tend to prefer Type II errors (false negatives), biasing conclusions toward “no differences” between treated and untreated honey bee colonies, when in fact there might be. Here, the established epistemic form leads researchers to ignore inconclusive results and the knowledge they embody. In sum, methodological choices and practical assumptions can lead researchers to fail to explore certain causal connections between factors that could plausibly contribute to CCD. This not only leads to knowledge gaps but also to knowledge, which can, in some sense, be false or misleading (Smithson 1985). This distorting dimension of ignorance derives from the extrapolation of laboratory-based approaches to the actual settings that commercially managed honey bees face (see Frickel and Edwards, Forthcoming). Further, the inability of the dominant disciplinary form to resolve the environmental complexity in which honey bee colonies operate creates a situation of “undoable science” (Frickel et al. 2010). To the limited extent that they are doable, this epistemic form of honey bee toxicology leaves crucial issues such as the effects of bioaccumulation of neonicotinoids and synergies with other environmental factors “undone” (Frickel et al. 2010). The production of these kinds of ignorance is maintained, to a large part, by the interests, stakes, and norms that honey bee toxicologists face in academic settings.

A final set of factors in producing ignorance among toxicologists doing bee research is the norms, structure of opportunities, and prestige scales among academic scientists. The pressures of securing peer-reviewed publications, grant funding, faculty appointments, and tenure reinforce the orientation of academic scientists toward adopting epistemic forms that facilitate the production of conclusive knowledge from the standpoint of dominant actors in the field. The high career stakes involved in producing definitive knowledge means that academic honey bee toxicologists would tend to make methodological choices that are more likely to show measurable “positive” effects from apparently isolatable causes. A reductive experimental form that considers individual chemicals at higher dose levels is more likely to do so than a cocktail of chemicals at very low doses. Conversely, serious consideration of low levels of toxins in interactions with multiple other factors entails a higher risk of failure than more reductive approaches, because there is a higher probability of getting inconclusive results, which are unlikely to be publishable in peer-reviewed scientific journals (Csada, James, and Espie 1996). As a result, there is a decreased incentive for academic honey bee toxicologists to consider complex real-world issues such as the cumulative effects of toxic synergies that involve low doses of neonicotinoids.

According to Dr. James Frazier, a leading academic insect toxicologist, honey bee scientist, and scientific advisor to the commercial beekeepers’ NHBAB, there is currently no appropriate toxicology to assess the realistic effects of the neonicotinoids on honey bee colonies in field settings. Consequently, experimental design forms that push the very limits of traditional scientific inquiry would be needed in order to encompass the multitude of variables and factors that could affect colony health in field.¹⁴

Regulating Scientists, Bees, and Beekeepers: “Good Laboratory Practice”

The ignorance that results from the institutionalized epistemic form on the basis of which bee toxicologists organize their research is reinforced by EPA policy. So-called Good Laboratory Practice (GLP) creates barriers to potentially innovative forms of experimental toxicology and prompts the creation of systematic “regulatory knowledge gaps” (Frickel and Vincent 2007). GLP specifies how experiments should be designed, performed, tracked, recorded, and reported, and by whom, in order for the results to be usable in federal rulemaking (Editor 2010). GLP calls for traditional approaches to isolating potential causal variables and to establishing experimental controls. In order to be deemed GLP-compliant, approaches have to be “validated” by scientific regulatory bodies via “consensus building” processes that involve researchers from academia and agrochemical industry (Editor 2010, 1104). Thus, in the case of CCD, the EPA does not require that pesticide manufacturers analyze the sublethal and chronic effects of insecticides on mature honey bee adults and immature brood during the registration process. This is the case even though multiple laboratory studies have reported adverse effects from sublethal doses of neonicotinoids on individual development, learning, and communicative abilities (Reviewed in Desneux, Decourtye, and Delpuech 2007; Alaux et al. 2009), with the realistic potential to damage colony health and cause CCD. An EPA official involved in the “risk management” process for insecticides like imidacloprid suggested to us that the EPA ignored sublethal, chronic effects because of “the complexity of the [biological] process” that produces these effects. This official also pointed to “the lack of resources” to undertake the complicated analyses that would be required to understand these intricate biological mechanisms of toxicity and measure subtler sublethal effects.^15,
16 In other words, the EPA accepts ignorance as a necessary result of the use of an institutionalized epistemic form and resource constraints.

Two further factors make the EPA’s regulatory culture unfavorable for a serious consideration of pesticide effects on honey bee populations, and thus reinforce the production of ignorance on the possible role of pesticides in causing CCD. First, regulators have historically tended to suspend or limit the usage of a pesticide based on a notion of “imminent hazard,”¹⁷ which in practice refers more to human rather than nonhuman health.¹⁸ By and large, instances when the EPA did take a precautionary stance have revolved around the potential carcinogenic effects of the concerned chemicals on humans.^19,
20 This tendency is exacerbated in the post–1996 landscape of “reduced risk” pesticides, such as the newer systemics, which are thought to have lower mammalian toxicity than the traditional pesticides (EPA 1999). In other words, the relative safety of “reduced risk” pesticides to human health leads the EPA to fail to demand serious consideration of their potential negative effects on honey bee health.

Second, according to a senior risk evaluator and environmental toxicologist at the EPA’s Office of Pesticide Programs, the very notion of assessing risks to insects is relatively recent at the EPA, because insects have historically been regarded as “target” taxa—organisms to kill, not preserve.²¹ A historical goal to kill insects means that cumulative, sublethal, and interactive effects of pesticides across the honey bee life cycles have not, until very recently, been of interest to the EPA. Indeed, in keeping with the dominant toxicological epistemic form, the EPA’s toxicity tests on honey bees have been geared to target insect pests and emphasize relatively rapid, lethal effects.

Agrochemical Industry

Agrochemical industry firms have aided the production of ignorance on questions of the role of newer systemic insecticides in contributing to CCD. For firms such as Bayer CropScience—the world’s largest manufacturer of newer systemic insecticides—the stakes are clearly high. Insecticides such as imidacloprid and clothianidin are some of Bayer’s biggest sellers and crucial to increasing their share of the global pesticides market.²² Bayer, thus, has an interest in maintaining ignorance about any purported role of these chemicals in CCD. But in this case, Bayer does not need to resort to systematic data fabrication or fraudulent activities in order to preserve uncertainty (cf. Proctor 2008; McGoey 2012). It is enough simply for the company to capitalize on the ignorance produced by the toxicological epistemic form that dominates academic and regulatory evidentiary norms and practices. As long as uncertainty persists about their insecticides’ role/roles in CCD, the company’s chemicals can remain on the US market. It is in this context that we should understand Bayer’s own toxicological studies on honeybees, and their influence in shaping academic standards and practices.²³

Bayer shapes ignorance production through active participation in, and contributing to, processes of knowledge production in the epistemic form that has gained broad legitimacy in academic and regulatory settings of honey bee toxicology. Following the standards of this form, Bayer’s numerous field studies consistently conclude that legally allowed levels of their newer systemic insecticides have “no adverse effects” on honey bees “under natural conditions” (reviewed in Maus, Curé, and Schmuck 2003; Schmuck and Keppler 2003). Several of these studies are specifically geared to providing regulators with information required or helpful in registering the insecticidal product. Beyond this, Bayer’s toxicologists/ecotoxicologists disseminate their experimental studies and associated understandings about honey bees through formal and informal interactions with regulatory officials, university scientists, and beekeepers in conferences,²⁴ workshops,²⁵ trade magazines,²⁶ and peer-reviewed journals.²⁷ In the process, Bayer’s (published) research calls into question any purported role for their newer systemic insecticides in contributing to phenomena of adverse honey bee health, including CCD. Moreover, in doing so, Bayer scientists create toxicological standards, such as the LD₅₀ and the NOAEL, which define the relative range of their systemic insecticides’ doses, where they are considered to be lethal, sublethal, and safe to honey bees. By defining the dose ranges and standards, which determine where one does (not) see insecticidal effects on bees, Bayer indirectly shapes how subsequent experiments are designed and interpreted in academic and regulatory settings. This further increases the likelihood that academic and regulatory studies will reflect Bayer’s knowledge and ignorance in the CCD debate.

Conclusion

Frickel, Hess, Gross, McGoey, Böschen, and their collaborators initiated an important set of inquiries into the systematic production of ignorance in the contexts of state regulatory agencies, social movements, and scientific cultures (Frickel and Vincent 2007; Hess 2007; Böschen et al. 2010; Frickel et al. 2010; Gross 2010; McGoey 2007; 2012). We seek to build on this project. In the CCD controversy, we suggest that the historically established dominance of a particular epistemic form leads to the production of ignorance in three senses. First, we have shown that there is research that does not get done because the accepted epistemic form makes doing such research highly unlikely. Ignorance is the result of undoable science. Of course, to say this research is undone and undoable is not exactly correct. It is largely undone and undoable by certified scientists following the established epistemic form. It is, however, being done by commercial beekeepers, but these findings are ignored because they do not adhere to the dominant epistemic form. Second, we showed that the dominant epistemic form prevents certain crucial questions about sublethal, interactive, and cumulative effects from being thoroughly addressed. In failing to consider important factors and interactions, such work may be misleading, and misleading results amount to another variety of ignorance, called “false knowledge” by some analysts (Smithson 1985) and “factual ignorance” by others (McGoey 2007). Finally, we suggested that institutionalized norms lead researchers to dismiss inconclusive findings. Here, by not considering inconclusive results worthy of legitimate consideration, researchers prefer a variety of ignorance to a variety of knowledge. Importantly, the ignorance resulting from the epistemic form accepted by bee researchers is reinforced by the EPA’s guidelines and agrochemical corporate positioning. Significantly, in this context, some actors have greater capacity than others to affect what counts as knowledge, and what issues we remain ignorant about. Together, we might call these different dimensions of ignorance normatively induced ignorance.

Academic honey bee scientists, government regulators, and agrochemical industry officials together define what will count as knowledge and what as ignorance in the CCD controversy. These actors point to laboratory and field experiments that show a lack of conclusive evidence of a causal link between the neonicotinoid insecticides and the honey bee colony collapses. Typically, these actors privilege toxicological “data” over beekeeper “anecdotes.” They often assert or assume that traditional scientific, toxicological practices are superior, more unbiased mediators of the realistic effects of toxic chemicals on honey bees. However, as we have suggested, it is not the inherent superiority of dominant methods and measures that explain the state of knowledge and ignorance in the CCD controversy. Instead, the accepted form of toxicological practice reflects the history of the development of bee-related science and character of academic culture, the norms of US regulatory agencies, and the stakes and interests of powerful agrochemical actors.

Toxicological field experiments seek to understand the causal role of neonicotinoid insecticides on honey bees and their colonies in realistic conditions. The degree of precision and control necessary for producing this causal knowledge, from the standpoint of the established epistemic form, leads researchers to extrapolate from models established for laboratory settings onto a field context. This extension entails making methodological assumptions that overlook the cumulative effects of, and the complex interactions between, the neonicotinoids and other endemic environmental variables. The knowledge gaps produced by these methodological norms are further maintained by the practical career constraints of academic toxicologists. The high stakes involved in securing publications, grant funding, and tenure orient academic actors toward adopting practices that lead to conclusive results. The interests, stakes, and epistemic forms of honey bee toxicologists lead them to prefer to overlook an important causal factor, rather than bear the professional fallout from falsely implicating it. Consequently, they tend not to study complex scenarios, such as those proposed by the beekeepers’ alternative epistemic form, which has a high likelihood of leading to inconclusive results. The result is undone science.

On the flip side, the knowledge that is produced (done science) is poorly representative of the dynamic settings in which CCD first emerged, and in this sense can distort subsequent knowledge and nonknowledge production. In the case of disciplinary toxicological field studies, this distorting dimension of ignorance is a result of the extrapolation of an epistemic form that does not embody the settings in which CCD actually arose. The regulatory GLP framework further intensifies the existence of knowledge gaps by devaluing alternative, innovative disciplinary forms. Toxicological ignorance production is also accentuated by the indirect influence of agrochemical industry toxicologists, whose work is shaped by the dominant epistemic form, which, perhaps not incidentally, serves the interests of the firms they work for.

The prevalent forms of toxicological ignorance justify a shift in the dominant scientific consensus away from anthropogenic agricultural chemicals as the primary causal agents of CCD. In the “sound science” risk paradigm of the EPA, the ignorance further justifies a lack of regulatory action on the neonicotinoids. This ends up benefiting powerful agrochemical actors and does not serve the interests of commercial beekeepers.²⁸

Commercial beekeepers, whose livelihood and profit margins are at stake, are led to adopt knowledge practices, which are shaped by the local, multivariable settings that impinge upon the health and strength of honey bee colonies. The real-time, informal, and in situ characteristics of the epistemic form, which underlies their practices, provide knowledge that is meaningful and useful in the dynamic settings in which they ply their trade, but is illegitimate in the worlds of professional honey bee toxicologists and government regulators. Such an epistemic form does not lend itself to isolatable and generalizable causal explanations, but rather to loosely connected, contextual correlations. Commercial beekeepers’ interests, stakes, and associated epistemic form lead them toward a precautionary policy approach, wherein they prefer to falsely implicate a correlated factor than to bear the realistic risk of ignoring the uncertainty regarding its effects. Accordingly, the NHBAB, a national association of commercial beekeepers, has called for a significant limitation or suspension of the commercial usage of these newer systemic insecticides, until agrochemical industry and academic actors fill knowledge gaps regarding the complex interactions of these insecticides with other environmental factors and their chronic, long-term effects. These are precisely the kinds of knowledge gaps that academic and corporate bee toxicologists are not led to fill, and their adherence to the dominant epistemic form leads to varieties of ignorance.

While surely serious consideration of beekeeper knowledge and knowledge production methods will not, by itself, lead to a complete understanding of CCD, the insights their observations may offer, and the livelihood stakes they have in a resolution to the disorder, suggest all parties ought to give the claims and epistemic form of beekeepers more weight than they have heretofore. Serious attention to “on the ground” epistemic forms of knowledge-gathering communities, which embody the actual settings in which the phenomena of technoscientific concern arise, would shift the terrain of done and undone science. Such epistemic forms tend to include practices that incorporate qualitative as well as quantitative measures, are less controlled, are correlative, and give preference to false positives over false negatives. Since they go against established dominant epistemic commitments, there would be a significant disincentive for traditional academic actors to adopt such epistemic forms. However, their utility would arguably be justified when dealing with crucial technoscientific phenomena such as CCD, which place at risk ecological, socioeconomic and human well-being. Their design and evaluation would be interdisciplinary and trans-disciplinary, involving not only disciplinary scientific actors but also actors versed in the social and political dimensions of the issue, as well as the meaningful participation of less-powerful stakeholders. At the same time, these alternative epistemic forms would prompt a reorientation of regulatory policy toward a precautionary approach that does not ignore what we do not know (Magnus 2008).

Footnotes

Acknowledgments

We thank David Hess, Scott Frickel, and two anonymous reviewers for suggesting ways to improve the article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by the National Science Foundation (Award # 0924346). Daniel Lee Kleinman's work is also supported by the National Research Foundation of Korea (NRF-2010-330-B00169).

Notes

References

Alaux

Brunet

J.-L.

Dussaubat

Mondet

Tchamitchan

Cousin

Brillard

Baldy

Belzunces

L. P.

Le Conte

. 2009. “Interactions between Nosema Microspores and a Neonicotinoid Weaken Honeybees (Apis mellifera).” Environmental Microbiology 12 (3): 774–82.

Altieri

M. A.

Nicholls

C. I.

. 2005. Manage Insects on Your Farm: A Guide to Ecological Strategies. Beltsville, MD: Sustainable Agriculture Network Handbook Series, Book 7.

Anderson

L. D.

Atkins

E. L.

., Jr. 1968. “Pesticide Usage in Relation to Beekeeping.” Annual Review of Entomology 13: 213–238.

Barrionuevo

. 2007. “Honeybees Vanish, Leaving Keepers in Peril.” The New York Times February 27. Accessed February 15, 2010. www.nytimes.com/2007/02/27/business/27bees.html?sq=colony%25.

Bayer

CropScience

. 2010. “Prosper FX-Its What All the Best Seeds are Wearing.” Crop Talk 18 (4): 2.

Bennett

S. H.

1957. “The Behaviour of Systemic Insecticides Applied to Plants.” Annual Review of Entomology 2: 279–96.

Bonmatin

J. M.

Moineau

Charvet

Colin

M. E.

Fleche

Bengsch

E. R.

. 2005. “Behaviour of Imidiacloprid in Fields. Toxicity for Honey Bees.” In Environmental Chemistry: Green Chemistry of Pollutants in Ecosystems, edited by Lichtfouse

Schwarzbauer

Robert

, 483–94. Heidelberg: Springer-Verlag.

Böschen

Kastenhofer

Rust

Soentgen

Wehling

. 2010. “The Political Dynamics of Scientific Non-Knowledge.” Science, Technology, & Human Values 35 (6): 783–811.

Brickman

Jasanoff

Ilgen

. 1985. Controlling Chemicals: The Politics of Regulation in Europe and the United States. Ithaca, NY: Cornell University Press.

10.

Bromenshenk

J. J.

Henderson

C. B.

Wick

C. H.

Stanford

M. F.

Zulich

A. W.

Jabbour

R. E.

Deshpande

S. V.

McCubbin

P. E.

Seccomb

R. A.

Welch

P. M.

Williams

Firth

D. R.

Skowronski

Lehmann

M. M.

Bilimoria

S. L.

Gress

Wanner

K. W.

Cramer

R. A.

., Jr. 2010. “Iridovirus and Microsporidian Linked to Honey Bee Colony Decline.” PloS ONE 5 (10): e13181.

11.

Cox-Foster

D. L.

Conlan

Holmes

E. C.

Palacios

Evans

J. D.

Moran

N. A.

Quan

P. L.

Briese

Hornig

Geiser

D. M.

Martinson

vanEngelsdorp

Kalkstein

A. L.

Drysdale

Hui

Zhai

Cui

Hutchison

S. K.

Simons

J. F.

Egholm

Pettis

J. S.

Lipkin

W. I.

. 2007. “A Metagenomic Survey of Microbes in Honey Bee Colony Collapse Disorder.” Science 318 (5848): 283–7.

12.

Csada

R. D.

James

P. C.

Espie

R. H. M.

. 1996. “The “File Drawer Problem” of Non-Significant Results: Does It Apply To Biological Research?” Oikos 76 (3): 591–3.

13.

Cutler

C. G.

Scott-Dupree

C. D.

. 2007. “Exposure to Clothianidin Seed-Treated Canola Has No Long-Term Impact on Honey Bees.” Journal of Economic Entomology 100 (3): 765–72.

14.

Desneux

Decourtye

Delpuech

J.-M.

. 2007. “The Sublethal Effects of Pesticides on Beneficial Arthropods.” Annual Review of Entomology 52: 81–106.

15.

EPA (US Environmental Protection Agency). 1999. Implementing the Food Quality Protection Act: Progress Report. Report No. 735-R-99001. Washington, DC: EPA, Office of Pesticide Programs.

16.

Fischer

Moriarty

. 2011. Pesticide Risk Assessment For Pollinators: Summary of a SETAC Pellston Workshop. Pensacola, FL: Society of Environmental Toxicology and Chemistry (SETAC) Press.

17.

Frickel

Edwards

. Forthcoming. “Untangling Ignorance in Environmental Risk Assessment.” In Powerless Science? The Making of the Toxic World in the Twentieth Century, edited by Boudia

Jas

, 1-29. Oxford and New York: Berghahn Books.

18.

Frickel

Gibbon

Howard

Kempner

Ottinger

Hess

D. J.

. 2010. “Undone Science: Charting Social Movement and Civil Society Challenges to Research Agenda Setting.” Science, Technology, & Human Values 35 (4): 444–73.

19.

Frickel

Vincent

M. B.

. 2007. “Katrina, Contamination, and the Unintended Organization of Ignorance.” Technology in Society 29 (2): 181–8.

20.

Gross

2010. Ignorance and Surprise: Science, Society and Ecological Design. Cambridge, MA: The MIT Press.

21.

Haraway

D. J.

1991. “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective.” In Simians, Cyborgs, and Women, edited by J

Donna

. Haraway, 183–201. New York: Routledge.

22.

Harding

2000. “Should Philosophies of Science Encode Democratic Ideals?” In Science, Technology, and Democracy, edited by Kleinman

D. L.

, 121–38. Albany: State University of New York Press.

23.

Hess

D. J.

2007. Alternative Pathways in Science and Industry: Activism, Innovation, and the Environment in an Era of Globalization. Cambridge, MA: MIT Press.

24.

Holstein

H. S.

2009. “Manufacturing Doubt: Journalists’ Roles and the Construction of Ignorance in a Scientific Controversy.” Public Understanding of Science 18 (1): 23–42.

25.

Johnson

R. M.

Evans

J. D.

Robinson

G. E.

Berenbaum

M. R.

. 2009. “Changes in Transcript Abundance Relating to Colony Collapse Disorder in Honey Bees (Apis mellifera).” Proceedings of the National Academy of Sciences 106 (35): 14790–5.

26.

Knorr-Cetina

1999. Epistemic Cultures. Cambridge: Harvard University Press.

27.

Lowe

V. H.

Parrott

P. J.

. 1902. San José Scale Investigations. Geneva: New York Agricultural Experimental Station. Bulletin No. 228.

28.

Magnus

2008. “Risk Management versus the Precautionary Principle: Agnotology as a Strategy in the Debate over Genetically Engineered Organinsms.” In Agnotology: The Making & Unmaking of Ignorance, edited by Proctor

Robert N.

Schiebinger

Londa

, 250–65. Stanford, CA: Stanford University Press.

29.

Maus

Curé

Schmuck

. 2003. “Safety of Imidacloprid Seed Dressings to Honey Bees: A Comprehensive Review and Compilation of the Current

State of Knowledge.”

Bulletin of Insectology 56 (1): 51–7.

30.

McGoey

2007. “On the Will to Ignorance in Bureaucracy.” Economy and Society 36 (2): 212–35.

31.

McGoey

2012. “Strategic Unknowns: Towards a Sociology of Ignorance.” Economy and Society 41 (1): 1–16.

32.

Morse

R. A.

Calderone

N. W.

. 2000. “The Value of Honey Bees as Pollinators of U.S. crops in 2000.” Bee Culture 128 (3): 1–15.

33.

Mullin

C. A.

Frazer

Frazier

J. L.

Ashcroft

Simonds

vanEngelsdorp

Pettis

J. S.

. 2010. “High Levels of Miticides and Agrochemicals in North American Apiaries: Implications for Honey Bee Health.” PloS ONE 5 (3): e9754.

34.

Nature. 2010. “The Weight of Evidence.” Nature 464 (7292): 1103-4.

35.

NRC (National Research Council). 2007. Status of Pollinators in North America. Washington, DC: The National Academies Press.

36.

Palladino

1996. Entomology, Ecology and Agriculture: The Making of Scientific Careers in North America 1885-1985. Amsterdam: Harwood Academic Publishers.

37.

Proctor

R. N.

2008. “A Missing Term to Describe the Cultural Production of Ignorance (and Its Study).” In Agnotology: The Making & Unmaking of Ignorance, edited by Proctor

R. N.

Schiebinger

, 1–33. Stanford, CA: Stanford University Press.

38.

Schmuck

Keppler

. 2003. “Clothianidin-Ecotoxicological Profile and Risk Assessment.” Pflanzenschutz-Nachrichten Bayer 56 (1): 26–58.

39.

Smithson

1985. “Toward a Social Theory of Ignorance.” Journal for the Theory of Social Behaviour 15 (2): 151–72.

40.

Spivak

2010. “The Status of the European Honey Bee in the U.S.” In Managing Alternative Pollinators: A Handbook for Growers, Beekeepers and Conservationists, edited by Mader

Spivak

Evans

, 15–24. Ithaca, NY: Natural Resource, Agriculture and Engineering Service.

41.

Stokstad

2007. “The Case of the Empty Hives.” Science 316(5827): 970–73.

42.

Suryanarayanan

Kleinman

D. L.

. 2011. “Disappearing Bees and Reluctant Regulators.” Issues in Science and Technology 27 (4): 31–6.

43.

USDA (US Department of Agriculture). 1907. Miscellaneous Papers on Apiculture. Washington, DC: USDA, Bureau of Entomology, 1907-1909. Bulletin No. 75.

44.

vanEngelsdorp

Cox-Foster

Frazier

Ostiguy

Hayes

. 2007. “Fall-Dwindle Disease: Investigations Into the Causes of Sudden and Alarming Colony Losses Experienced by Beekeepers in the Fall of 2006-Preliminary report: First revision.” Accessed March 1, 2010. www.doacs.state.fl.us/pi/plantinsp/apiary/fall_dwindle_report.pdf.

45.

vanEngelsdorp

Meixner

M. D.

. 2010. “A Historical Review of Managed Honey Bee Populations in Europe and the United States and the Factors That May Affect Them.” Journal of Invertebrate Pathology 103 (1): S80–S95.