Preventing Biological Weapon Development Through the Governance of Life Science Research

Cold War Science and Security Concerns

During the Cold War, the United States' principal security concern about the publication of scientific research was that it not help the Soviet Union erode U.S. advantages in militarily relevant science and technology. Tensions between the scientific and security communities in the late 1970s and early 1980s led the National Academy of Sciences to conduct a study on scientific communication and national security under the leadership of Cornell University President Dale Corson. ⁴ This study, published in 1982, formed the basis for a directive issued by President Reagan in 1985 that recognized that the free exchange of ideas was crucial to American science, which in turn underpinned U.S. national and economic security. ⁵ The directive, National Security Decision Directive or NSDD 189, specified that as far as possible, fundamental research results should remain unrestricted, and that where controls were necessary, the control mechanism should be security classification. It said that “no restrictions may be placed upon the conduct or reporting of federally-funded fundamental research that has not received national security classification” unless other provisions of law directed otherwise. ⁵ As a 2005 study on science and security stated,

This Directive does not assert that the open dissemination of unclassified research is without risk. Rather, it says that openness in research is so important to our own security – and to other key national objectives – that it warrants the risk that our adversaries may benefit from scientific openness as well.^6(p2)

This directive is still in force, having been reiterated by senior government officials in both the George W. Bush and Obama administrations.

Self-Governance and the Asilomar Process

Restrictions such as NSDD-189 on the federal government's ability to restrict scientific information do not prevent the scientific community from exercising self-governance in reviewing proposals for experiments that raise dual-use issues, or in communicating results. Indeed, the life sciences community has a precedent with self-governance in the way it addressed potential safety and environmental dangers in the field of genetic engineering.

At a conference in Asilomar, California, in February 1975, scientific leaders addressed risks that might result from moving genes from one organism to another, containment procedures that might mitigate these risks, and the conditions under which a voluntary moratorium that had been instituted on such recombinant DNA experiments might be lifted. At the conclusion of the meeting, a new NIH committee, the Recombinant DNA Advisory Committee (RAC), was convened and given the responsibility for generating guidelines to govern all recombinant DNA work at institutions that receive any federal funds for that purpose. These guidelines ⁷ are updated as new scientific judgments on risk are made, and they have become a de facto international standard. Despite early attempts by some in Congress, these guidelines have never been codified in law or regulation, making them much easier to update when appropriate.

“Contentious Research”

The governance structure developed after the Asilomar meeting, including the formation of the NIH RAC and its associated guidelines, creation of Institutional Biosafety Committees for local review, and the sharing of oversight responsibility between these local committees and the federal government, was established to guard against accidental or unintentional harm to health and the environment resulting from recombinant DNA research. It did not, and was never intended to, address the malevolent use of research results. In fact, security considerations have never featured prominently in biological research in the U.S. Even prior to 1968, when the U.S. had an offensive biological weapons program, only a very small fraction of U.S. life scientists worked in that program or elsewhere in the national security community. The vast majority of U.S. life science research has been funded by civilian agencies, especially the National Institutes of Health, and nongovernment entities, such as the Howard Hughes Medical Institute, and very few life scientists addressed the security implications of their work.

One important analysis that did address the security implications of advances in biotechnology was initiated in 2000 by the Defense Science Board (DSB), as the Department of Defense grappled with its role in supporting civilian responses to mass consequence terrorism.

Task Force on Defense Against Biological Weapons

The DSB task force on Defense Against Biological Weapons ⁸ recognized the possibility that adversaries might seek to attack the U.S. with biological weapons, and that advances in genetic engineering and biotechnology could be exploited to cause unprecedented levels of damage. In outlining the ways that technologies could both aggravate as well as ameliorate the threat, some members of this task force believed that “information about microbial virulence that could assist our enemies is already entering the public scientific literature without adequate constraints.”^9(p54) The task force recommended that:

DoD, in concert with others in the public and private sector, should assess whether open-source information on the genetic mechanisms of microbial virulence and biotechnology advances related to the production and manipulation of microorganisms presents an increasing security risk and whether certain categories of “bio” information generated by academia and industry should be classified.^9(p86)

In response, the White House Office of Science and Technology Policy convened a meeting of senior government technical officials to consider the national security implications of fundamental biological research and see if there was sufficient cause to reexamine longstanding U.S. government policy against restricting the conduct and dissemination of fundamental research. ²

Meeting participants acknowledged that NSDD-189 was not drafted with the development of new biological threats in mind, but they did not yet see sufficient cause to revise it. They recognized that the government has extremely limited power to classify privately generated information. Much of the information that could pose concern was already in the public domain, and the materials and equipment needed to rederive such information were widely available, making classification of limited utility at best. However, meeting participants also recognized the importance of ongoing discussions with the scientific community on risks that might emerge from biological research and on options for mitigating them.

The Australian Mousepox Paper

One of the first examples of microbiological research that brought dual-use issues to the attention of audiences well beyond the scientific community was the 2001 publication of an experiment with a mousepox virus that is closely related to the virus that causes smallpox in humans. ¹⁰ Testing a genetically modified variant of mousepox initially intended as a contraceptive, Australian investigators found that it not only killed all mice that were normally resistant to the virus, but it also killed over half the mice that had been vaccinated against it. The concern was whether the virus causing smallpox in humans could be transformed in a similar way, producing a devastating biological weapon against which there would be no defense. At the time, no process was in place to decide whether this experiment should have been attempted with the mousepox, or what, if anything, should have been done with its surprising results.

U.S. National Academy of Sciences Activities

Prior to the increased emphasis on mass destruction terrorism adopted by the Clinton administration in the late 1990s and overwhelmingly reinforced by the confluence of the September 2001 terrorist attacks and the October 2001 anthrax letters, biological weapons were addressed largely as an issue involving nation states. In 1986, the National Academy created a Working Group on Biological Weapons Control within its Committee on International Security and Arms Control (CISAC) to examine improved methods for verifying compliance with the Biological Weapons Convention, the international treaty that bans the development, production, acquisition, and stockpiling of biological weapons.† By the late 1990s, this working group had begun a series of meetings and consultations on an international approach to dealing with biological weapons research, a particular interest of the group's vice chair, John Steinbruner. ¹¹ Focusing on deterring or discovering research deliberately intended to develop weapons, Steinbruner had proposed governance mechanisms such as international registration and oversight of all research with highly dangerous pathogens. ¹²

However, at a planning meeting for its international approach in June 2001, CISAC encountered pressing domestic concerns. A recent report from the National Commission on Terrorism had called on the U.S. government to implement physical protection and security standards for pathogens in research laboratories that were as stringent as those used for nuclear weapons materials. ¹³ Some meeting participants were concerned that Congress might enact legislation as a result that would have the effect of dramatically inhibiting medical and life science research. ¹¹

In response to these concerns, the National Academies prepared to initiate a project to review current U.S. mechanisms to prevent the destructive application of biotechnology research and to recommend improvements to those mechanisms that would still permit legitimate research to proceed. An Academy panel chaired by MIT geneticist Gerald Fink (thereby referred to as the Fink Committee) held its first meeting on April 1, 2002. Taking place after the September 2001 attacks and the October 2001 anthrax mailings, this meeting was held in a very different environment than the period in which the study had been conceived.

Synthesis of Poliovirus

Soon after the Fink Committee began its work, one of the most politically charged examples of the type of contentious research the committee was formed to address was published: the synthesis of the poliovirus, a human pathogen, by scientists at the State University of New York, led by Eckhard Wimmer. ¹⁴ Wimmer and his collaborators created an infectious virus without the use of a preexisting virus particle as a template or the involvement of a living cell. Emphasizing that this synthesis was chemical, not biological, the investigators labeled the poliovirus particle with its chemical formula: C_332,652 H_492,388 N_98,245 O_131,196 P₇₅₀₁ S₂₃₄₀.

Coming less than a year after the September 11 attacks and the October 2001 anthrax mailings, this publication attracted considerable attention. Some scientists derided it as “inflammatory” or “a stunt.” ¹⁵ More ominous for the scientific community was the introduction of a resolution by 8 members of the U.S. House of Representatives expressing serious concern about the publication of “a blueprint that could conceivably enable terrorists to inexpensively create human pathogens for release on the people of the United States.” ¹⁶ Even though no further action was taken on this resolution, it did provide a powerful illustration of how the political process can become alarmed by scientific developments that appear to offer more risk than benefit.

Statement on Scientific Publication and Security

After the September 2001 terrorist attacks and the anthrax mailings, the American Society of Microbiology (ASM) confronted the issue of whether it should withhold details from scientific publications to reduce the risk of misuse by those with malicious intent. Such a decision would effectively redefine what constituted science, since the ability to independently reproduce scientific results has long been considered an essential component of the scientific process. Not wanting to act unilaterally, in July 2002 ASM President Ronald Atlas asked Bruce Alberts, President of the National Academy of Sciences, to organize a meeting for life science publishers to discuss practices for an environment in which “there is increasing concern that technical articles may inadvertently aid those engaged in acts of terrorism.” ¹⁷ Dr. Alberts agreed, ¹⁸ in return asking Dr. Atlas to chair the workshop's steering committee.

The National Academy of Sciences brought the Center for Strategic and International Studies (CSIS), which had recently completed a study of science and security at the U.S. Department of Energy's national laboratories, ¹⁹ into this process, and the 2 organizations jointly convened a conference on “Scientific Openness and National Security” on January 9, 2003. ²⁰ Parney Albright, a senior official with the White House Office of Science and Technology Policy who spoke at the conference, urged the scientific community to be proactive, explaining that “the science community ought to come up with a process before the public demands the government do it for them, and that will be driven by the rate at which controversial papers hit the streets.”^21(p197)

The NAS/CSIS symposium explored the security sensitivities raised by life science research in some depth, but it did not derive principles for the review and possible control of such research. The following day, however, editors and publishers of leading life science journals met with a number of symposium participants to discuss what would constitute “sensitive or dangerous information” in the life sciences, and what steps, if any, journals should take to lessen the risk that publication of such information could be misused to facilitate terrorism. ²² This meeting resulted in a Statement on Scientific Publication and Security that was published by Science, Nature, and the Proceedings of the National Academy of Sciences. ²³ The statement emphasized the importance of the scientific publication process, both to ensure quality and to communicate results, and it called attention to the need to “protect the integrity of the scientific process by publishing manuscripts in sufficient detail to permit reproducibility.”^23(p1149) However, it also acknowledged that “there is information that, although we cannot now capture it with lists or definitions, presents enough risk of use by terrorists that it should not be published.”^23(p1149) It called for editors and publishers to implement effective review procedures for papers raising security concerns.

This statement was a key articulation of principle. However, it provided little guidance to those who must decide what information should not be published, acknowledging that “[h]ow and by what processes it might be identified will continue to challenge us.”^23(p1149) This challenge is complicated by the importance that the information of concern could have for promoting public health and for countering the very terrorism it might be thought to facilitate.

Some commentators picked up on the lack of operational procedures. Falkow, for example, criticized the absence of more specific guidelines that could help individual investigators decide what information might be too sensitive to publish. ²⁴ Investigators should be making such judgments, in Falkow's view, and not editors or reviewers. Donald Kennedy, editor of Science and a member of the group that had drafted the statement, responded that the set of potential harms that editors needed to anticipate was too broad to capture in a set of guidelines, but also that changes to or rejections of papers were expected to be very infrequent and would be subject to a highly consultative process. ²⁵

The Fink Report

In October 2003, the National Academy of Sciences released the Fink Committee's report, Biotechnology Research in an Age of Terrorism. ²⁶ This study has become the seminal document in addressing the national security risks of life science research. It reviewed the regulatory environment for life science research as well as information restriction and control regimes as applied in other domains of science and technology. The report recognized that no processes existed to review whether the tools, technology, or knowledge generated in life science research posed risks of misuse that outweighed their potential benefits. It recommended enhancing the existing regulatory and oversight system to address these risks while permitting science to advance. Some of its specific recommendations included the following:

Review of Plans for Experiments

The Fink Report called for review of 7 categories of experiments that raised particular concerns about misuse.‡ All of these “experiments of concern” involved infectious disease agents, and all were deemed to be feasible at the time or with projected near-term advances. According to the report, these experiments should already have been subject to review by the Institutional Biosafety Committees (IBCs) that had been established to comply with the NIH guidelines for recombinant DNA research. The report recommended that these IBCs be given the additional task of weighing the potential benefit of these experiments against their risk of misuse, referring those that required further scrutiny to the NIH Recombinant DNA Advisory Committee (RAC) or to the NIH Director to approve or deny. The Fink Committee recognized that IBCs would require additional education and expertise to be able to address dual-use concerns and suggested that the RAC develop guidelines, but otherwise it provided no guidance as to how the future risks and benefits of dual-use life science research would be assessed or compared.

Publication Review

Unlike the review of experiments, for which the Fink Committee anticipated that the federal government would assume an advisory or even a decision-making role, the Fink Committee rejected a federal role in reviewing scientific publications for national security risks. Consistent with the Editors' and Authors' Statement, ²³ it urged relying on self-governance by scientists and scientific journals, and it strongly discouraged government imposition of “sensitive but unclassified” information controls in the life sciences. ²⁷ Mandatory security controls on life sciences would be cumbersome, expensive, and unlikely to yield security benefits, and the chilling effect on biodefense research of “inevitably general and vague categories” of information control was judged to be a significantly greater danger to national security than the risk of inadvertent publication of potentially dangerous results. ²⁶

National Science Advisory Board for Biodefense

The Fink Report called for the creation of a high-level National Science Advisory Board for Biodefense (NSABB), consisting of leading scientists and national security experts, to serve as a resource for the oversight system's self-governing aspects and to periodically review and recommend revisions for the regulatory aspects. ²⁶

Educating the Scientific Community

In emphasizing self-governance solutions, the Fink Report recognized that life scientists, professional societies, and related organizations had to be made aware of the nature of the dual-use dilemma in biology and of their responsibilities to mitigate dual-use risks. Individuals cannot be expected to ensure that their research is never misapplied, according to the report, but they do have the responsibility to take reasonable steps to prevent it. It would be up to the research community to define what those reasonable steps are, and that could happen only if researchers were aware of the risks and could draw on the experience of others in dealing with them.

Venter's Chemical Synthesis of a Virus

The Fink Committee report was released at an opportune time. As it was being completed, Craig Venter and colleagues at the Institute for Biological Energy Alternatives were preparing a publication on the chemical synthesis of bacteriophage φX174, a virus that infects bacteria. Recognizing that this new technology could make it possible to synthesize deadly viral pathogens in a matter of weeks, Venter and his funders at the U.S. Department of Energy brought these results to the attention of the White House prior to publication. ²⁸ Some officials there thought that the results merited being classified on national security grounds. During discussions, U.S. government officials learned of the Fink Committee and its task to address just such issues as Venter's results raised. The fact that the scientific community was already addressing dual-use concerns, and that a major National Academies report on the topic was about to be released, helped ease concern about the Venter publication. The Department of Energy announced Venter's accomplishment at a press conference on November 13, 2003, and Venter's paper was published the next month. ²⁹

The National Science Advisory Board for Biosecurity (NSABB)

The federal government's response to the Fink Report was to create the recommended new advisory board and have that board consider the rest of the recommendations. In March 2004, senior government officials announced the formation of a National Science Advisory Board for Biosecurity to advise all federal agencies involved with dual-use life science research on topics that included guidelines and strategies for biosecurity oversight, communication of potentially sensitive life science research, education and training in biosecurity issues, and development of codes of conduct for life scientists. ³⁰

The NSABB was to include members with expertise in a wide range of life science and laboratory disciplines as well as national security, law and law enforcement, ethics, industry, scientific publishing, and “‘public’ perspectives.” It was also to include ex officio members from a wide range of federal agencies.

More than a year passed, however, before the board's first meeting, which occurred on June 30 and July 1, 2005. Five workgroups were formed on the topics of criteria for identifying dual-use research, codes of conduct for the life sciences, communication of dual-use research, international perspectives and collaboration, and the chemical synthesis of viral and bacterial genomes. ³¹ A sixth working group on the oversight of dual-use research was added within a year. As a federal advisory committee, the NSABB gives advice to the U.S. government but does not speak for the government or make policies on its behalf. Nevertheless, its reports comprise the most authoritative analyses of issues related to the oversight and communication of dual-use research in the United States, and its recommendations form the starting point for the development of policy by U.S. government agencies.

Synthesis and Study of 1918 Influenza

Soon after the NSABB first met, Secretary of Health and Human Services (HHS) Michael Leavitt asked it to review a pair of imminent publications describing U.S. government–funded research on the influenza virus responsible for the devastating pandemic of 1918. Although the virus was extinct in nature, scientists had recreated its genetic code and reconstructed it to study what made it so virulent.

The publications under review described the virus's effects and made public those parts of its genetic code that had not previously been published. Meeting in closed session because of the unpublished nature of the manuscripts, the NSABB evaluated the scientific and public health significance of the research, the risks of its misuse, the benefits of communicating the research to other scientists and the public, and the consequences of restricting public access to that information. It then unanimously recommended publication. However, it also recommended adding information to emphasize both the protective safeguards that had been taken in handling the virus and the public health benefits of the research. ³² The articles appeared in the journals Science and Nature during the week of October 3, 2005,^33,34 with the Science publication accompanied by an editorial by Nobel Laureate Phillip Sharp that described the NSABB review, explained the benefits of publication, and concluded that the review showed that “the system is working.”^35(p17)

Donald Kennedy, the editor of Science, was less impressed with the system. The NSABB review was triggered when Science flagged the paper as raising security issues and had urged the authors to consult with the heads of the Centers for Disease Control and Prevention and the National Institute for Allergy and Infectious Diseases. Almost 2 weeks later, immediately before the issue containing the 1918 flu article was to be sent to the printer, word came back that the HHS Secretary had requested the NSABB to review it—even though the NSABB's charter did not extend to the review of individual papers. Had the NSABB not approved the paper's publication, Kennedy said, Science would have published it anyway, short of a government classification order. ³⁶

The publication and the research proved to be quite controversial. Some scientists and commentators asserted that the danger of the resurrected virus's accidental release or malicious reconstruction exceeded its scientific value. ³⁷ But supporters and opponents of publication both agreed that the last-minute timing of the review was unfortunate. It is not clear whether the security implications of this research were reviewed in any way when these studies were initiated.

Oversight Frameworks

University of Maryland's Biological Research Security System

Two proposals for dual-use review systems that would evaluate research projects before they were initiated were released early in 2007. One was developed by the Controlling Dangerous Pathogens Project at the University of Maryland, headed by John Steinbruner. ³⁸ That project's proposed Biological Research Security System would evaluate both deliberate and inadvertent dangers resulting from the conduct of a research project. Under the principle that individuals or research groups should not have authority to make research decisions “that might put an appreciable fraction of the human species as a whole at risk”^38(p6) without independent oversight, this system would subject research projects to scrutiny from those with technical expertise as well as those in a position to judge social consequences. The Steinbruner study criticized the Fink Report and the NSABB's evolving oversight approach (as described in interim briefings) for defining experiments of concern too broadly, ignoring the risk of inadvertent harm, and failing to differentiate among degrees of risk. The study outlined a proposed oversight system that would create 3 risk-based tiers of activities involving pathogens: activities judged to be of “extreme concern” that would be subject to international oversight and approval; activities of “moderate concern” that would be regulated by national bodies; and activities of “potential concern” that would be overseen locally. Such a system would be legally binding on all relevant research—including any classified research—regardless of funding source. Both the institutions and the researchers performing regulated research activities would need to be licensed. Oversight at the national and local levels would be based on harmonized criteria, much as institutional reviews of experiments that use animals or human subjects are conducted today.

NSABB's Oversight Framework

Shortly after release of the Maryland report, the NSABB approved the transmittal to the U.S. government of its own proposed oversight framework for dual-use life science research. ¹ Its focus was on “dual-use research of concern (DURC),” defined as “[r]esearch that, based on current understanding, can be reasonably anticipated to provide knowledge, products, or technologies that could be directly misapplied by others to pose a threat to public health and safety, agricultural crops and other plants, animals, the environment, or materiel.”^1(p17) The NSABB wanted to capture research for which misapplication of results could have broad consequences, a risk that was not being addressed by U.S. government policy. Its oversight framework did not deal with the risks of accidental release or deliberate misappropriation of dangerous pathogens themselves, which were governed through existing mechanisms (ie, HHS guidance ³⁹ and the Select Agent Program,§ respectively).

The NSABB delineated 7 categories of research, based on the Fink Report's experiments of concern, that would likely constitute DURC.∥ But those categorizations were just a starting point; not all such research would constitute DURC, nor would all DURC be found within these categories. Perhaps most important, the NSABB emphasized that DURC was not a pejorative categorization. Such a designation would not necessarily mean that the research should be foregone or its results withheld; it merely designated that the conduct and communication of such research should be carefully evaluated at its outset and throughout its execution.

Key features of the oversight process recommended by the NSABB include:

• the development of federal guidelines for the oversight of dual-use life science research;

• ensuring that all parties are aware of and educated about dual-use concerns involved;

• local evaluation and review of research for dual-use potential;

• assessment and management of dual-use risks during the process of conducting and communicating research;

• periodic evaluation of the need for and effectiveness of the oversight system; and

• instituting of appropriate mechanisms for compliance.

The process would start with principal investigators determining whether their research qualified as DURC (see Figure 1). The NSABB anticipated that very few projects would. Those that did would trigger an institutional review process that would address whether and how information, technologies, or agents from the research project might be misused, the potential consequences of misuse, and strategies to mitigate the risks of misuse.

OPEN IN VIEWER

Figure 1.

Major Steps in NSABB’s Framework for Local Oversight of Dual-Use Life Sciences Research. ¹ Color images available online at www.liebertonline.com/bsp

These reviews would be conducted entirely within a research institution. Unlike proposals for recombinant DNA experiments, which in some circumstances were reviewed at the federal level, the NSABB oversight framework does not specify a federal government role in reviewing specific projects. Initially, oversight would be applied to research that was conducted or funded by the federal government, but NSABB anticipated that it might also be implemented for non–federally funded research, just as many performers without any federally funded recombinant DNA research voluntarily adopted a version of the recombinant DNA guidelines. Voluntary or funding-linked compliance would be evaluated before other mechanisms, such as legal mandates, would be considered.

The NSABB oversight framework also discussed the responsible communication of dual-use research. It presented a set of principles, laid out points to consider in assessing the risks and benefits of communicating research with dual-use potential, and provided considerations for developing a communications plan for such research. This discussion recognized that communication occurs at all points of the research process, not just when results are published.^{1(Fig. 2, p26)} The NSABB also pointed out that how something is communicated, and the context in which that communication takes place, can be as important as what is communicated. That context may not mitigate the risk of abuse of the underlying information, but it may be essential for assuring the public that dual-use risks were considered, were minimized to the extent possible, and were ultimately judged to be worth taking—to the extent, of course, that such a process was actually performed.

Internationalizing the Discussion

The Fink Committee (2002-2003) and the National Academies/CSIS symposium (2003) took place in the U.S., and the first government advisory body intended to address such issues, the NSABB, was formed there as well (2004). Although science is a thoroughly global enterprise in measures such as the spread of information and expertise, participation in meetings and exchanges, and research collaborations involving scientists from different countries, scientific funding and governance mechanisms are almost entirely national. Many of the Fink Report's recommendations were therefore directed at the U.S. government or institutions. Even so, virtually every discussion of the dual-use dilemma in the life sciences has stressed that the problem ultimately has to be addressed at the international level. For that reason, the Fink Report called for development of an International Forum on Biosecurity that could work to develop harmonized national, regional, and international measures analogous to those recommended for the U.S.^26(p124)

Staff at the National Academy of Sciences recognized that an international engagement effort driven by a U.S. organization would be less effective than one with a more international makeup. However, venues for dialogue at the international level on these issues were challenging to develop. When it comes to the “international scientific community,” in the words of a senior National Academy staffer deeply immersed in these activities, “there is very little ‘there’ there,” ¹¹ and few independent international scientific organizations exist that deal with policy issues. Those that do generally saw security considerations as a threat to science. For example, the International Council for Science (ICSU), a nongovernmental organization with a global membership of national scientific organizations and international scientific unions, has incorporated into its statutes the principal of the Universality of Science, which calls for “freedom of movement, association, expression and communication for scientists as well as equitable access to data, information and research materials.” ⁴⁰ In a 2004 statement, ICSU warned that measures constraining the conduct of science in a number of countries, “often imposed on security grounds,” were undermining universality, and that compensatory measures undertaken by the scientific community in response, “including self-censorship by scientific publishers, and pre-emptive behavior by scientific organizations to avoid contravening national security regulations,” could constitute threats to universality in their own right. ⁴¹ This statement made no specific reference to biosecurity, but it did address governance measures that had been proposed to address dual-use concerns.

The International Forum on Biosecurity

In searching for international partners, the National Academy of Sciences reached out to 2 that it had helped form: the InterAcadamy Panel (IAP), a network of 94 science academies around the world, and the InterAcademy Medical Panel, a global network of 64 academies of science and medicine dedicated to improving health. In addition, it partnered with ICSU, with which it had a longstanding relationship. Even so, it took almost a year to gain the agreement of these organizations to cosponsor an International Forum on Biosecurity, in large part due to their reluctance to address security matters.

The forum was held in Como, Italy, in March 2005, and involved more than 50 participants, mostly scientists, from 20 developed and developing countries. Also attending were individuals from policy organizations working on dual-use issues, as well as representatives of interested organizations such as the International Committee of the Red Cross, the Organization for Economic Cooperation and Development, and the World Health Organization (WHO). It was the first exposure to the dual-use issue for many attendees. A wide range of views were represented, from those fully supportive of the arguments made in the Authors' and Editors' Statement and the Fink Report, to the skepticism, if not hostility, toward addressing security issues that was expressed in the ICSU 2004 statement (this observation is from the author's experience as a participant in this meeting). Although no conclusions or recommendations were issued, the meeting informed later activities by the participating organizations.

The Lemon-Relman Report

Another step toward internationalizing the discussion of dual-use life science was a deeper look at the global nature of the life sciences, addressing governance measures that might be more broadly adopted than a single country's oversight system. Such a perspective was taken in a National Academies report that examined how the accelerating evolution of the life sciences and related research might change the nature of potential biological threats (natural, accidental, or deliberate). The report, Globalization, Biosecurity, and the Future of the Life Sciences (often called the Lemon-Relman Report after its 2 co-chairs), was released in January 2006. ⁴² Its recommendations included reinforcing the free and open exchange of information in the life sciences, broadening the conception of biological threats beyond specific lists of pathogens and toxins, and adopting a common culture of awareness and a shared sense of responsibility within the global community of life scientists.

Multiplicity of International Discussions

Following the Como meeting, scientists discussed dual-use issues with their international counterparts and with other stakeholder communities in a variety of venues. The National Academies and the international scientific organizations it had partnered with for the 2005 Como meeting held subsequent workshops in Budapest, Hungary (2008), ⁴³ and in Warsaw, Poland (2009). ⁴⁴ Meetings associated with the Biological Weapons Convention (BWC) brought government and nongovernment life scientists, representatives of international scientific organizations, and other members of civil society together with diplomatic delegations to discuss codes of conduct for the life sciences (2005) ⁴⁵ and to address topics that included the oversight of life science research (2008). ⁴⁶

WHO, which had traditionally been very reluctant to address security-related issues for fear that its public health mission would be compromised, conferred considerable international legitimacy on discussions of the dual-use dilemma when it convened international meetings on “Life science research and development and global health security” in 2006 ⁴⁷ and 2008. ⁴⁸ It also cosponsored a series of 3 international roundtables with the National Science Advisory Board on Biosecurity's international working group, 2 in 2007^49,50 and the third in 2008. ⁵¹ WHO went on to issue a major guidance document on the topic in 2010. ⁵²

In addition to the international workshops organized by the U.S. National Academies and partners, academic groups and nongovernment organizations have also actively promoted international discussions of dual-use governance. Between 2006 and 2010, the University of Maryland's Controlling Dangerous Pathogens Project held workshops in Hungary, Brazil, South Africa, Egypt, Singapore, and Thailand, in addition to the U.S. ⁵³ A group at the University of Bradford in the United Kingdom has held seminars on dual-use research in Argentina, Australia, Finland, Germany, India, Israel, Japan, Kenya, the Netherlands, South Africa, Sweden, Switzerland, Uganda, Ukraine, the United Kingdom, and the United States. ⁵⁴

Implementation by Governments

The multiplicity of international discussions has generally not translated into the adoption of dual-use oversight systems or policies by national governments or institutions, although some countries have taken steps. In 2007, the Royal Netherlands Academy of Arts and Sciences published a national Code of Conduct for Biosecurity at the request of the Dutch Ministry of Education, Culture, and Science. Among other provisions, the code calls for those involved in the life sciences to screen research activities for dual-use implications as they are being proposed and while they are being conducted, to weigh potential risks against potential benefits when dual-use aspects are found, and to reduce the risk that publication of research results will “unintentionally contribute to misuse.”^55(p11) However, it does not specify a process by which such reviews would be conducted, the assessments performed, and the risks and benefits weighed.

The Israeli government established a dual-use oversight process with the passage of the Regulation of Research into Biological Disease Agents Act in November 2008. In addition to setting up a program analagous to the U.S. Select Agent Program under which the government will regulate the possession of certain biological agents, this law establishes local committees to supervise research at institutions possessing disease-causing agents. These committees include institutional safety and security personnel as well as scientists. The law requires that advance approval be obtained for research projects involving biological agents listed in the law. It also specifies that research projects not initially subject to approval but that develop findings that would increase the contagiousness or virulence of non-listed disease agents, or that extend the range of a non-listed agent so that it can pass to humans, must be halted and submitted to review by the institutional committees. ⁵⁶

Although the U.S. government has not yet responded to the NSABB's proposed oversight framework, dual-use risks were addressed in the National Strategy for Countering Biological Threats, issued by the White House in November 2009. ⁵⁷ One of the 7 objectives delineated in that strategy deals specifically with reducing the potential for the life sciences to be exploited for harm. In addition to optimizing security and safety procedures governing access to pathogens that are at high risk of being used in a deliberate attack, the strategy calls out the need to manage risks posed by dual-use information. It promises that the federal government will provide “detailed guidelines” to aid in identifying such information and enabling actions to reduce its potential for misuse, while recognizing the importance of such information for legitimate activity. ⁵⁷ However, it leaves the responsibility for taking such actions to the life sciences community; the only reference to a more directive or coercive government role refers to “potential mitigative actions” (not otherwise specified) that might need to be imposed in response to “novel or emerging capabilities” that significantly affect risk.^57(p14) As is discussed below, federal guidelines would be very helpful for instituting a dual-use oversight process. However, those guidelines may be considerably more difficult to develop—or less useful in specifying action—than this language anticipates.

Recent Controversy

The most recent dual-use research projects to attract worldwide public attention involved the conversion of H5N1 avian influenza, a pathogen that is highly lethal to humans but rarely infects them and is not effectively transmitted between them, into a version that is contagious among ferrets and very likely to be transmissible from person to person. When research manuscripts describing the creation of such strains were brought to the attention of the NSABB prior to publication, the NSABB for the first time found that complete publication of the methods and results posed an unacceptable risk of enabling replication of the work by those seeking to do harm.# The NSABB recommended that the papers' conclusions be published, but without data on the specific mutations that could enable person-to-person transmission or descriptions of the full experimental procedures used to create the strains. In line with the concerns it had expressed 6 years earlier when it had reviewed (and at that time approved) publication of research on the reconstruction of the 1918 influenza virus, the NSABB also recommended that text be added to the papers explaining the public health benefits of the research, the risk assessments undertaken prior to initiating it, and the biosafety and biosecurity practices that were applied. ⁵⁹ HHS has no legal authority to restrict publication of these papers, but it passed the NSABB recommendations to the papers' authors and publishers. It also announced that it was working to establish a mechanism to provide secure access to the redacted information to those in the influenza surveillance and research communities who needed it to advance public health. ⁶⁰

The publishers and authors appear to be reluctantly going along with the NSABB recommendation. Science Editor-in-Chief Bruce Alberts said that the journal's response would depend heavily on the government's ability to set forth a “written, transparent plan” by which all responsible scientists who request the redacted information will be able to receive it. ⁶¹ Alberts reiterated Science's support for the NSABB and for the 2003 Statement on Scientific Publication and Security, which recognized there may be occasions when scientific papers should be modified or withheld from publication, but he also expressed concern about withholding potentially important public health information. Nature Editor-in-Chief Philip Campbell stated that complete details of the studies needed to be made available to researchers and that Nature was discussing how that could be done consistent with the NSABB recommendations. ⁶² The authors continue to believe their results should be published in full but nevertheless are submitting or have submitted revised versions of their papers to the respective journals. ⁶³

Widespread news coverage of the NSABB's unprecedented recommendation attracted considerable attention and controversy to the underlying research. Within the scientific community, many critics of the NSABB's decision supported the research, were not persuaded that security concerns justified restricting its publication, and thought that constraining dissemination of the results damaged science and increased societal risk by impairing the public health response to a possible H5N1 flu pandemic. Other scientists thought the research should never have been done in the first place but that any attempt to restrict circulation of the results after the fact was futile. Still others in the scientific and policy communities believed that conducting the research was a mistake and that publishing the results in full would compound that error. Concerns over the accidental escape or the deliberate misappropriation of transmissible H5N1 strains from containment factored into the public and scientific community discussion at least as much as security concerns about misuse of the research results.

As difficult as it will be to implement effectively (see the section below on the Responsible Communication of Dual-Use Research), the NSABB's recommendation is highly significant in that it validates not only that security considerations can be important enough to change how science is done, at least in exceptional cases, but also that a committee consisting largely of scientists is capable of coming to that conclusion. The board's 2005 decision not to seek restrictions on publication or continued research with 1918 influenza virus had led some to question whether it would ever encounter dual-use research proposals or publications that it thought should be modified in a fundamental way on security grounds.

And as with the 1918 flu case, controversy over the publication of completed research called attention to the lack of a formal process to review dual-use research projects at their inception—particularly since, like the 1918 flu research, the H5N1 research at issue was funded by the U.S. government. ⁶⁴ The National Institutes of Health press release announcing the NSABB decision made no reference to NIH's support of the H5N1 research or to any consideration that may have been given to dual-use concerns when the proposed projects were reviewed for funding. ⁶⁰

Evaluation and Analysis

Framing the Issue

How the question of dual-use research oversight is framed can be as important as how it is addressed. U.S. National Academy of Sciences officials have found that considering dual-use oversight within the broader context of the responsible conduct of research, or the general set of obligations scientists have to the broader society, is more conducive to dialogue in the international community than posing it as a problem of national security. ⁴⁴ Similarly, an extensive treatment of dual-use research oversight by WHO presents the choices and tradeoffs involved in research oversight as questions of ethics. ⁵² Research is already overseen or regulated for a wide variety of reasons today, such as biosafety, use of animals, use of human subjects, and research integrity. Placing dual-use oversight in the context of these other objectives, and taking maximum advantage of existing processes, will be more efficient and better accepted than treating it in isolation.

At its core, a dual-use oversight process should be a means of ensuring that the judgment of informed, independent observers with a variety of perspectives is brought to bear on weighing the societal benefits of the proposed research against its potential security liabilities. Participation of individuals with backgrounds in law enforcement, national or homeland security, or intelligence, for example, would be useful—not for the purpose of adding specialized expertise or information, but because such individuals are likely to recognize and weigh the possible benefits and risks of a given research project differently than scientists or research administrators, encouraging a richer discussion of alternatives and outcomes.** As explained below, dual-use reviews will not be reducible to specific criteria or codified procedures; the best that can be expected is to foster a full discussion of dual-use risks, a subjective evaluation of potential mitigation measures, and either a justification of whatever approach is decided on or a decision to refer the project to a higher level of review.

Dual-use governance processes ultimately act through their effects on what research projects are initiated, what procedures and protocols are involved in conducting those projects, and how results are communicated. Education and awareness efforts are essential, since no oversight system or culture of responsible research can successfully be implemented unless those involved are aware of the risks the system is addressing and understand their responsibilities to address those risks. ⁶⁵ However, education and awareness do not reduce dual-use risks unless educated and aware scientists initiate, conduct, or communicate research differently than insufficiently educated or less aware scientists would have.

One major distinction among dual-use oversight approaches is whether the ultimate authority to approve research projects rests with researchers and research institutions or with government. From a technical decision-making level, this distinction may not be very significant. Either approach can succeed—or fail—at bringing independent, diverse sources of judgment to bear and arriving at a conclusion that balances benefits against risks. But there is a marked conceptual difference between an institution's deciding whether and how to proceed with research projects, and society's (through government) granting permission to do so. Government clearly has the authority to regulate the conduct of research, like any other societal activity, on the basis of the potential hazards it poses to those directly involved, to bystanders, or to the environment. However, liberal democracies presume that governments should not infringe on freedom of inquiry, which translates to a presumption not to control activities on the basis of the knowledge that they might generate. ⁶⁶

Moreover, from a practical perspective, the life science enterprise is large enough that any process that imposed national level decision making over individual life science projects would impose a crippling burden unless the majority of research projects were not affected at all, most of the rest could be expeditiously reviewed locally, and only a very small number would be subjected to national-level examination. The retrospective analysis of dual-use research recounted above indicates that only a few projects over a 10-year period have attracted the highest levels of attention on dual-use grounds, but it is not clear whether that would still be the case if all life science research were more systematically subjected to a formal dual-use oversight process and more life scientists were aware of dual-use concerns. More experience with dual-use review is necessary before imbuing authority for dual-use research project approval with the federal government.

Need for a U.S. Dual-Use Oversight Process

H5N1 influenza research is only the most recent confirmation that new scientific activities or results will continue to raise concerns both within and outside the scientific community regarding implications for biological weapons. Whether or not any of the studies described earlier in this article actually did pose unjustified security risks, the lack of any formal process to weigh risks and benefits leaves the scientific community vulnerable to accusations that these risks are not being responsibly addressed.

Although the NSABB released its recommended framework for the oversight of dual-use life science research almost 5 years ago, the U.S. government has not yet responded with a set of proposed governance policies and/or regulations, either to build on the NSABB proposal or to take an alternate approach.

It should do so as soon as possible by proposing a system largely along the lines of the NSABB's proposed framework. The NSABB was created and supported by the U.S. government specifically to bring a multidisciplinary perspective to dual-use issues in the life sciences, and it includes among its membership a number of highly respected senior life scientists who have both the credibility to articulate to their colleagues the need for an oversight process as well as an understanding of the scientific enterprise sufficient to develop an oversight process compatible with it. As a result, the NSABB's proposed oversight framework should be the starting point for any U.S. oversight system. Its approach is well reasoned and well justified and accommodates the need to address—and be seen to be addressing—dual-use research issues while minimizing inconvenience to scientific research projects that do not raise significant dual-use issues. It is not specific enough to be implemented by the U.S. government as is, and in any event it should be augmented in certain respects as described below. However, it does describe an approach that builds on existing procedures and processes and could be fleshed out and implemented in a relatively seamless manner. Five years is too long for this proposal to sit without a government response, and reports in December 2011 that the U.S. government will release its proposed oversight process for comment within a month or 2 are overdue. ⁶⁷

At the same time, it is not clear—in the absence of much more experience with the operation of such a governance system than exists to date—to what extent potentially dangerous activities or unjustified risks will actually be averted if a governance system is put into place. Detailed guidance is not available regarding how security risks are to be weighed against scientific, public health, economic, and security benefits; whether the greatest degrees of dual-use risk are limited to work with the most lethal and transmissible infectious agents (which do indeed present the greatest biosafety risks); what criteria would indicate that otherwise meritorious scientific studies are too dangerous from a security perspective to conduct; what changes in the conduct of an experiment would mitigate security concerns while preserving their scientific value; or what changes to communication of research results might accommodate security concerns while remaining consistent with the practice of science. Indeed, the ambiguities and unknowns inherent in assessing the future benefits and risks of dual-use research give little reason to anticipate that a standard set of procedures for making these judgments will be forthcoming, or that the overall security value of introducing an oversight process can be estimated. A decision process that takes security risks explicitly into account has some possibility of mitigating them, and performing these evaluations in a visible way should provide assurance that a responsible approach is being taken. But the benefits of imposing a dual-use governance system cannot be so reliably estimated as to justify approaches that are substantially more disruptive and costly than the NSABB's.

Furthermore, the more burdensome and disruptive an oversight system is, the less likely it will be to elicit the cooperation of those who must make it work. Given the difficulty in codifying a set of standard dual-use oversight procedures and analyses, effective implementation will not be a matter of preparing and enforcing checklists but will require the application of subjective judgment by researchers and overseers alike. If the participants in this system do not believe the process to be credible or worthwhile, they may go through the motions of implementing it but are not likely to put in the effort, thought, and creativity to make it work. Other oversight processes that already govern life science research, such as those addressing recombinant DNA, biosafety, use of animals in research, and use of human subjects, deal with harms that can be more readily and objectively specified and prevented than those having to do with the risk of misuse. These other processes can protect laboratory and environmental safety, animal welfare, or human subject health and dignity whether or not the scientists subjected to them agree with their rationales and methods. Dual-use oversight, on the other hand, cannot succeed without the active support of its participants.

So while the argument to implement a governance approach is compelling, the argument to pay a substantial price for one—in terms of the cost, administrative burden, or foregone research—is not, at least for now. In the expectation of modest or uncertain benefits, only modest costs should be imposed. Whatever system is put into place should build, to the greatest extent possible, on existing administrative structures and procedures, as does the NSABB's framework. Further experience with dual-use governance will provide government with a firmer basis for bolstering the dual-use governance process, if it is seen to be feasible and effective, or for scaling it back, if it is not viewed as lessening security concerns at an acceptable cost to research.

Dual-Use Oversight in the Initiation of Research

One shortcoming of the proposed NSABB oversight framework is that it makes no specific reference to obligations that funders of life science research should assume for addressing the dual-use implications of the research they are asked to fund. Funders, of course, cannot necessarily anticipate surprises that arise during the conduct of research. Yet some of the most significant dual-use concerns described above, such as the reconstruction of the 1918 influenza virus or the creation of a more transmissible version of H5N1 avian influenza, resulted from the intended outcome of research, not from unanticipated surprises. These issues should have been apparent, and should have been assessed, before funding was approved.

At present, the National Institutes of Health has no institute-wide policy on dual-use reviews for research proposals. It should implement one—both by requesting proposers to identify and explain how they intend to deal with the dual-use risks of their research, and by taking the opportunity to identify and evaluate such concerns in funding reviews even if the proposers have not raised them.†† Three major funders of life science research in the United Kingdom—the Medical Research Council (MRC), the Biotechnology and Biological Sciences Research Council (BBSRC), and the Wellcome Trust—have agreed to modify organizational guidelines on good research practice to include specific reference to misuse, to develop clear guidance on dual-use concerns for funding committees, to mention risks of misuse in guidance to proposal reviewers, and to ask applicants to consider the risk of misuse in their applications. ⁶⁸ Fewer than a dozen out of more than 10,000 proposals to the BBRSC have been flagged as raising dual-use concerns, and none have generated serious discussion weighing risk and benefit. ⁶⁹ It is not clear, however, whether proposals of dual-use concern are truly rare or whether those involved in the process are insufficiently aware of dual-use concerns, and the evolution of these numbers as education and awareness efforts proceed will be instructive.

Dual-Use Oversight in the Development and Execution of Research Projects

Any governance approach will need to identify research activities that merit review, identify the potential harms and benefits of activities that have been so selected, and take appropriate risk mitigation actions when warranted.

Identifying Projects for Further Scrutiny

In the initial screening process, there will always be a trade- off between efficiency and completeness: The more objective and crisply defined the criteria, the easier to evaluate individual projects, but the greater the probability of missing risky research projects that do not neatly fit the criteria. Lists and categorizations such as the NSABB's dual-use research of concern can be useful as starting points, but they should not prevent additional projects from being identified for scrutiny as well.

The NSABB oversight proposal has been criticized for giving principal investigators the responsibility to evaluate their own work for dual-use research of concern. This is a red herring. Although it is possible that a PI would not recognize or would seek to downplay the dual-use potential of his or her own research, it is more likely that for the most significant cases, the need for dual-use review would be obvious. If the PI did not flag such a proposal for review, anybody else involved in proposal preparation, whether at the institution or at the funding agency, could do so—and would wonder why the PI had not. Institutions and investigators should not want to be in the position of explaining why they had failed to review a project that proves to be highly consequential from a dual-use perspective, and occasional audits could determine whether there was a pattern of failing to review projects that warrant it.

Assessing Potential Harms and Benefits

Dual-use reviews concern the future use of information that results from life science research, rather than addressing the materials or methods used in the course of the research. The risk of misuse is not strictly a technical criterion, nor is it a function solely of the research itself; it depends in addition on the nature of potential misusers, such as their existence, intentions, capabilities, and resources. The societal benefits against which those risks must be weighed likewise are highly contingent, depending on actions and actors external to the research.

The perspectives of those evaluating risks and benefits is likely to color their perceptions. Scientists may portray the benefits of their research as immediate and tangible while seeing security risks as distant and theoretical. On the other hand, security professionals may see a direct mechanism by which the research results could be misused, while viewing the societal (rather than scientific) benefits of that research as distant and theoretical. It will be important for dual-use reviews to involve a set of perspectives diverse enough to capture both aspects.

Complicating these analyses still further is that those deciding whether or not to conduct or to publish research are not just evaluating whether that research might lead to more harm than benefit; they must also consider to what extent refraining from conducting or publishing that research will avert those harms or forego those benefits. The further into the future one tries to extrapolate, and the more complex the assumed process of application or misapplication is, the less feasible any of these analyses will be. Yet the consequence of misuse will not necessarily diminish over time, since information that is put into the public record will remain there indefinitely.

The NSABB sought to make these analyses tractable by defining dual-use research of concern as research that produces results that can be reasonably anticipated to cause harm if directly misapplied. These qualifications protect the oversight framework from being held to an impossible standard, but at the possible expense of bounding out of the analysis the most significant long-term consequences.

It is hoped that the extent to which misuse can be reasonably anticipated will be better understood with time and experience. To maximize learning, those performing such analyses should be able to look at how these assessments were made in previous cases. Some of these lessons may comprise dual-use information of concern in their own right, since they may contain information on how research might be misused that would not be common knowledge or readily derivable. However, since these assessments would be explicitly security related and would not necessarily contribute to the advancement of scientific knowledge, they would more appropriately be restricted than scientific results.

Mitigating Dual-Use Risks

Perhaps the most challenging—and most important—part of a dual-use oversight process will be developing ways to mitigate risks of misuse. Unless they rise to the level of calling for the research to be halted, there would be little point in identifying and evaluating dual-use risks if nothing could be done about them. In its guide for assessing and managing dual-use risks, ¹ the NSABB proposes several mitigation strategies for experiments of concern but does not elaborate on them. These strategies include review, modification, or discontinuation of experiments; consideration of the content, timing, and audience for communicating dual-use information; and provision of additional context in communicating research information.

“Modification” is of particular interest. The most desirable outcome of a dual-use review process would be modification of a high-risk experiment to lessen its risk while preserving its scientific value. However, very few such cases have been described. One case from Duke University, which had developed its own process for conducting dual-use reviews, is instructive. An investigator wanted to study the effects of blocking transmission from some specific neurons in an animal model by killing those neurons with a neurotoxin. Since the bare toxin was unable to enter those cells, the investigator planned to deliver it by engineering a retroviral vector to express the toxin within the targeted cells. As a result of questions asked by a Duke review committee, the investigator decided to abandon the neurotoxin and to express a marker gene in targeted neurons that would identify them without killing them. This modification did lower the risk—the experiment no longer served to increase the transmissibility of a toxin—but in marking the neurons rather than killing them, the investigator was no longer able to study the effects of blocking neural transmission.^51(pp14-15) Further experience with dual-use review will be very informative to see to what extent modifications of experimental protocols or procedures are able to mitigate dual-use concerns without impairing scientific value. There is little reason as yet to expect such outcomes.

What has been shown is that scientists are willing to modify their behavior to lessen dual-use risks. A survey of life scientist members of the American Association for the Advancement of Science (AAAS) indicated that 15% had made some changes in how they conduct, manage, or communicate research because they were concerned that misuse of their research might facilitate bioterrorism. ⁷⁰ (Those surveyed were not asked whether they had modified experimental protocols, but rather whether they had avoided conducting certain types of research, avoided certain collaborators, or limited their communications in some way.) Low response rates and methodological concerns preclude extrapolating these results to the broader U.S. life science community, but the finding is intriguing.

Modifications to the NSABB Oversight Framework

A few changes should be made to the NSABB oversight framework to improve its ability to deal with dual-use risks. The NSABB states that institutional dual-use review entities should be constituted “in a manner so as to have the necessary expertise to consider the dual-use implications of research and to recommend and oversee risk management strategies.”^1(p12) That breadth of expertise is necessary, but a breadth of perspective is also important: These panels should include people who see science as instrumental in achieving public health and other societal objectives, but do not necessarily see it as an end in itself. Members of professional communities other than scientists will likely have such a view, as indeed may many scientists. Review entities need to be comprised of people who are aware of the possibilities and motivations for misuse, but it would be counterproductive to specify the disciplinary or professional composition of these committees too narrowly, given the diversity of institutions that will need to maintain them and the personnel each will have available to fill them.

To make the most complete assessment of risks and benefits, dual-use risks should be assessed in conjunction with biosafety risks (ie, accidental lab exposure or environmental release of dangerous pathogens) and laboratory biosecurity risks (ie, theft or misappropriation of dangerous pathogens), even though other means exist to deal with these other risks. Research that poses the most significant biosafety and laboratory biosecurity hazards likely poses high dual-use risks as well, as was the case with H5N1 influenza research, although the converse (that high dual-use risk is limited to projects with high biosafety and laboratory biosecurity risk) may not necessarily be the case.

Finally, the decentralized, institution-based dual-use oversight process that was recommended by the NSABB should be connected more explicitly to national resources, including the ability to elevate particularly problematic cases to centralized review—a role that the NSABB itself has already assumed, on occasion, in reviewing the proposed publication of dual-use research. High-profile or highly problematic dual-use issues merit the full attention of a national expert board such as the NSABB. Even if the NSABB or some analogous body cannot be convened frequently enough to provide guidance each time a local institution seeks help, a mechanism is needed to provide requested information or to make expertise available to local bodies on request, and to convene a higher-level advisory process when necessary for the highest consequence research projects, such as those with the greatest potential to facilitate the illicit creation of highly lethal, highly contagious pathogens.

A federal audit and “lessons learned” function that assesses the consistency of local decision making and cumulates expertise is also necessary.

Responsible Communication of Dual-Use Research

Even including the recent H5N1 influenza case, journals have rarely considered modifying or rejecting a publication on dual-use grounds. Most life science journals have not instituted dual-use review processes, but some are in place.^71,72 Between 2003 and 2007, only 28 papers out of 74,000 submissions to the Nature Publishing Group were flagged for dual-use scrutiny, and none were rejected.^51(p17) In 6 years of publication, the journal Biosecurity and Bioterrorism has flagged 3 papers for dual-use concerns. One was modified and published, and 2 were rejected, one of which was resubmitted and published elsewhere.^51(pp17-18)

Controlling Dual-Use Information

It remains the case, as it was when the Authors' and Editors' Group issued its statement in 2003, that identifying information that should be withheld from publication because of the risk of misuse “will continue to challenge us.” Mechanisms that editors can utilize for advice and assistance in exceptional cases, such as referral to the NSABB, should be available to them. However, given the extraordinary nature of a decision to restrict publication, it is unlikely that general guidance will be very helpful in resolving specific cases.

The NSABB's recommendation to restrict publication of H5N1 influenza research raises a number of practical problems concerning the likely effectiveness of the restriction and what to do with the information that is withheld from widespread dissemination. Drafts of the unredacted H5H1 research papers at issue have already been circulated among research group members, journal officials, and peer reviewers, and many more scientists will have access to them once a mechanism is developed to share them on a restricted basis. In an age of Wikileaks, any of these people who is sufficiently opposed to publication restrictions can unilaterally make the papers widely available. Blog postings, e-mail messages, or even oral communication could readily convey the list of mutations in the viral genome that the NSABB sought to protect. No matter what system is put in place to restrict this information, it cannot be expected to do so successfully for very long. Even so, in a world awash with information, much of it of dubious quality, denying the imprimatur conferred by open publication in respected research journals may have some value in impeding reliable access to the information by those who do not have legitimate need for it.

Creating a system to securely distribute the papers will prove quite challenging—if not from an information technology perspective, given the ability of firms to keep proprietary information out of the hands of nonpaying customers, then on policy grounds. The U.S. government has said it will develop a mechanism to disseminate the H5N1 papers “to those with a legitimate need in order to achieve important public health goals,” ⁶⁰ whereas Science magazine is waiting for a system that will provide it to “all those responsible scientists who request it” ⁶¹ and Nature has stated that it is essential that “full details … be available to researchers.” ⁶² Do all responsible scientists have a legitimate need for it? How can “responsibility” be judged? Will a new system have to be developed each time information is restricted, since different groups may have “legitimate need” for it each time? Since public health is a shared responsibility among all nations, all of whom benefit from each other's efforts, it is difficult to see how access to this information can be precluded solely on the basis of nationality—a sharp contrast to how the U.S. or any other country grants access to information that is classified for national security purposes, for which nationality is one of the most important criteria.

Another important distinction between protecting dual-use information and protecting classified information is the duration for which restrictions are meaningful. National security classification is typically intended to protect an advantage or to delay an adversary—for example, to protect negotiating positions or to impede adversary efforts to copy or defeat new military technologies. After some period of time, most classified information loses its sensitivity, such as after a treaty has been concluded or a successor generation of military hardware deployed. The risk of misusing dual-use information for harm, on the other hand, may be a persistent one. One would like to delay its abuse for as long as possible, but there may never be a time when one no longer worries about the possible consequences of its exploitation, making the modest expectation in successfully being able to contain it even more troubling.

The practical and conceptual difficulties in restricting the dissemination of H5N1 influenza research illustrate how exceptional this decision has been. Given the stakes, attempting to limit open dissemination was reasonable in this instance. However, the exception if anything proves the rule, which stresses the importance to science of open participation and open communication, even in the presence of security concerns.^4,73,74 An excellent analysis of the problems widespread information restriction would cause in the biological sciences is presented in the Lemon-Relman report. ⁴² And in the event that dual-use information is judged too risky for any scientific use, deciding what to do with it instead poses additional challenges. Guidance would be useful with respect to what responsibility editors would have in such a circumstance to follow up with other publishers, with the author, his or her research institution, the funder, or the government.^51(p50)

Security Classification

The Lemon-Relman report did not categorically reject the idea that some life science information might require government security classification, but it recommended that the process of determining what circumstances would warrant classification be subject to public scrutiny.^42(p226) As a general tool to manage dual-use information in the life sciences, classification has significant limitations. For one, the U.S. government has very little legal authority, and often no ability, to control information it did not generate or pay for, ⁷⁵ including the large and increasing fraction of the world's life science information that originates in other countries, completely outside the scope of U.S. classification.

The U.S. government certainly has the authority to classify work performed in its own laboratories or (depending on how the funding is provided) by those whose research it supports, but the more it does so in the hope of preventing the misuse of that information, the more it aggravates a separate problem: the perception that the information is being used instead in an offensive biological weapons program that would violate the Biological Weapons Convention, and that it is being hidden through secrecy. It is in the U.S.'s interest to provide assurance that it is remaining fully compliant with its treaty obligations whenever it cannot be fully transparent about its own activities, and its current compliance reviews could be improved by more formally linking them across government agencies, by involving nongovernment representatives, and even by involving representatives from other governments. ⁷⁶

Responsibilities of Other Professional Communities

The governance of dual-use research will involve consideration of a number of societal values by many professional communities. As a WHO guidance document says,

Decision-makers will need to make judgments to resolve difficult cases of conflicting values. Scientific freedom, scientific progress, public health, safety and security are all important values, and none should be given absolute priority over the others.^52(p29)

Although this article has concentrated on the role of the scientific community in dealing with the governance of dual-use research, other communities such as law enforcement, national security, and intelligence have a role as well; the utility of involving individuals from these communities in dual-use review processes has been discussed. These communities also must recognize that despite dual-use concerns, the continued advance of the life sciences is necessary not only to meet public health needs and other societal objectives unrelated to security, but to address security requirements as well. A vibrant life sciences enterprise is needed to detect, diagnose, counter, mitigate, respond to, and attribute threats that could be generated with decades-old science and technology, let alone new threats that advancing science might make possible.

Law enforcement, national security, and intelligence agencies in the U.S. appear to understand this. Recognizing that relations between these communities and the life sciences community have not always been easy, the Federal Bureau of Investigation (FBI) and the intelligence community have been particularly active participants in the U.S. government's efforts to engage the life science community on security issues—a program that is characterized not by attempts to constrain the conduct of science, but rather to raise the awareness of dual-use issues and responsibilities within the life sciences community, and to equip the community with tools and mechanisms to help protect itself from those who would seek to subvert it. ⁷⁷

The FBI has been working with AAAS, HHS, and the Department of State to reach out to the synthetic biology, amateur biology, and biosafety communities on security issues. Intelligence community support, in a fully transparent manner, has been instrumental to the creation of the Virtual Biosecurity Center, a “one-stop” online shop created by the nongovernmental Federation of American Scientists to bring together information on the responsible use of life sciences and biotechnology to counter the threat of biological weapons. ⁷⁸ The intelligence community has also reached out to the scientific community by creating a Biological Sciences Experts Group, a body of senior nongovernment life scientists and others who not only help ensure the intelligence community has access to state-of-the-art expertise in the life sciences, but who are also able to bring an awareness of intelligence issues and priorities back to their “day jobs” in the scientific and other organizations that they represent.

The law enforcement, intelligence, and scientific communities also need to work together to increase the likelihood of discovering illicit technical activity, an aspect of preventing the development of biological weapons that builds on, but goes beyond, the dual-use oversight processes discussed here. Illicit activity undertaken in isolation will remain difficult to find, but illicit activity conducted in social settings under the guise of legitimate effort may be noticed and questioned by others. All relevant stakeholders and professional communities have the responsibility to explore mechanisms through which those who have such questions or concerns can contact colleagues, institutional officials, law enforcement officers, or government representatives to have these concerns addressed in ways that respect privacy, individual rights, and public safety. As dual-use technologies, information, and technical capacity increase in power and geographic scope, there is little reason to believe that the risk of abusing them to commit acts of serious harm will diminish. Dual-use oversight systems will be an important, but limited, tool to address these risks, and every other tool that can responsibly address them needs to be explored as well.

Conclusion

Considerable thinking, analysis, dialogue, and interaction have taken place over the past decade to deal with the dual-use dilemma in biology. Despite the ongoing challenge in developing and gaining widespread acceptance of a dual-use oversight structure, the need to establish one remains unassailable: Society has every right to have confidence that life science research will be more beneficial than harmful. When scientists are asked whether they have addressed the potential that their research might be misused to inflict deliberate harm, the answer must never be, “We can't afford to constrain science” or “You know, we never thought about that.”

This is not to say that a process can be readily codified or confidently relied upon to tell whether science will prove to be more beneficial than harmful. Nobody can be expected to foresee the unforeseeable or to exclude the possibility that research information will contribute to misuse. This remains true no matter how coercive the oversight structure is, whether it is assumed by scientists or mandated by government, or whether decisions are determined by strict objective criteria and categorizations or by the subjective assessment of informed and educated participants.

But scientists, research administrators, funders, publishers, security policymakers, and other stakeholders are obligated to do what they can. A process is required that permits research to proceed unencumbered when no dual-use concerns are apparent; that when necessary can solicit reviews from people with a range of perspectives and expertise, all of whom are aware of the dual-use dilemma and their responsibility for addressing it; that draws on collective judgment, including that of people who have no vested interest in the work under review; that builds as much as possible on existing processes; and that anticipates and addresses concerns about misuse that are apparent to those sensitized to look for them. If and when a research project is challenged on dual-use considerations, the research community has to be able to demonstrate that the concerns being raised had been thought of, that alternatives were considered, and that a deliberate process had concluded that the best course of action was to proceed with the research.

Without extraordinary visibility into the actions of those who would misuse biology, it may be impossible to know the extent to which those actions might be facilitated by advances in life science research, or the degree to which a governance system could succeed in mitigating these risks. But the loss of public trust in the life sciences if they are perceived as being done irresponsibly is easier to envision—and just as important to prevent.