Abstract
In this article, we report on a pilot study examining perceptions of dual use among neurotechnologists and neuroethicists. We carried out 10 semistructured interviews with participants from established universities in the United States, the United Kingdom, and Australia about the risks they saw with the new technology and who has responsibility for safely developing it. We used a grounded theory approach to code and develop themes from the data and establish a foundation for analysis. Our findings showed that dual use was not considered binary, but rather multifaceted and fluid, with the commercialization and globalization of the emerging science shifting participants' understandings of what neurotechnologies are considered beneficial and harmful, rendering future uses unknowable.
Introduction
Neuroscientific research is generally produced under the maxim of nonmaleficence, promoting beneficial applications for its future users. However, it is nearly impossible to guarantee that no harm will result from neuroscientific innovations and their potential alternate or unintended uses. 1 While emerging neurotechnology applications are continually expanding, in this article we limit the term “neurotechnology” to techniques that are encompassed to specifically investigate, emulate, or intervene in the brain. This includes brain-inspired neural networks and cognitive computing and robotics; brain–computer interfaces (BCIs) and wearable technologies; neuromorphic computing; and brain stimulator devices.2-5 While these neurotechnologies have multiple uses, including medical, educational, lifestyle, and defensive uses, we are interested in the arms control concept of dual use, which refers to the potential of a technology or a piece of scientific research to be used for good as well as harm. Dual use is the unavoidable fact that scientific and medical breakthroughs can be mutually used for purposes unrelated to the goals of the researchers. 6 We agree with others who argue that the term “dual use” focuses on moral interpretations of what constitutes good and bad, and we acknowledge this can be subjective. 7 We are interested in what this means in practice and have undertaken this pilot study to examine perceptions of dual-use potentiality among neurotechnologists as it applies to their work.
In the case of biological, chemical, and nuclear technologies, international regulations exist to ensure that these technologies are not used for developing weapons. There are also controls to ensure that certain electronics, computers, software, sensors, or telecommunications technology are not used in conventional weapons. In all cases, the underlying technologies in question have useful and beneficial purposes. While these regulations consider the software and hardware components of neurotechnologies, they do not consider the specific potential for misuse and are therefore inadequate. Relevant discussions are taking place at the United Nations on lethal autonomous weapons systems, 8 particularly on aspects of human–machine interactions, the loss of human control, and accountability. While these discussions are limited to weaponry, informal discussions at the United Nations are also examining broader issues around artificial intelligence and militarization, including military decision making, intelligence gathering, and command and control systems.
None of the international regimes or current discussions, however, provide guidance on how people should consider the beneficial and harmful potential of neurotechnology—a growing area of research among scholars as militaries begin developing the technology.
The US Department of Defense and its Defense Advanced Research Projects Agency (DARPA) are the global forerunners of developing neuroscientific interventions and technologies aimed at improving military and civilian health. It is possible that neurotechnology development may lead a similar trajectory as the internet and the Global Positioning System, which are both examples of technologies that were originally created for military purposes but went on to benefit society at large. Research activities that involve possible brain-inspired neurotechnologies for soldier enhancement include intelligence gathering, image analysis, threat and deception detection, manipulation of emotional states, and incapacitation of adversaries. 5 Ambitious projects in these areas have the potential to work as restorative and augmentative applications; however, at the same time, the particular agenda of national militaries and their investments in neurotechnology raise significant social and ethical questions. Neurotechnology becomes problematic when its applications developed by the military for national security purposes also have the potential to disrupt healthcare, communications, and other components of the civilian arena. 9 Correspondingly, defense organizations might “harvest” commercial neurotechnologies and apply them to suit their own purposes. 5 Structured examination of these wider potential implications of emerging neurotechnologies is fundamental to create appropriate policies and oversight systems that ensure sound dual-use assessments while allowing for continued scientific flourishing. The pilot study we describe in this article aimed to gather firsthand insights into perspectives of neurotechnologists and neuroethicists from a variety backgrounds to explore the risks they saw with the emerging technology and who has responsibility for safely developing it.
Methods
We conducted 8 interviews with senior neurotechnologists from laboratories at established universities in the United States, the United Kingdom, and Australia. Their disciplinary backgrounds included invasive and noninvasive BCIs, neuromorphic computing, and cognitive robotics, with 2 participants working on technology to advance the understanding of intelligence and the brain, 4 working on clinical or therapeutic BCI technology, and 2 working on direct-to-consumer BCI technology—that is, neurotechnology applications that are sold directly to a consumer market for personal use without clinical oversight. We conducted 2 additional interviews with neuroethicists who had conducted extensive research on dual use. Semistructured conversational interviews were carried out using a list of predetermined areas. We asked 4 core questions, broadly covering:
Whether the neurotechnologists felt they ought to monitor dual-use potentiality Where their funding came from and whether it mattered to them Whether they perceived a sense of responsibility for the access and future direction of their work What sorts of unintended consequences they have considered for their own work
The pilot study examined the views of a small sample from particular parts of the neuroscience and technology community, with an intention to generate a theory based on the participants' perceptions that were grounded within the data using a grounded theory approach. 10 Interview questions were open-ended and the responses were then transcribed by the researchers. The research team coded each transcript line-by-line and then used axial coding to develop major categories and bring the data together to develop themes and lay foundations for analysis. 11 Generalizability was limited in this small-scale study due to time and resources. However, the intention of the pilot study was to gather in-depth, rich insight that would facilitate further crucial discussion on a topic that will become more prevalent to society at large. Approval for this study was granted by King's College London Research Ethics Committee (MRS-18/19-11212).
Results
Monitoring Dual-Use Potentiality
Six of the technologists interviewed were aware of the concept of dual use, with 3 stating that misapplication was not a likely possibility for the type of technology they were producing (eg, BCIs, neuromorphic computing, cognitive robotics). Two technologists explained that the technologies they had created in the past had gone on to be used for entirely unexpected purposes that would have been impossible to predict and described themselves as being “away from the front line.” In contrast, 2 technologists stated they had not come across the concept of dual use as part of wider socioethical considerations about their technologies. Impressions gathered included that dual use is always possible, that “reverse” dual use can occur where defense organizations “harvest” commercial neurotechnologies and apply them to suit their own purposes, and it was frequently noted throughout the interviews that the terms “dual use” and “misuse” or “misapplication” should not be intertwined due to their differing connotations. Misuse was understood as being the morally bad use of that technology, whereas misapplication was understood to be the concept of classifying research as “of concern.” The technologists who were aware of dual use described it as “multifaceted” and “fluid” on a case-by-case basis. These variations in understanding and interpreting dual use was corroborated by a neuroethicist's opinion that the dual-use label can work as a “misnomer.”
Funding Sources
Funding was an ongoing consideration for the participants. Most technologists said they did not see commercialization of their work as a priority, but to make their technology more widely accessible, commercialization was often seen as the only solution. Four technologists explained they had spinout companies using similar technology. The pursuit of private capital led 2 of the neurotechnologists we spoke with to move to Silicon Valley in California, a place where, as one participant explained,
We don't even have to explain it. You just say BCI or neurotech and everyone gets it in Silicon Valley. (Interviewee 6)
Six technologists, from each of the 3 countries, had been previous recipients of direct or indirect DARPA funding, including the 2 technologists who were not aware of the concept of dual use, with each citing that military funding had been beneficial, easier to attain than state government grants, and that their specific projects were unlikely to be used directly for military purposes.
The notion of advancing science or technology was raised throughout the interviews, with 3 technologists referencing the idea of a capabilities race being felt both within nations and internationally, citing the term “technological supremacy” as being at the forefront of many researchers' minds. They explained that concerns about “unregulated nations” were emerging, creating a global landscape for neuroscience.
A Sense of Responsibility?
All 8 of the interviewed technologists viewed the notion of responsibility as shared by the relevant stakeholders. They felt responsibility at an individual researcher level should encompass monitoring the technology's efficiency and efficacy and encouraging further consideration of potential alternate applications of a technologist's work. Five of the technologists stated that it was an individual researcher's responsibility to consider dual-use application of their technology, each attributing importance on how possible dual use is revealed, open or closed publication access, and consideration of low-probability but high-impact scenarios. At an institutional level, they highlighted the responsibilities of universities and institutions to ensure adequate training and monitoring of institutional oversight. Multiple participants had been involved in ethics committees, writing chapters and articles, and presenting lectures on researcher responsibility. Two participants stated:
Working with the European Commission always has a 2-edge feel to it. They fund open research and demand you publish it openly and then expect you to support commercial innovation because commercial innovation depends on having some kind of proprietary control of intellectual property. (Interviewee 1) The expectation of openness changes from different funding bodies. I'm a strong believer in open science, and open data is part of that. (Interviewee 2)
Societal involvement was also highlighted as a responsibility to increase awareness of technological advances and to allow for broader perspectives as to whether the technologists continue their scientific work. One participant stated, “I almost wonder if it's like a Pandora's box—once this technology is out there you can't put it back in.” Several technologists raised the possibility of neurotechnology being used for subliminal or direct marketing campaigns, with accessibility to brain data becoming easier with platforms like Facebook, Google, and Elon Musk's emerging BCI company Neuralink, and they said people ought to be warned about this. Another frequently mentioned point was the need for a global perspective with regard to responsibility and access, due to increased research tourism to, and collaborations with, less regulated countries.
Possible Unintended Consequences
Enhancement featured heavily in discussions of unintended consequences. Most participants did not have an issue with enhancement, citing that it was an inevitable progression of the human species, and emphasizing the importance of taking a long view on the matter. Enhanced soldiers were considered an ethical gray area because they would be “superhuman with abilities beyond what nature has provided,” who were not being enhanced for clinical or restorative purposes but for warfare. Enhancement for weaponization purposes increases the potential of harmful applications and prompts an avenue of further ethical investigation. Two technologists mentioned they had been involved in trials focusing on improving performance of pilots using BCI technologies. Impressions gathered about enhancement of the general population tended to perceive restoration positively, with cochlear implants being cited as an example of a positive neurotechnology application, whereas augmentation was seen as making society more uncomfortable. Several participants suggested that a universal baseline is necessary to define enhancement. In contrast, other participants perceived enhancement neurotechnology as an application to be understood as an improvement tool.
When asked about access to such neurotechnology applications, issues of maldistribution both within and between countries, distributive justice, and expanding irregularities in intelligence and class were raised. In regard to possible consequences of using their technology as a form of enhancement, a number of participants described it as still “science fiction” that their technology was “not there yet,” with concerns of long-term effects and risks being unknown, and that ultimately it was impossible to be technological and economical “fortune tellers.”
Discussion
Dual Use As Multifaceted
One of the central findings of our study was that the term “dual use” is multifaceted. The concept of dual use was known to the majority of participants, even if they did not consider it to directly apply to their own work. The notion of being “away from the front line” may work as a researcher shield or deflection from having to consider possible dual-use implications too deeply. Indeed, other studies suggest neuroscientists are more likely to consider their colleagues' work as having more potential dual-use risk than their own work. 12 Our study also demonstrated that although DARPA indirectly funded more than half of the participants' work, the participants did not believe their scientific work was being applied for military application. However, they recognized that their research could possibly be later applied for military purposes. Minimal practical guidance is available to assess the risks of neurotechnology applications being used for harm, or to determine the potential contribution of these applications to military programs.
Of additional significance was that the 2 technologists producing direct-to-consumer neurotechnology had not encountered the concept of dual use, which may reflect a broader lack of awareness among technologists producing direct-to-consumer neurotechnologies. Although opinions from 2 participants are not generalizable, they may suggest the degree of socioethical understanding and consideration of those involved in the commercial enterprise of neuroscience.
Frequent requests for clarification on the definition of dual use throughout the interviews suggest the term is not clear to researchers. Mahfoud et al 7 argue the term “dual use” should be reconsidered and that failure to move away from this dualistic framework could result in an inability to govern its circulation, particularly with regard to those who fund, develop, and regulate research in neurotechnology and neurorobotics. Our study's findings complement this recommendation, with the binary aspects of dual use not being perceived by our participants as cohesive to their neurotechnological research.
Commercialization of Neurotechnology
In 2019, the neurotechnology market was estimated to generate over US$12 billion in annual sales by 2021. 13 To achieve commercial gains, companies specifically target technological products that are attractive for military, security, and defense contacts, with many of the products focusing on communication technologies, robotics, and artificial intelligence. 5 Venture capitalists have increasingly been investing in brain science, with angel investors in Silicon Valley contributing vast amounts of funding to biotechnology startups. Prominent technology investors such as Elon Musk and his company Neuralink, 14 or technology companies crossing over between academic research and commercial innovation such as DeepMind 15 or Bitbrain, 16 suggest a change in the future of neuroscience funding. Such a change may promote benefits like wider access to neurotechnology, but as the study participants noted, harms may increase through the marketization of personal brain data and private companies becoming privileged beneficiaries and gatekeepers of vast amounts of brain data. These possibilities create the potential for malevolent application of neuroscience and neurotechnology in civilian, political, security, and commercial applications. Certain neurotechnology such as direct-to-consumer BCI wearables, which are not being sold for clinical benefit, have the capacity to be made readily available on the market, without rigid ethical approval requirements. While dual use has traditionally been defined as scientific findings that can be used for both military and civilian applications,17,18 the commercialization of neurotechnology opens up a less distinct, more hybrid form of dual use. Study participants noted that the direct-to-consumer neurotechnology wearable marketplace is growing, with cheap and operational tools being used to enhance cognitive performance. They also noted an increasing number of do-it-yourself “neurohackers” who attempt to optimize their cognitive capacities with the consumer kits being developed.19-21 As the applications for neurotechnology broaden, the potential for dual use increases with malicious biohacking, dangerous uses of medical neuromodulation, and neuroimaging-based intelligence interrogations all becoming possible. 22 A confirmed case of a malicious attack has yet to be reported, but the feasibility of obtaining private information without authorization from users of electroencephalogram-based BCIs has been demonstrated. 23 While the US Food and Drug Administration has declared direct-to-consumer BCIs “low risk,” concerns about possible harms have been raised1,24 and calls are emerging for responsible innovation and better engagement for direct-to-consumer BCI technologies.25,26 A starting point to address these concerns may include opening up a dialogue with technologists, investors, and ethicists.
Globalized Neuroscience Research Landscape
The acceleration of global advances in neuroscience was interpreted differently between the technologists and the neuroethicists. The former described it as “competitive” and a “capabilities race,” whereas the latter described it as a “collaborative” form of research tourism. This discrepancy is significant as it provides an insight into the participants' distinct views. If neurotechnologists perceive their field as competitive and as a race to achieve further understanding of the brain, there may be little room left for socioethical consideration of future implications. The variance of socioethical regulations poses a risk of what a neuroethicist described as “ethical dumping,” in which it is assumed all nations will adhere to Western values and moral standards. Without an overarching international set of responsible researcher guidelines for neuroscience and neurotechnology, a space for unregulated research tourism remains. This was illustrated by 2 participants who had moved from Australia to the United States specifically to accelerate their research. The InterAcademy Partnership views researchers as a global community and recommends they participate in discussions about possible consequences generated from their work, recognizing the intricacies of open-access research and stricter government oversight as ongoing challenges. 27
Key potential harms raised by participants that they saw stemming from an increasingly globalized neuroscience landscape were the inequality and maldistribution of neuroresearch and neurotechnologies between and within countries. As state and nonstate actors form neuroscience research teams, without an overarching international set of responsible research guidelines for neuroscience, a lucrative research tourism market will be created. Some saw this as inevitable:
Disparity in health delivery across the world is already a major problem. The wealthy versus the poor countries. I don't have a solution to that, but I would say you can't halt the development of technology because it's not going to be equally distributed just because people won't be able to afford it. (Interviewee 7)
In other words, wealthy countries are able to remain rich by using biotechnology and government procurement of defense technologies, which contribute to competition between wealthy countries by securing a dominant share of new markets. 28 Other participants felt that active efforts should be made to redress the situation. One of the neuroethicists voiced concern about intellectual property laws, whereby the powers of ownership and control depend on the country and the researcher within it. Variations of intellectual property laws between regions can mean that a therapeutic device might be patented, but novel treatment methods cannot be patented. If a state owns everything generated within its boundaries, this may be contrary to the perceived competition between states that the participants felt and might behold a conflict of interest for technologists who seek funding from countries outside of their own. Development of a “neuroeconomy” may embolden inflated claims of successful neuroscience research and rush products to market. 29 A neuroeconomy encourages commercial companies that significantly fund universities, departments, or research centers to gain priority patent rights, an increased tendency to inflate research and capitalize on scientific knowledge. 29
Conclusion
With a growing number of experts now being able to use technologies to envision the possibilities and potential problems emerging from neuroscience, authorities have an obligation to govern both the present and the future of brain research. 29 Fears of creating a technocentric Pandora's box were voiced by participants, but whether this box has already been opened is subjective and depends on the perceived impacts that neurotechnology has already had on society. An example is Elon Musk's previously mentioned Neuralink, a startup company that aims to form “a symbiosis between [artificial intelligence] and the human brain.” 30 Neuralink aims to enable humans with spinal cord paralysis to control phones or computers. Assuming Musk's promissory claims translate into actual capacity to deliver, this neurotechnology may be perceived as a friendly application, if not therapeutic, and has generated much public interest. However, it also holds great dual-use potential, including possibilities for neurohacking or thought-hijacking at a user level, as well as privacy intrusions from state and nonstate actors via smartphone, computer, or cloud access or vice versa, and even possible manipulation from the artificial intelligence systems operating the brain chip.
While transparency and openness in scientific research have been promoted for a long time (eg, British Royal Society for Social Responsibility in Science in 1969-1991 and, more recently, the European Union's Responsible Research and Innovation framework 31 ), the ability to control emerging technology and its possible unintended effects remains challenging. Moreover, open science should not be considered a potential silver bullet because it can also lead to potential unforeseen dual-use applications. This was recently demonstrated by the use of reinforcement learning (pioneered by DeepMind) 32 in the DARPA's AlphaDogfight trials, where a US company used it to create an algorithm that defeated a human F-16 pilot in a simulation.33,34
The perceptions gathered in our pilot study indicate that most, though not all, technologists were aware of dual use, but they viewed it as multifaceted and fluid, with the commercialization and globalization of the emerging science shifting understandings of what is beneficial vs harmful neurotechnology, rendering future uses unknowable.
Study participants were acutely aware that what they intend for an application and how that technology is eventually applied can be 2 very different things. They did not perceive dual use as binary. Consideration of potential future harms and benefits should be thought of as a continuous process, not a single event, that encompasses informed involvement and distributed responsibility across multiple stakeholders at a global level. All stakeholders involved in scientific research should be accountable and actively promote inclusive deliberation and reflexivity, not just the individual researcher. Scientific institutions, funders, and publishers all have roles to play in managing the uncertainties and risks of harm from emerging technologies.
This study also highlights that stakeholders need better tools to assess the practical and socioethical implications of neurotechnological innovations and their possible future applications. Agreement on what is perceived as beneficial or harmful will need to be actively reached on a case-by-case basis, and the consequences of misestimating outcomes and impacts should be considered as part of the equation.
Given the small sample size of this study, further investigation into the perceptions of neurotechnologists is needed to formulate an explanatory theory. A future study would include a cross-comparison study of technologist's perceptions on dual use and ethical responsibility from the United Kingdom and the United States, particularly in regard to the different funding bodies available and the differing ethical guidelines between the Human Brain Project and the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative.
Footnotes
Acknowledgments
We express our thanks to the experts who gave their time to participate in this pilot study, and to the anonymous reviewers and editors for their constructive feedback on our work.