Anticipating and addressing the ethical implications of deepfakes in the context of elections

Deception

Deception is the most dominant harm that cuts across the scenarios. ¹ Many of the deepfakes depicted in our eight scenarios are used to induce voters to believe something that is not true about a candidate. For example, in scenario 2, the deepfake falsely documents a candidate saying disparaging and hateful things, and in scenario 3, a video depicts testimonials falsely attesting to seeing a candidate engage in unseemly behavior. However, deception need not be only about the candidate. Scenario 4, for instance, involves deception about polling locations conveyed by a trusted leader, and in scenario 6, a compromising deepfake targeting an individual voter involves the threat of deception about that voter’s behavior to friends, family, and the public.

Importantly, in most of the scenarios, the deception is unambiguously intentional. This is because producing a deepfake is an intentional act. The role of intent in deception is important here because it grounds the distinction between misinformation (no intention to deceive) and disinformation (intention to deceive) (Wardle and Derakhshan, 2017). The deepfakes used in scenarios 2 through 7 involve disinformation, since they were intentionally produced to deceive voters or voters’ family and friends (as in scenario 6). In scenario 1, the authors of the synthesized depiction of a battle scene might have believed that they were accurately recreating what happened, so the synthesized video cannot simply be considered disinformation. Moreover, insofar as the video is an accurate recreation, it may not qualify as misinformation. The problem with the video in scenario 1 is that viewers were not informed that some of the footage was synthesized so they might presume the video consisted entirely of real footage. Viewers may have a different stance toward depictions once they are recognized as interpreted representations. Intent also plays a role when it comes to deepfake parodies, as illustrated in scenario 8. Parodies are not intended to deceive; those who create them expect viewers to share the joke (Tandoc et al., 2018). In scenario 8, however, deception occurs because a deepfake parody is moved from its original context to another context, that is, social media platforms, where there is no indication that the video is a parody. Consequently, some viewers take what they see to be real. In this situation, whether or not the deepfake could be considered misinformation or disinformation would depend on the intentions of the person who moved the deepfake into a new context without labeling. If the sharer did not intend deception, but a viewer was nonetheless deceived this would be misinformation, whereas if the sharer intended viewers to be deceived, this would be an instance of disinformation.

Deception is a profound harm to individuals because it impedes their ability to make informed decisions in their own best interests. Intentionally distributing false information about a candidate in an election manipulates voters into serving the interests of the deceiver. False information about candidates may distort a voter’s reasoning about how to vote. Even if a person would have chosen the candidate regardless of the deception, the voter’s autonomy has been impeded in the sense that the voter has been disabled from making a decision on the basis of accurate information. Although this is true for any false information distributed during an election, deepfakes are unusual insofar as they increase the ostensible authenticity and credence of their misrepresentations.

A single act of deception may be harmful to a single individual, but when deepfakes are widely distributed, harm is amplified, that is, many more voters are deceived. Obviously, the difference between one-person to one-person deception and amplified deception is the number of people harmed. Both the breadth and speed of amplification facilitated by digital and social channels make it more difficult to counteract the harm.

Intimidation

To intimidate is to make a person fearful (of injury or harm); intimidation involves a threat that generates fear. The use of deepfakes to intimidate people has already emerged as an ethical issue related to deepfake pornography (Cook, 2019). Indeed, as mentioned earlier, deepfakes initially gained notoriety for their use in creating pornography. Previous research has found that they are overwhelmingly gendered and sexualized in their current usage, and that much of the real harm already done by the technology has been directed toward women (Ajder et al., 2019; Paris and Donovan, 2019).

Scenario 6 illustrates how in the election context deepfakes can be used to threaten targeted voters into not voting. The threat is to release a defamatory pornographic deepfake of the voter if they vote. Here, the intent is to instill fear of the possibility that family, friends, and others will see and believe the deepfake. Of course, even if those who were to see the distributed deepfake would not believe it, the voter would still likely experience a loss of dignity and a feeling of humiliation or shame. The more broadly the deepfake spread, the more humiliation. Intimidation is harmful in itself and in the election context, if the intimidation works, it deprives voters of a fundamental right, the right to vote.

Amplification is limited in scenario 6 both because the distribution medium is text messaging and because the deepfakes are personalized. However, the technical capability to personalize deepfakes may become easier in the future, and this could lead to a different kind of amplification. That is, in addition to amplification by means of a single deepfake being spread broadly through public or social media (i.e. one-to-many amplification), the ease of personalized deepfakes could lead to a profusion of personalized deepfakes spread through more narrow non-public (e.g. chat) channels (i.e. one-to-few amplification repeated many times).

Reputational harm

In scenario 6, deepfakes are used to intimidate voters by threatening harm to their reputations. In four of the other scenarios, reputational harm is also at issue but the harm is to candidates. In scenarios 2 and 5, videos depict a candidate saying something that suggests the candidate is not who she or he purports to be; in scenario 3, people testify that a candidate behaved in a way that makes the candidate seem hypocritical and an unsavory person; and in scenario 7, a fabricated incident suggests that a candidate cheated in a public debate. In these scenarios, the subject of the deepfake is not threatened with release of an unattractive video, the video or audio is simply released. The harm is not a threat, the harm is done through the act of publication.

Legally, deepfakes of this kind could fall under the concept of defamation (the act of damaging a person’s good reputation); however, the courts have been reluctant to interfere with campaign speech. Courts in the United States have been divided on the constitutionality of laws that prohibit false campaign speech (Hasen, 2013; Marshall, 2004). This has rendered the law with regard to campaign speech extremely complex with the result that false speech is generally not disallowed. Yet, reputation is, in an important sense, the commodity of campaigns; campaigns are interested in shaping beliefs about who a candidate is, what the candidate has and has not done, what kind of character the candidate has, and what the candidate is likely to do in the future. Competitors are interested in countering their rival’s positive reputation. The reluctance of US law to ban false campaign speech has more to do with the dangers and challenges of trying to regulate false claims than denying the harmfulness of false speech. Since our concern here is with the ethical rather than legal implications of deepfakes, the harmfulness of false speech as constituted in deepfakes is incontrovertible.

The amplified consequences of reputational harm result both from the large number of people reached and the speed with which the damage is done, making it difficult to counteract. Deepfakes that are released in the final weeks of an election could be nearly impossible to effectively counter, while deepfakes released early in an election might still be difficult to counteract but at least there would be time to publicize their falsity.

Misattribution

Producing a deepfake involves using a subject’s image and/or voice either by manipulating available renderings or by synthesizing entirely new ones. Because of this, deepfakes seem to violate ownership rights of the subjects. That is, independent of the reputational harm that might be done to an individual using their image or likeness in a deepfake, there seems to be an additional harm in using and distributing the individual’s image or likeness without permission. Because the law in this area is complex and varies across jurisdictions (Rothman, 2018), it does not provide an incisive approach to this harm. Public figures have publicity rights in their images, but this right is based on a commercial interest, that is, the public figure is protected to be able to reap economic benefits from their image (Berkman, 1976). Although candidates for public office are public figures and therefore have publicity rights in their image, it is unclear how a legal suit claiming harm from the use of a candidate’s image in a deepfake would fare in an election context where the harm is not commercial or even pecuniary (Spivak, 2019), and, as already mentioned, courts are reluctant to regulate false campaign speech.

What, then, is the additional harm—beyond reputational harm—to using another person’s image or likeness (without their permission) in a deepfake? The harm can, perhaps, best be understood as “persona plagiarism”—an inversion of plagiarism focused on the source rather than the content of a message. Plagiarism is the act of “using or closely imitating the language and thoughts of another author without authorization and the representation of that author’s work as one’s own.” ² Legal scholars have attempted to define it simply as “non-consensual fraudulent copying” (Posner, 2007). In plagiarism, A presents B’s words and takes credit for them, claiming the words to be her own. Plagiarism is misattribution; B has created a text and A misattributes the text to herself. In the act of producing a deepfake, A manipulates B’s image and/or voice to create something new and then misattributes it to B. In short, in plagiarism, A takes B’s words and misattributes them to A; in deepfakes, A takes B’s persona (likeness and voice), inserts A’s own expression according to A’s goals, and misattributes that expression to B. This can also be thought of in terms of credit. In plagiarism, the original author of words is not given credit for what he or she created; in deepfakes, credit is not given to the creator of the deepfake and is falsely given to the subject of the deepfake. In plagiarism, credit has a positive valence and the real author is denied that. In persona plagiarism, credit with a positive or negative valence is misattributed to the subject without their consent. The typical punishment for plagiarism in society involves social sanctions against the plagiarist once their copying is detected (Posner, 2007); however, in the case of deepfakes, the lack of attribution to the creator of the deepfake frustrates a comparable normative sanctioning against that creator.

Misattribution is also at issue with parodies, though in a complicated way. The deepfake parody creator takes a candidate’s likeness and voice and inserts his or her own expression into it by attributing exaggerated, unattractive or embarrassing characteristics or behavior to the candidate. Although there is a lack of consent, there is no deceptive attribution of credit, and therefore, there is no persona plagiarism. In other words, as long as viewers understand they are viewing parody there has been no misattribution. However, when the deepfake is moved to another context in which viewers do not recognize it as a parody and believe the attributed characteristics and behavior to be real, the misattribution becomes a form of persona plagiarism. Of course, identifying who committed the plagiarism is problematic because it was not the intent of the parody creator to deceive, and those who moved it to another context may or may not have understood the implications of what they were doing.

To understand the significance of amplification of misattribution, the parallel with plagiarism is helpful again. When a person plagiarizes another’s work in a single document only seen by one or a few people (e.g. a student turning in a plagiarized paper to a teacher), the harm is already done. But when a person plagiarizes a document that is read by thousands (e.g. in a widely read newspaper), the harm is greater because the number of people led to misattribute credit is greater. It is the same with deepfakes. A deepfake depicting a candidate doing something he or she did not do shared in a non-public channel with a few individuals is not as harmful as a deepfake spread widely across social media thereby creating much more misattributed negative credit.

Undermining trust in elections

In their broad discussion of deepfakes, Chesney and Citron (2019) list nine different harms to society that could result from deepfakes, including election manipulation. They also mention distortion of democratic discourse and eroding trust in institutions, both of which are highly relevant to democratic elections. Insofar as deepfakes are a form of disinformation, the National Democratic Institute’s (NDI) statement on Disinformation and Electoral Integrity unambiguously articulates the threat: “Deliberately blurred lines between truth and fiction amplify voter confusion and devalue fact-based political debate…Manipulation of voter and civic information dampens participation and degrades trust in election management bodies.” ³

As already discussed above, the law with regard to campaign speech is not very helpful in addressing the threat of deepfakes because election law is shaped by a compelling concern for the protection of first amendment rights. This is especially so when it comes to deepfake parodies for they are generally seen as a form of free expression. The law favors false campaign speech over violations of free speech for fear that regulating campaign speech would become political (Marshall, 2004; Rowbottom, 2012). Despite the law, however, the harm that deepfakes cause in undermining trust in electoral outcomes stands. Deepfakes along with other kinds of false speech distort campaign results and threaten public trust in those results.

Scenarios 6 and 7 illustrate specific ways in which deepfakes can be used to manipulate voters and dampen participation which in turn generates mistrust in the integrity of electoral processes. In scenario 6, deepfakes are used to deter voters from voting by means of a threat to release deepfaked pornographic images. Even if it was unclear how many people had been deterred, once voters become aware of the tactic, trust in the integrity of election results may be eroded (Daniels, 2009). In scenario 7, a deepfake is used to instill mistrust by falsely suggesting that a candidate cheated in a public debate, thereby calling into question the legitimacy of the political process.

Although these two scenarios illustrate specific ways in which deepfakes can be used to thwart electoral processes, it is also important to note that all of the harms exposed in the other scenarios can also lead voters to mistrust the processes by which officials are elected. For instance, deception undermines individual decision autonomy and when discovered may undermine the trust a voter has in her fellow voters to make well-informed decisions. Similarly, reputational harm and misattribution distort how voters perceive and understand candidates and, even if an individual viewer is aware of the manipulations, he or she may believe that others are not, which could further degrade their trust in the ability of others to make well-informed voting decisions. With new uncertainties injected into the question of whether voters think their peers are well-informed, trust in democratic processes is undermined.

Education and media literacy

Education and media literacy training have a role to play among the strategies necessary to address deepfakes (Witness & First Draft, 2018). The idea here is that individuals could be encouraged to develop an awareness of the capabilities of the technology so as to be able to spot characteristic flaws in deepfakes, or more generally acquire skills in how to verify, fact check, and do research to assess online sources of information. Such increased literacy would allow individuals to take more responsibility for their consumption and sharing of information (synthesized or not). Even if it does little to address the sources and motivations for the production of deepfakes, by equipping viewers with critical assessment abilities this strategy is compelling because it directly mitigates the potential for deception and furthermore diminishes reputational harm by reducing amplification via social channels.

The challenge of educating the public and promoting media literacy is daunting. However, from the perspective of forward-looking responsibility, there are many actors that could facilitate this including news organizations, media platforms, political campaigns, policy makers, technologists, and other education-oriented organizations. Journalists, for instance, can contribute by raising awareness of the capabilities of deepfake technology and the ethical harms they inflict. Platforms can develop media literacy modules that are interjected into users’ experience at opportune moments. Campaigns can contribute by talking about the issues of deception and synthetic media. Policy makers can create or fund programs which help educate the public. And technologists can help educate the public as well as other specialized stakeholders on what to look for to determine if a piece of media is synthesized.

Of course, there may be limits to the effectiveness of this strategy because deepfake technologies are becoming increasingly more realistic. For instance, when it comes to out of context parodies, it may be difficult to train individuals to uniformly identify what might be considered comically absurd. Moreover, an unintended consequence of increased public education about deepfakes is the possibility of what Chesney and Citron (2019) have termed the “liar’s dividend.” This is the idea that as individuals learn to be more critical and skeptical of media, they may begin to doubt real video and audio evidence. This in turn makes it easier for some to deny and cast doubt on the occurrence of real events (Golingai, 2019). In short, while education serves to mitigate deception directly, it could also generally depress trust in social institutions such as democratic elections. Hence, educational interventions will need to be paired with other interventions discussed below.

Subject defense

The harms to the subjects of deepfakes suggest a suite of interventions enabling subject defense, either reactively or proactively. These measures apply to primary as well as auxiliary subjects.

In the election context, campaigns could adopt a strategy of monitoring platforms where deepfakes circulate, so that any manipulated representations of the candidate can be quickly debunked. In addition to detection, campaigns could also create crisis response teams that have plans in place for how to react quickly should a deepfake of the candidate be used to attack the candidate’s reputation. The plans could include legal response strategies for harms specifically relating to defamation, false light, or right of publicity (Chesney and Citron, 2019; Spivak, 2019).

Of course, each of these reactive approaches has limitations. Not all media platforms can be monitored because deepfakes can circulate in private personal messaging systems, groups, and chat apps that have even been encrypted. The effectiveness of legal responses will be limited because correctly identifying the legally accountable source of the deepfake, and/or reaching perpetrators outside the jurisdiction of US legal processes is challenging (Chesney and Citron, 2019). And, of course, if any of the identified deepfakes have been created as parody, they may be protected as free speech. The problem with reactive defense interventions in general is that they do not stop the deepfake from being seen; they either counter the disinformation with accurate information or hinder its spread. Many of the harms are still done as in the case of intimidation and undermining of trust; others are only lessened as with deception and misattribution because all viewers cannot be reached. Reactive strategies also cannot operate at a speed commensurate with the spread of media on many platforms.

In terms of proactive defense, options are more limited. Since deepfakes use audiovisual material as training data, campaigns could try to restrict the audiovisual material of a candidate in a way that would prevent the creation of deepfakes. However, this would work against freedom of expression, would limit a campaign’s capacity to reach voters and, in any case, the technology is moving too quickly toward realistic synthesis with only small amounts of training data. Another proactive strategy would be for candidates to employ immutable life logs—verifiable recordings documenting their every utterance—which could be used to authoritatively debunk any potential deepfake (Chesney and Citron, 2019). An even more futuristic scenario would entail candidates using facial or vocal digital implants that imprint a verifiable encrypted signal in any audiovisual recording of the individual. Audiovisual recordings that did not contain the watermark signal could then be identified as fakes. The advantage of these proactive measures is that they address harms both to subjects and to viewers. But at the same time, proposals such as immutable life logs or implants create new harm to subjects by requiring them to relinquish some degree of privacy, that is, with respect to bodily intrusions from implants or as a result of tracking their behavior with life logs.

Although subject defense primarily assigns responsibility to subjects, other actors are in a position to help. Policy makers can craft legal protections that buttress legal actions by campaigns while being careful not to chill free expression (Hasen, 2019). Policy makers might also create and fund programs that support technologists in developing technologies to support proactive defenses.

Although campaigns may be motivated to monitor for deepfakes of a candidate, their ability to do so will be constrained by limited access to data from the major platforms. Platforms are in a better position to enable effective and timely monitoring of candidate personas, which provides a strong argument for ascribing responsibility to them. Their responsibility might take the form of having individuals enroll with the platform (e.g. through the disclosure of a photo, video, or voice sample) to be alerted when synthesized media of the individual is detected. The role of the enrolled individual in this arrangement would then be to monitor, evaluate, and react to alerts from the platform. Alternatively, an intermediary monitoring entity might bring together these alerts across different platforms as well as monitor the broader Internet for deepfakes of the enrolled individual.

Verification

A potentially powerful antidote to deepfakes is the use of verification techniques and technologies that help reveal how the audio and/or video was constructed. These provide evidence about whether a piece of media was synthesized (entirely or in part), and include automated algorithms as well as semi-automated forensics procedures for detection and provenance determination. Verification techniques increase the likelihood that fakes will be identified as such and, therefore, minimize harm to viewers. Verification can also provide support to other intervention strategies. For instance, it can counteract erosion of trust and the liar’s dividend by supporting established and rigorous verification procedures, such as those practiced by professional news media and fact checking organizations, augment the efficacy of subject defense strategies by helping identify misattribution, and work in the service of legal interventions by tracing provenance and helping attribute media to an accountable entity.

Despite recent advances in automated deepfake detection and verification algorithms, for example, reaching as high as 92% to 98% accuracy when applied to synthesized faces of leading politicians (Agarwal et al., 2019), these strategies are nonetheless limited in scope. For instance, they can have degraded performance with reduced encoding quality and exhibit statistical uncertainties that confound clear conclusions (Johnston and Elyan, 2019; Rössler et al., 2019). According to Farid (2019), “No technique can prove authenticity…authentication is simply a failure to find evidence of tampering.”

Yet another problem with automated verification is that it can introduce new harms having to do with curtailment of free speech. Automated techniques for detecting and verifying media at scale introduce probabilistic uncertainties (Ananny, 2019), which means some false positives, that is, some media will be detected as fake when it is not. Since automated techniques are unable to detect the intent of a deepfake’s author, parody and satire may be weeded out along with deceptive content. Hence, although verification techniques can help reduce some of the harms of deepfakes, they will need to be paired with expert human moderators capable of assessing intent and appropriateness (Paris and Donovan, 2019). In general, verification interventions will benefit from humans in-the-loop to interpret forensics evidence, reach sound conclusions, and flexibly cope with an ever evolving range of deceptive practices and deepfake technologies.

Publishers of media will need to take some responsibility for media verification, be the publisher a social media platform, a phone chat app, or a news organization. This is not a matter of blameworthiness but rather because publishers are in the best position to stop the negative effects of deepfakes. Any harmful effects of deepfakes can be largely subverted if they are simply not published. Nevertheless, publishers need not bear this responsibility for verification on their own. Their effectiveness can be supported by campaigns monitoring media for deepfakes of their candidates, news media investing in debunking and fact checking initiatives, platforms limiting the spread of deepfakes, and individuals becoming media literate. Technologists, especially those who are building, and therefore most familiar with, media synthesis techniques, can take responsibility for developing new and better automated detection algorithms and semi-automated verification tools that are understandable and easy to use by these various stakeholders (Partnership on AI, 2020). Policy makers can contribute by funding research and development on such forensics techniques.

Publicity modulation

With the advent of the Internet and social media, everyone is a publisher with the potential to add publicity (i.e. to amplify public attention) to a piece of media via any number of digital channels. As noted previously, publicity is a key driver of both subject and viewer ethical harms. A potential intervention strategy is, therefore, to modulate the publicity that deepfakes receive. An extreme version of this would be to ban all deepfakes. A softer approach would be to throttle the degree of publicity a deepfake can receive by strategically moderating a deepfake’s amplification so as to diminish its potential harm (Donovan and boyd, 2019). Publicity can also be affected through counterspeech, including disclosure notices or labels (e.g. indicating nature and veracity) and separate debunks.

Whether the intervention is banning, throttling, or counterspeech, the question arises as to whether paid publicity, such as advertising using deepfakes, should be treated differently. This in turn points to a broader question: Should all deepfakes have their publicity modulated, or only those that meet particular criteria, such as those that are unlabeled or which have been debunked? Answering these questions to arrive at a publicity policy for deepfakes involves two non-trivial steps: defining what types of deepfakes should be subject to moderated publicity and then identifying which pieces of media fit that definition. If automated detection algorithms are not able to accurately identify media according to the definitions, this intervention will require significant human labor and resources.

The three options, banning, throttling, and counterspeech, each address ethical harms somewhat differently. Banning would be problematic because it would threaten free speech. It would, among other things, stifle satire, parody, and entertainment. In comparison, counterspeech would address potential harm to viewers by facilitating the correction of deception, though it has the problem that it still allows individuals to view the original deepfake, thus, risks having influence and causing reputational damage and misattribution, albeit with a smaller number of individuals. Counterspeech could be used at the point of distribution of deepfakes through labeling, or it could involve debunks that are distributed afterwards, though the efficacy of these different approaches vary in their ability to prevent deception. Counterspeech privileges individual autonomy for the viewer in choosing to heed it (or not), but can impinge on the autonomy of creators and distributors in their free speech when it is compelled (Volokh, 2018). Finally, throttling the amplification of a deepfake would address harms to subjects (i.e. reducing breadth of reputational damage), as well as to viewers (i.e. diminishing opportunities for deception). Throttling would minimally affect free speech, since creators are still entitled to make and post deepfakes. Each intervention would have to also be considered in light of the temporal dynamics of publicity. For instance, counterspeech may be appropriate in situations where time is less critical, such as in the earlier phases of an election, when there is adequate time for debunks to circulate and expose individuals to corrective information. On the contrary, throttling may be more attuned to situations of high time pressure right before an election because it directly addresses harms to subjects and viewers and does not rely on the publicity of counterspeech subject to complex algorithmically mediated distribution environments (Napoli, 2019).

As with the other interventions discussed, several types of stakeholders can be assigned responsibility for modulating the publicity of deepfakes. During election periods, social media platforms might configure distribution algorithms to throttle the spread of synthetic media intended to deceive. News media might limit publicity of deepfakes and only publish those that are being clearly debunked. Individuals could curtail their amplification of deepfakes and campaigns could pledge not to amplify them (or create them for that matter). ⁴ Policy makers, such as those in California, have already begun to regulate the specifics of publishing and labeling deceptive audiovisual media of candidates during election periods. The forward-looking responsibility for modulating publicity could be shared, but would be most effective if the publishers that produce the largest amount of publicity had the largest responsibility, for example, the social media platforms and search engines that drive the majority of online attention.